NEEDLEWORK

Can NEEDLEWORK generate IP addresses for different segments?

Yes, It can.
(NEEDLEWORK generates the IP address described in the scenario.)
Also, you don't need any equipments to split segments between NEEDLEWORK and FW.

like as follows:

Can I set the tag-based VLAN on each port?

Yes. all ports（3ports）can be tagged.
You can specify the source and destination VLAN IDs in the test scenario.
You don't need to set the tag-based VLAN to the port in advance.

NEEDLEWORK has only 3 ports for testing. Is it able to test a device with over 4 ports?

It has 3 ports physically. Using L2 switch, you can test devices with more than 4 ports at the same time also.
The configuration example is as follows:

An example of simultaneous testing that uses all 4 ports of the target device using 2 ports of NEEDLEWORK.
Use tag-based VLANs between NEEDLEWORK and L2 switch to accommodate multiple segments in one port.

Does NEEDLEWORK support PPPoE?

Yes, it does.

Can tag-based VLANs be used on interfaces running PPPoE?

No, it's not available.

Do I have to write the test scenario myself?

Yes, you have to write it yourself based on the communication requirements.

We releasing an auxiliary tool that automatically generates scenarios from some firewall’s configs or logs.
※Please use it for the replacing test.

This is an auxiliary tool(OSS). So please understand that support will be best-effort, also does not guarantee it works well always.
About the tool

How does NEEDLEWORK solve ARP?

ARP requests from NEEDLEWORK

Typically, ARP requests are sent from the device's IP address. But NEEDLEWORK does not have the device's own IP address. Thus ARP requests are sent as following ways:

①Send ARP requests from all NEEDLEWORK ports using the FW IP address described in the test scenario or IP address ± 1 to the value of the 4th octet of the next hop IP as the source.
　※ Since it does not have segment information, above two IP addresses are used as source IPs.

②Receives ARP responses from the FW(test object), and NEEDLEWORK learns the MAC address of the FW’s IP address.
　※ Send packets from the port where received ARP responses.
Please refer to here for more information on ARP request specifications.

ARP replies from NEEDLEWORK

Please refer to "6.6 ARP settings" in the Operating Manual

I forgot the IP address for the remote connection.

The MGT port(eth3) has a fixed IP address of 192.0.2.1/24.

Connect the operating terminal and the MGT port of the needlework directly with a network cable, management access is available in 192.0.2.1.
※Please set any IP address within 192.0.2.0/24 range on the operating terminal.

Which port numbers are used to between the operating terminal and the NEEDLEWORK communicate with?

NEEDLEWORK uses the following port numbers.
If it was filtered through firewall, please allow following ports between the operating terminal and the NEEDLEWORK.

TCP 8080
TCP 8081
TCP 8082
TCP 8084
TCP 8085
TCP 8088
TCP 8087

I'd like to know about network devices that have been tested.

Network devices our company have tested are as follows.

○：Equipment with UTM functions test results (URL filtering, anti-virus)
*：Equipment with session test results
□：Devices with SSL inspection test results
△：Equipment with FQDN specifying test results
☆：Equipment with multi-PPPoE function test results

What are the rules for NEEDLEWORK to generate MAC addresses?

NEEDLE WORK generate MAC addresses according to the following rules.

The first byte is 0x20, which indicates the local MAC address, and the target IP address (※) is set in hexadecimal in the last 4 bytes.

Example) In case of 192.168.1.1, it will be "02: 00: c0: a8: 01: 01".
                    Decimal   ： 192 | 168 | 1 | 1
                    Binary    ： 11000000 | 10101000 | 00000001 | 00000001
                    Hexadecimal   ： c0 | a8 | 01 | 01
                  

How to write a test scenario in a NAT environment?

Each test function (policy, network, throughput, session) has a different scenario description method. Please refer to this document for the description method for each function.

Is NEEDLEWORK only compatible with the network devices listed in the test results?

Since NEEDLEWORK generates and tests L3 or L4 level (TCP / IP) communications, so common firewalls can be tested using NEEDLEWORK. (Some use L7 level communication)
We rent out evaluation machine. Please check if you can test with the target model.

Does NEEDLEWORK support application level testing?

It supports testing of the following applications.

• HTTP
• HTTPS
• DNS（TCP/UDP）
• FTP（Active/Passive）
• IMAP

Is there a limit to the number of IP addresses and segments that can be generated?

There is no limit.
Please describe the combination of source and destination IP addresses on a one-to-one basis on one line of the test scenario. Then NEEDLEWORK processes each line from the top of the scenario. Therefore, it doesn't matter how many IP addresses you have in the entire scenario.

Can I use NEEDLEWORK to test ACLs(Access Lists) of routers, etc?

Yes you can. NEEDLEWORK makes judgments for each protocol based on the following criterias so it can be tested if the ACL meets those.

• ICMP：Confirmation by ICMP Echo-Request / Reply
  If there is a reply, it is considered as communication is permitted.
• TCP：Confirmation by 3-way handshake + FIN
  Establishment of 3-way handshake and communication be terminated by FIN, it is considered as communication is permitted.
• UDP：Confirmation by round trip of UDP packet
  If there is a UDP response packet, it is considered as communication is permitted.

How long does it take to run the test?

Although there are differences depending on the environment, it can be tested in the following time.
※Drop time can be changed by adjusting the parameters.

Is it possible to test even if there are multiple gateways for the FW?

Yes. It is possible.

How many lines of test scenarios (CSV) can I import?

There is no upper limit on the number of lines. In addition, we have confirmed the operation in our development environment with 10,000 lines.

Is it possible to test the FW in transparent mode(L2 configuration)?

You can test in the following ways.

• Specify the value of the test scenario as follows to test.
• Source FW IP : IP address listed in the destination IP address field.
• Source interface : Interface number of NEEDLEWORK connected to the source side.
• Destination FW IP: IP address listed in the source IP address field.
• Destination interface : Interface number of NEEDLEWORK connected to the destination side.

Constitution Example

• Scenario description example
• Source IP ：192.168.1.100
• Source FW IP : 192.168.1.200
• Source interface : 0 ※eth0
  
  
• Destination IP：192.168.1.200
• Destination FW IP : 192.168.1.100
• Destination interface : 1 ※eth1

Is it possible to test the UTM functions?

Yes, you can test the following UTM functions:

• Anti-virus function
• Send packets with test virus.
• URL filtering function
• Request HTTP GET to the specified URL.
• Anti-spyware function
• Sends DNS packets that resolve the specified domain.

Can I change the ping execution interval and timeout interval?

Currently it cannot be changed.
Values are fixed as follows.

Execution interval：500ms
Timeout：400ms

Can I change the maximum number of hops and timeout value of trace-route?

The timeout value can be changed. (Specify the parameters at test scenario.)
The default values are as follows.

Maximum hops：40
Timeout for each hops：50ms

Is it possible to test even if the IP address is translated(NAT) on the way?

Sure.
Please refer to this document how to describe the test scenario under the NAT environment.

Is there a time limit for running network tests?

There is no limit to the test running time,
but we do not recommend running test for a long time as it is intended for network failure tests.
It seems that the failure test is often run in a few minutes per pattern(link down of network device, etc.), so it is recommended that the network test running time be about the same.

Tell me the maximum throughput speed reachable by NEEDLEWORK.

It varies depending on the number of scenarios and parameters. We have confirmed in our evaluation environment that the output is about 900-950Mbps under the following conditions.

Number of scenario：1
Packet size：1518 Byte

Is it possible to test even if the IP address is translated(NAT) on the way?

Yes.
Please refer to this document how to describe the test scenario under the NAT environment.

How many sessions can NEEDLEWORK establish?

The performance limits of the device are as follows.

• Maximum number of sessions：500,000
• Number of sessions per second：10,000 sessions

Is it possible to test even if the IP address is translated(NAT) on the way?

Yes.
Please refer to this document how to describe the test scenario under the NAT environment.