NEEDLEWORK

I can't access to NEEDLEWORK console with browser.

Check Proxy settings

The operation terminal is connected to the NEEDLEWORK console via HTTP communication(browser connection).

If the Proxy setting is enabled in the browser settings of the operating terminal, you may not be able to access successful.
Disable the Proxy setting, or exclude the management IP address (192.0.2.1) of the device or the IP address set for remote connection from the Proxy target.

Check cable connection / Network reachability

Check if the operation terminal is connected to the MGT port(ETH3) of the device.
For remote installation NEEDLEWORK, check if it has network reachability.

Download

Test to Juniper SRX fails(be error).

NEEDLEWORK does not respond to ARP immediately after startup (until the initial test is run).
Please refer to the following page for details of operating specifications.

About ARP responding specification（2019/06/28）

SRX periodically sends ARP requests to the gateway and does not forward packets until the gateway responds with them. (※).
Therefore, the test fails(be error) at the first test after startup.
Since the ARP response is received from the second time onward, the test can be performed successfully.

※Results in our evaluation environment

NEEDLE WORK is no longer available after uploading a monthly license

If you upload an expired license, it will no longer be available.
You can use it by re-uploading a valid license.

TCP communication drops on Cisco ASA

The ASA's "TCP Sequence Number Randomization" function randomizes the sequence numbers, so NEEDLEWORK receiver considers the sequence number incorrect.

This issue is fixed by the firmware version 3.0.1 of the device itself.

The communication that should be Pass is Drop.

Depending on the firewall model, packet payload inspection functions such as "ALG" or "Inspection" are enabled.
NEEDLEWORK generates a packet containing dummy data, not an actual application, so it will be Drop on ports where these functions are enabled.
※Some applications are supported.

Therefore, disable these functions on the port being tested before testing.
The following is an example that disables the payload inspection function.

Cisco ASA setting example

policy-map global_policy
                    　　class inspection_default
                    　　　　no inspect dns
                  

Cisco ASA setting example

unset alg dns enable
                  

The test result is not as expected in the Anti-virus and URL filtering tests.

Depending on the firewall model, if IPS or other UTM detection/protection functions are enabled, test communication (HTTP) from NEEDLEWORK may be considered invalid, and blocked.

In addition, Eicar, a test virus, is used in anti-virus testing. It may be detected and blocked by IPS function before it is detected by the anti-virus function.

Please check the firewall log to see if it is blocked by any function other than the Anti-virus or URL filter function.

The test results may change each time.

NEEDLEWORK is designed so that it can test without setting an IP address on the device itself.
Since the IP address is not retained, test started according to the following procedure.

① Make ARP requests from all ports of NEEDLEWORK.
　※ARP resolution of the IP address described in test scenario FW IP item.
② Start a test using the port that returned the ARP reply first.

Normal operation
※There is an ARP reply only from the port that holds the segment for ARP resolution.

Cause of the changed results

Normally, there is an ARP reply only from the port that holds the segment to be ARP resolved, but some models have ARP replies from all ports.

So test result be Drop because NEEDLEWORK sends packets to ports that do not hold the segment.

Abnormal operation

※There are ARP replies from all ports that sent ARP requests.

The model has been identified in our evaluation environment that perform the above operation is listed below.

• Palo Alto Networks Next Generation Firewall

For the above model, please use the following workarounds.

The method for workaround

Add 2 values to s-if(option) and d-if(option) of test scenario CSV. And specify the port number of NEEDLEWORK connected to the target segment.

In the following, the source interface(port) is set to 1 and the destination interface(port) is set to 0.

Add 2 values to test scenario.

Test result is Drop in FortiGate proxy mode.

If anti-virus function is enabled in the policies of test target ForiGate,
Please type "Proxy mode" in the other-settings of the test scenario.
※It is also described in the manual.

HTTPS test result is Drop in FortiGate.

NEEDLEWORK uses self-signed certificates for HTTPS test.
Make the following settings with FortiGate.

"Security Profile" → "SSL / SSH Inspection" Enable "Allow Invalid SSL Certificates" for the profile you are applying to the test target policy.

Ping is dropped during network test

Since Ping is executed in parallel, if there is a device in the route that does not have enough specification(maximum number of sessions retention), the session table overflows and Ping may fail.

Please adjust Ping session retention time of the device, etc.

The throughput is much lower than expected.

If the destination port number is a well-known port such as UDP53, inspection will be performed up to the application level depending on the specifications of the device to test.(FW, etc.) This may increase the CPU load.
Therefore, throughput may not be generated due to insufficient resources in the equipments to test.

In that case, you can solve the problem by changing the port number.

The loss rate of the stress test is higher than expected.

With the default setting(PPS, frame size) sends test communications to the upper limit of the device specifications.
Therefore, the load(test communications) exceeds the performance limit of the target device, and the loss rate may increase.

You can adjust the load(test communications) by setting the 「frame size」 or 「maximum PPS」 of the scenario.

Example：
                    frame size 	：1000
                    maximum PPS			：10000
                     
                    （frame size：1,000Byte × maximum PPS：10,000） × 8 = 80Mbps
                     
                    With the above settings, it is possible to apply a load of about 80 Mbps.
                  

There is a big difference in the number of sessions displayed between NEEDLE WORK and the device under test.

Some network devices return ACK to SYN even when the processing limit is reached. In that case, NEEDLE WORK cannot count the number of sessions properly. So please check the number of sessions on the network device side.