it's acting weird!

[neginsadeghi]

Hello there, I'm running lzr with zmap, it gives me correct output for one time and then it returns 0 results for all other scans!

note: my main interface is ens160, there is only one IP address on this interface, I've used this to make the lzr
make all source-ip=x.x.x.x/32

my target network has at least 21 results on port 80, zmap can verify that but I don't know what happens that it's giving me this results !

one time that it gave me results ( just 5 item, it should give me about 21 items ) !

{"saddr":"x.x.x.13","daddr":"185.x.x.x","sport":80,"dport":56425,"seqnum":3014031342,"acknum":110,"window":29200,"ttl":60,"Counter":0,"ACK":true,"ACKed":true,"SYN":false,"RST":false,"FIN":false,"PUSH":true,"HandshakeNum":1,"fingerprint":"http","Timestamp":"2023-11-13T12:37:18.393362549+03:30","expectedRToLZR":"data","data":"HTTP/1.1 301 Moved Permanently\r\nServer: openresty\r\nDate: Mon, 13 Nov 2023 09:07:18 GMT\r\nContent-Type: text/html\r\nContent-Length: 166\r\nConnection: keep-alive\r\nLocation: https://x.x.x.13/\r\n\r\n\u003chtml\u003e\r\n\u003chead\u003e\u003ctitle\u003e301 Moved Permanently\u003c/title\u003e\u003c/head\u003e\r\n\u003cbody\u003e\r\n\u003ccenter\u003e\u003ch1\u003e301 Moved Permanently\u003c/h1\u003e\u003c/center\u003e\r\n\u003chr\u003e\u003ccenter\u003eopenresty\u003c/center\u003e\r\n\u003c/body\u003e\r\n\u003c/html\u003e\r\n"}
{"saddr":"x.x.x.12","daddr":"185.x.x.x","sport":80,"dport":46649,"seqnum":1393460254,"acknum":110,"window":29200,"ttl":60,"Counter":0,"ACK":true,"ACKed":true,"SYN":false,"RST":false,"FIN":false,"PUSH":true,"HandshakeNum":1,"fingerprint":"http","Timestamp":"2023-11-13T12:37:18.393380333+03:30","expectedRToLZR":"data","data":"HTTP/1.1 301 Moved Permanently\r\nServer: openresty\r\nDate: Mon, 13 Nov 2023 09:07:18 GMT\r\nContent-Type: text/html\r\nContent-Length: 166\r\nConnection: keep-alive\r\nLocation: https://x.x.x.12/\r\n\r\n\u003chtml\u003e\r\n\u003chead\u003e\u003ctitle\u003e301 Moved Permanently\u003c/title\u003e\u003c/head\u003e\r\n\u003cbody\u003e\r\n\u003ccenter\u003e\u003ch1\u003e301 Moved Permanently\u003c/h1\u003e\u003c/center\u003e\r\n\u003chr\u003e\u003ccenter\u003eopenresty\u003c/center\u003e\r\n\u003c/body\u003e\r\n\u003c/html\u003e\r\n"}
{"saddr":"x.x.x.11","daddr":"185.x.x.x","sport":80,"dport":47539,"seqnum":2199870258,"acknum":110,"window":29200,"ttl":60,"Counter":0,"ACK":true,"ACKed":true,"SYN":false,"RST":false,"FIN":false,"PUSH":true,"HandshakeNum":1,"fingerprint":"http","Timestamp":"2023-11-13T12:37:18.393385312+03:30","expectedRToLZR":"data","data":"HTTP/1.1 301 Moved Permanently\r\nServer: openresty\r\nDate: Mon, 13 Nov 2023 09:07:18 GMT\r\nContent-Type: text/html\r\nContent-Length: 166\r\nConnection: keep-alive\r\nLocation: https://x.x.x.11/\r\n\r\n\u003chtml\u003e\r\n\u003chead\u003e\u003ctitle\u003e301 Moved Permanently\u003c/title\u003e\u003c/head\u003e\r\n\u003cbody\u003e\r\n\u003ccenter\u003e\u003ch1\u003e301 Moved Permanently\u003c/h1\u003e\u003c/center\u003e\r\n\u003chr\u003e\u003ccenter\u003eopenresty\u003c/center\u003e\r\n\u003c/body\u003e\r\n\u003c/html\u003e\r\n"}
{"saddr":"x.x.x.10","daddr":"185.x.x.x","sport":80,"dport":51667,"seqnum":2846776825,"acknum":110,"window":29200,"ttl":60,"Counter":0,"ACK":true,"ACKed":true,"SYN":false,"RST":false,"FIN":false,"PUSH":true,"HandshakeNum":1,"fingerprint":"http","Timestamp":"2023-11-13T12:37:18.393396227+03:30","expectedRToLZR":"data","data":"HTTP/1.1 301 Moved Permanently\r\nServer: openresty\r\nDate: Mon, 13 Nov 2023 09:07:18 GMT\r\nContent-Type: text/html\r\nContent-Length: 166\r\nConnection: keep-alive\r\nLocation: https://x.x.x.10/\r\n\r\n\u003chtml\u003e\r\n\u003chead\u003e\u003ctitle\u003e301 Moved Permanently\u003c/title\u003e\u003c/head\u003e\r\n\u003cbody\u003e\r\n\u003ccenter\u003e\u003ch1\u003e301 Moved Permanently\u003c/h1\u003e\u003c/center\u003e\r\n\u003chr\u003e\u003ccenter\u003eopenresty\u003c/center\u003e\r\n\u003c/body\u003e\r\n\u003c/html\u003e\r\n"}
{"saddr":"x.x.x.14","daddr":"185.x.x.x","sport":80,"dport":57849,"seqnum":1098684816,"acknum":110,"window":29200,"ttl":60,"Counter":0,"ACK":true,"ACKed":true,"SYN":false,"RST":false,"FIN":false,"PUSH":true,"HandshakeNum":1,"fingerprint":"http","Timestamp":"2023-11-13T12:37:18.39340616+03:30","expectedRToLZR":"data","data":"HTTP/1.1 301 Moved Permanently\r\nServer: openresty\r\nDate: Mon, 13 Nov 2023 09:07:18 GMT\r\nContent-Type: text/html\r\nContent-Length: 166\r\nConnection: keep-alive\r\nLocation: https://x.x.x.14/\r\n\r\n\u003chtml\u003e\r\n\u003chead\u003e\u003ctitle\u003e301 Moved Permanently\u003c/title\u003e\u003c/head\u003e\r\n\u003cbody\u003e\r\n\u003ccenter\u003e\u003ch1\u003e301 Moved Permanently\u003c/h1\u003e\u003c/center\u003e\r\n\u003chr\u003e\u003ccenter\u003eopenresty\u003c/center\u003e\r\n\u003c/body\u003e\r\n\u003c/html\u003e\r\n"}

output for wrong results:
root@ubuntu20:/tmp/lzr# port=80 ; target=86.x.x.x/24 ; zmap -c 5 --retries=3 $target -i ens160 --target-port=$port --output-filter="success = 1 && repeat = 0" -f "saddr,daddr,sport,dport,seqnum,acknum,window" -O json | lzr --handshakes wait,http,tls -sendSYNs -sendInterface ens160
Nov 13 13:05:50.540 [WARN] blocklist: ZMap is currently using the default blocklist located at /etc/zmap/blocklist.conf. By default, this blocklist excludes locally scoped networks (e.g. 10.0.0.0/8, 127.0.0.1/8, and 192.168.0.0/16). If you are trying to scan local networks, you can change the default blocklist by editing the default ZMap configuration at /etc/zmap/blocklist.conf. If you have modified the default blocklist, you can ignore this message.
Nov 13 13:05:50.542 [INFO] dedup: Response deduplication method is full
++Writing results to file: default_20231113130550.json
++Handshakes: wait,http,tls
++Sending SYNs
++Using Sending Interface: ens160
++Worker threads: 1
++Timeout Interval (s): 5
++Retransmit Interval (s): 1
++Number of Retransmitions: 1
Nov 13 13:05:50.617 [INFO] recv: duplicate responses will be passed to the output module
Nov 13 13:05:50.620 [INFO] recv: unsuccessful responses will be passed to the output module
0:00 0%; send: 0 0 p/s (0 p/s avg); recv: 0 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 0.00%
0:00 0%; send: 0 0 p/s (0 p/s avg); recv: 0 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 0.00%
0:01 21%; send: 256 done (2.49 Kp/s avg); recv: 21 21 p/s (19 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 8.20%
0:02 40%; send: 256 done (2.49 Kp/s avg); recv: 21 0 p/s (10 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 8.20%
0:03 60%; send: 256 done (2.49 Kp/s avg); recv: 21 0 p/s (6 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 8.20%
0:04 80%; send: 256 done (2.49 Kp/s avg); recv: 21 0 p/s (5 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 8.20%
0:05 99% (1s left); send: 256 done (2.49 Kp/s avg); recv: 21 0 p/s (4 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 8.20%
Nov 13 13:05:56.633 [INFO] zmap: completed
Killed

[neginsadeghi]

I even created services.txt in this format and I've verified all of them has port 80 open:
IP:port

cat services.txt | pv -L3 -l --qui | lzr --handshakes wait,http,tls -sendSYNs -sendInterface ens160 -gatewayMac xx:xx:71:f7 -sourceIP 185.x.x.x

and here is the results:

++Writing results to file: default_20231113132411.json
++Handshakes: wait,http,tls
++Sending SYNs
++Using SourceIP: true
++Using Sending Interface: ens160
++Using Gateway Mac: xxxxx
++Worker threads: 1
++Timeout Interval (s): 5
++Retransmit Interval (s): 1
++Number of Retransmitions: 1
Finished Reading Input
Runtime: 7.448603923s
{"TotalResponses":0,"ZeroWindow":0,"ACKed":0,"Data":0,"No_SYNACK":0,"Rst":0,"Fin":0,"Resp_ack":0,"HyperACKtive":0}

[LizIzhikevich]

Hi! Can you take a look at the issue I linked below and read through it and 1. take similar debugging steps as I recommended there and 2. see if the eventual solution there is applicable to your case?

#13

[neginsadeghi]

look we have 52% hitrate based on Zmap output, I can't figureout what's happening that LZR reacts like this

root@ubuntu20:~# sudo zmap 185.x.x.0/24 --target-port=80 --output-filter="success = 1 && repeat = 0" -f "saddr,daddr,sport,dport,seqnum,acknum,window" -O json --source-ip=185.x.x.x -i ens160 | sudo lzr --handshakes wait,http,tls -sendInterface ens160 -gatewayMac xxx:b3:71:f7 -feedZGrab | zgrab2 multiple -c /tmp/zgrab2/testrun/base-configurations/all.ini
Nov 14 09:23:49.289 [WARN] blocklist: ZMap is currently using the default blocklist located at /etc/zmap/blocklist.conf. By default, this blocklist excludes locally scoped networks (e.g. 10.0.0.0/8, 127.0.0.1/8, and 192.168.0.0/16). If you are trying to scan local networks, you can change the default blocklist by editing the default ZMap configuration at /etc/zmap/blocklist.conf. If you have modified the default blocklist, you can ignore this message.
Nov 14 09:23:49.291 [INFO] dedup: Response deduplication method is full
INFO[0000] started grab at 2023-11-14T09:23:49+03:30
++Writing results to file: default_20231114092349.json
++Handshakes: wait,http,tls
++Using Sending Interface: ens160
++Using Gateway Mac: xxxx3:71:f7
++Feeding ZGrab with fingerprints
++Worker threads: 1
++Timeout Interval (s): 5
++Retransmit Interval (s): 1
++Number of Retransmitions: 1
Nov 14 09:23:49.391 [INFO] recv: duplicate responses will be passed to the output module
Nov 14 09:23:49.391 [INFO] recv: unsuccessful responses will be passed to the output module
0:00 0%; send: 0 0 p/s (0 p/s avg); recv: 0 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 0.00%
0:01 13%; send: 256 done (2.25 Kp/s avg); recv: 0 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 0.00%
0:02 25%; send: 256 done (2.25 Kp/s avg); recv: 134 134 p/s (64 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 52.34%
0:03 38%; send: 256 done (2.25 Kp/s avg); recv: 134 0 p/s (43 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 52.34%
0:04 50%; send: 256 done (2.25 Kp/s avg); recv: 134 0 p/s (32 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 52.34%
0:05 62% (4s left); send: 256 done (2.25 Kp/s avg); recv: 134 0 p/s (26 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 52.34%
0:06 75% (3s left); send: 256 done (2.25 Kp/s avg); recv: 134 0 p/s (22 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 52.34%
0:07 87% (2s left); send: 256 done (2.25 Kp/s avg); recv: 134 0 p/s (18 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 52.34%
0:08 99% (1s left); send: 256 done (2.25 Kp/s avg); recv: 134 0 p/s (16 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 52.34%
Nov 14 09:23:58.400 [INFO] zmap: completed
Finished Reading Input
Runtime: 9.130180185s
{"TotalResponses":0,"ZeroWindow":0,"ACKed":0,"Data":0,"No_SYNACK":0,"Rst":0,"Fin":0,"Resp_ack":0,"HyperACKtive":0}
INFO[0009] finished grab at 2023-11-14T09:23:58+03:30
{"statuses":{},"start":"2023-11-14T09:23:49+03:30","end":"2023-11-14T09:23:58+03:30","duration":"9.399893063s"}

[neginsadeghi]

I guess it's a problem in LZR core

[neginsadeghi]

I've used the release version instead of master and now it's kinda working

root@xxx:/home/xxx/zmap_R_D/lzr/v1/lzr-1/cmd/lzr# sudo zmap x.x.x.20/30 --target-port=80 --output-filter="success = 1 && repeat = 0" -f "saddr,daddr,sport,dport,seqnum,acknum,window" -O json --source-ip=x.x.x.3 -i ens160 | sudo ./lzr --handshakes wait,http,tls -sendInterface ens160 -gatewayMac 0xxxx6:07:e4:61 -feedZGrab | zgrab2 multiple -c /home/xxx/zmap_R_D/zgrab2/zgrab2-configurations/base-configurations/all.ini
++Writing results to file: default_20231115100922.json
++Handshakes: wait,http,tls
++Using Sending Interface: ens160
++Using Gateway Mac: 00:50:56:07:e4:61
++Feeding ZGrab with fingerprints
++Worker threads: 1
++Timeout Interval (s): 5
++Retransmit Interval (s): 1
++Number of Retransmitions: 1
INFO[0000] started grab at 2023-11-15T10:09:22Z
Nov 15 10:09:22.984 [INFO] recv: duplicate responses will be passed to the output module
Nov 15 10:09:22.984 [INFO] recv: unsuccessful responses will be passed to the output module
0:00 1%; send: 1 1 p/s (22 p/s avg); recv: 0 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 0.00%
0:01 13%; send: 4 done (91 p/s avg); recv: 4 4 p/s (3 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:02 25%; send: 4 done (91 p/s avg); recv: 4 0 p/s (1 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:03 38%; send: 4 done (91 p/s avg); recv: 4 0 p/s (1 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:04 50%; send: 4 done (91 p/s avg); recv: 4 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:05 63% (3s left); send: 4 done (91 p/s avg); recv: 4 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:06 75% (2s left); send: 4 done (91 p/s avg); recv: 4 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:07 88% (1s left); send: 4 done (91 p/s avg); recv: 4 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:08 101% (0s left); send: 4 done (91 p/s avg); recv: 4 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
Nov 15 10:09:32.032 [INFO] zmap: completed
Finished Reading Input
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 0
Runtime: 27.127366037s
{"TotalResponses":4,"ZeroWindow":0,"ACKed":0,"Data":0,"No_SYNACK":4,"Rst":0,"Fin":0,"Resp_ack":0,"HyperACKtive":0}
unknown: 4
{"ip":"x.x.x.20"}
{"ip":"x.x.x.22"}
{"ip":"x.x.x.21"}
{"ip":"x.x.x.23"}
INFO[0027] finished grab at 2023-11-15T10:09:50Z
{"statuses":{},"start":"2023-11-15T10:09:22Z","end":"2023-11-15T10:09:50Z","duration":"27.236364744s"}

but it couldn't detect HTTP, it says unknown
also I don't know what are these :

Finished Reading Input
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4
Finishing Last: 4

didn't saw them before on LZR

[neginsadeghi]

here is the output file content

root@xxx:/home/xxx/zmap_R_D/lzr/v1/lzr-1/cmd/lzr# cat default_20231115100922.json
{"saddr":"x.x.x.20","daddr":"x.x.x.3","sport":80,"dport":41528,"seqnum":0,"acknum":0,"window":29200,"ttl":0,"Counter":1,"ACK":false,"ACKed":false,"SYN":false,"RST":false,"FIN":false,"PUSH":false,"HandshakeNum":2,"fingerprint":"unknown","Timestamp":"2023-11-15T10:09:44.961352982Z","expectedRToLZR":"sa"}
{"saddr":"x.x.x.22","daddr":"x.x.x.3","sport":80,"dport":36285,"seqnum":0,"acknum":0,"window":29200,"ttl":0,"Counter":1,"ACK":false,"ACKed":false,"SYN":false,"RST":false,"FIN":false,"PUSH":false,"HandshakeNum":2,"fingerprint":"unknown","Timestamp":"2023-11-15T10:09:44.961360806Z","expectedRToLZR":"sa"}
{"saddr":"x.x.x.21","daddr":"x.x.x.3","sport":80,"dport":49518,"seqnum":0,"acknum":0,"window":29200,"ttl":0,"Counter":1,"ACK":false,"ACKed":false,"SYN":false,"RST":false,"FIN":false,"PUSH":false,"HandshakeNum":2,"fingerprint":"unknown","Timestamp":"2023-11-15T10:09:44.961365646Z","expectedRToLZR":"sa"}
{"saddr":"x.x.x.23","daddr":"x.x.x.3","sport":80,"dport":59933,"seqnum":0,"acknum":0,"window":29200,"ttl":0,"Counter":1,"ACK":false,"ACKed":false,"SYN":false,"RST":false,"FIN":false,"PUSH":false,"HandshakeNum":2,"fingerprint":"unknown","Timestamp":"2023-11-15T10:09:44.961375023Z","expectedRToLZR":"sa"}

[neginsadeghi]

Here I've used -sendsyn + source ip too, but it gave me error

root@xxxx:/home/xxxx/zmap_R_D/lzr/v1/lzr-1/cmd/lzr# sudo zmap x.x.x.1/30 --target-port=80 --output-filter="success = 1 && repeat = 0" -f "saddr,daddr,sport,dport,seqnum,acknum,window" -O json --source-ip=x.x.x.3 -i ens160 | sudo ./lzr -sendSYNs -sourceIP x.x.x.3 --handshakes wait,http,tls -sendInterface ens160 -gatewayMac xxxx:07:e4:61 -feedZGrab | zgrab2 multiple -c /home/xxxx/zmap_R_D/zgrab2/zgrab2-configurations/base-configurations/all.ini
++Writing results to file: default_20231115101434.json
++Handshakes: wait,http,tls
++Sending SYNs
++Using SourceIP: true
++Using Sending Interface: ens160
++Using Gateway Mac: xxxx:07:e4:61
++Feeding ZGrab with fingerprints
++Worker threads: 1
++Timeout Interval (s): 5
++Retransmit Interval (s): 1
++Number of Retransmitions: 1
INFO[0000] started grab at 2023-11-15T10:14:34Z
Nov 15 10:14:34.300 [INFO] recv: duplicate responses will be passed to the output module
Nov 15 10:14:34.301 [INFO] recv: unsuccessful responses will be passed to the output module
0:00 0%; send: 1 1 p/s (36 p/s avg); recv: 0 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 0.00%
panic: Error parsing input list

goroutine 54 [running]:
github.com/stanford-esrg/lzr.convertFromInputListToPacket({0xc5374dacf0, 0x81})
/home/xxxx/zmap_R_D/lzr/v1/lzr-1/packet.go:152 +0x3e5
github.com/stanford-esrg/lzr.ConstructIncomingRoutine.func1()
/home/xxxx/zmap_R_D/lzr/v1/lzr-1/construct_main_routines.go:73 +0x185
created by github.com/stanford-esrg/lzr.ConstructIncomingRoutine in goroutine 1
/home/xxxx/zmap_R_D/lzr/v1/lzr-1/construct_main_routines.go:58 +0x66
0:01 13%; send: 4 done (145 p/s avg); recv: 4 4 p/s (3 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
INFO[0001] finished grab at 2023-11-15T10:14:35Z
{"statuses":{},"start":"2023-11-15T10:14:34Z","end":"2023-11-15T10:14:35Z","duration":"1.10129257s"}
0:02 25%; send: 4 done (145 p/s avg); recv: 4 0 p/s (1 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:03 38%; send: 4 done (145 p/s avg); recv: 4 0 p/s (1 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:04 50%; send: 4 done (145 p/s avg); recv: 4 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:05 63% (3s left); send: 4 done (145 p/s avg); recv: 4 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:06 75% (2s left); send: 4 done (145 p/s avg); recv: 4 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:07 88% (1s left); send: 4 done (145 p/s avg); recv: 4 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
0:08 100% (0s left); send: 4 done (145 p/s avg); recv: 4 0 p/s (0 p/s avg); drops: 0 p/s (0 p/s avg); hitrate: 100.00%
Nov 15 10:14:43.302 [INFO] zmap: completed