Smtp server detection 1125 v2.6

[catenacyber]

Link to ticket: https://redmine.openinfosecfoundation.org/issues/
https://redmine.openinfosecfoundation.org/issues/1125
https://redmine.openinfosecfoundation.org/issues/6821
https://redmine.openinfosecfoundation.org/issues/5491

Describe changes:

• smtp server detection (ie to_client)
• ftp server detection (ie to_client)
• smtp recognize more reply codes

SV_BRANCH=OISF/suricata-verify#1894

#11327 with less SMTP wrongly recognized when it is FTP

[codecov]

Codecov Report

Attention: Patch coverage is 86.74699% with 11 lines in your changes missing coverage. Please review.

Project coverage is 82.39%. Comparing base (6256391) to head (86bb366).
Report is 6 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #11333      +/-   ##
==========================================
- Coverage   82.41%   82.39%   -0.03%     
==========================================
  Files         934      934              
  Lines      247239   247303      +64     
==========================================
- Hits       203773   203770       -3     
- Misses      43466    43533      +67     

Flag	Coverage Δ
fuzzcorpus	60.24% <84.05%> (+<0.01%)	⬆️
livemode	18.78% <11.59%> (+0.03%)	⬆️
pcap	43.78% <81.15%> (-0.02%)	⬇️
suricata-verify	61.32% <81.15%> (-0.04%)	⬇️
unittests	59.31% <46.98%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

[suricata-qa]

WARNING:

field	baseline	test	%
	SURI_TLPW1_stats_chk
.app_layer.error.smtp.parser	409	42	10.27%
	SURI_TLPR1_stats_chk
.app_layer.flow.smtp	335817	347573	103.5%
.app_layer.flow.failed_tcp	178240	167210	93.81%
.app_layer.tx.ftp	101030	95372	94.4%
.app_layer.error.smtp.parser	527	144	27.32%
.ftp.memuse	10637	2878	27.06%

Pipeline 21147

[catenacyber]

@ct0br0 where were we with this ?

Looks like this new version still has less FTP transactions and memuse...
Do we have minimized pcaps showing it ?

[ct0br0]

@ct0br0 where were we with this ?

Looks like this new version still has less FTP transactions and memuse... Do we have minimized pcaps showing it ?

We have a hardware issue, I haven't been able to do much of anything. I am going to try to use a different box today but no telling how that will go.

[catenacyber]

Thanks for the status Corey, no rush here

[ct0br0]

ok that's good. This box has a spinning disk and is absolute pain lol

[catenacyber]

From what I see, these are mostly IRC flows that get less FTP transactions

jq 'select(.event_type=="ftp") | .dest_port' master.json | sort | uniq -c | sort -n
...
 141 4040
 147 443
 190 80
6855 21
26297 6667

jq 'select(.event_type=="ftp") | .dest_port' pr.json | sort | uniq -c | sort -n
...
  69 4040
  72 6668
  82 444
 180 80
6867 21
24927 6667

[catenacyber]

Continued in #11493