Smtp server detection 1125 v2.2

[catenacyber]

Link to ticket: https://redmine.openinfosecfoundation.org/issues/
https://redmine.openinfosecfoundation.org/issues/1125
https://redmine.openinfosecfoundation.org/issues/6821
https://redmine.openinfosecfoundation.org/issues/5491

Describe changes:

• smtp server detection (ie to_client)
• ftp server detection (ie to_client)
• smtp recognize more reply codes

SV_BRANCH=OISF/suricata-verify#1850

#11128 with additional commit for smtp reply codes to get less smtp parser errors

[catenacyber]

There remain some unexpected SMTP reply codes :

• 400
• 472
• 4.7.0 (instead of 454)

and also some SMTP servers will reply unexpected command + the junk (TLS handshake from client) including multiple lines of it, so we get junk lines from the server

[codecov]

Codecov Report

Attention: Patch coverage is 85.39326% with 13 lines in your changes are missing coverage. Please review.

Project coverage is 82.76%. Comparing base (daa6f6f) to head (4011cec).
Report is 6 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #11193      +/-   ##
==========================================
- Coverage   82.93%   82.76%   -0.17%     
==========================================
  Files         942      942              
  Lines      250797   250908     +111     
==========================================
- Hits       207994   207676     -318     
- Misses      42803    43232     +429     

Flag	Coverage Δ
fuzzcorpus	61.31% <82.66%> (+<0.01%)	⬆️
livemode	18.81% <16.00%> (+0.03%)	⬆️
pcap	44.56% <81.33%> (-0.08%)	⬇️
suricata-verify	61.10% <80.00%> (-0.34%)	⬇️
unittests	60.66% <48.31%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

[suricata-qa]

WARNING:

field	baseline	test	%
	SURI_TLPW1_stats_chk
.app_layer.flow.smtp	7556	7827	103.59%
.app_layer.error.smtp.parser	409	42	10.27%
	SURI_TLPR1_stats_chk
.app_layer.flow.smtp	335817	347722	103.55%
.app_layer.flow.failed_tcp	178240	161818	90.79%
.app_layer.tx.ftp	101030	95212	94.24%
.app_layer.error.smtp.parser	527	177	33.59%
.ftp.memuse	10637	2921	27.46%

Pipeline 20871

[victorjulien]

Nice... Now I'm curious about the remaining 42 errors :)

[catenacyber]

Nice... Now I'm curious about the remaining 42 errors :)

See #11193 (comment)

[victorjulien]

Since the 42 are from TLPW1, can we turn the unique error classes from it into SV tests?

[catenacyber]

Since the 42 are from TLPW1, can we turn the unique error classes from it into SV tests?

Done in OISF/suricata-verify#1894

[catenacyber]

Rebased in #11261