Skip to content

Comments

Dataset string memcap 3910/v5#11105

Closed
inashivb wants to merge 5 commits intoOISF:masterfrom
inashivb:dataset-string-memcap-3910/v5
Closed

Dataset string memcap 3910/v5#11105
inashivb wants to merge 5 commits intoOISF:masterfrom
inashivb:dataset-string-memcap-3910/v5

Conversation

@inashivb
Copy link
Member

@inashivb inashivb commented May 21, 2024

Link to redmine ticket: https://redmine.openinfosecfoundation.org/issues/3910

Previous PR: #11104

Changes since v4:

  • removed notice stmt

@inashivb inashivb force-pushed the dataset-string-memcap-3910/v5 branch from a1ad5c5 to 27bff45 Compare May 21, 2024 12:48
@codecov
Copy link

codecov bot commented May 21, 2024

Codecov Report

Attention: Patch coverage is 43.66197% with 40 lines in your changes are missing coverage. Please review.

Project coverage is 75.18%. Comparing base (8560564) to head (03a00a1).

Additional details and impacted files
@@             Coverage Diff             @@
##           master   #11105       +/-   ##
===========================================
- Coverage   84.09%   75.18%    -8.92%     
===========================================
  Files         925      852       -73     
  Lines      250594   137097   -113497     
===========================================
- Hits       210733   103071   -107662     
+ Misses      39861    34026     -5835     
Flag Coverage Δ
fuzzcorpus 64.19% <42.25%> (-0.02%) ⬇️
livemode ?
pcap 46.43% <18.30%> (-0.09%) ⬇️
suricata-verify 62.76% <43.66%> (-0.04%) ⬇️
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

inashivb and others added 5 commits May 21, 2024 21:28
In order to have access to the length of datatypes with variable lengths
to correctly update memuse to calculate memcaps.

Bug 3910
So far, when the data size was passed to the THash API, it was sent as
a sizeof(Struct) which works fine for the other data types as they have
a fixed length but not for the StringType.
However, because of the sizeof construct, the length of a string type
dataset was always taken to be 16 Bytes which is only the size of the struct
itself. It did not accomodate the actual size of the string that the
StringType holds. Fix this so that the memuse that is used to determine
whether memcap was reached also takes into consideration the size of the
actual string.

Bug 3910
Add a callback and helper function to handle data expiration.

Update datasets to explicitly not use expiration.
@inashivb inashivb force-pushed the dataset-string-memcap-3910/v5 branch from 27bff45 to 03a00a1 Compare May 21, 2024 16:04
@suricata-qa
Copy link

Information: QA ran without warnings.

Pipeline 20713

@inashivb inashivb marked this pull request as ready for review May 22, 2024 05:01
Copy link
Contributor

@jufajardini jufajardini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a suggestion for the upgrading doc.
The other two questions are more out of curiosity, to learn more...

@inashivb inashivb closed this May 23, 2024
@inashivb inashivb deleted the dataset-string-memcap-3910/v5 branch May 23, 2024 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants