-
Notifications
You must be signed in to change notification settings - Fork 548
feat: Add ParseKVLax for Flexible Key-Value Parsing #4007
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
feat: Add ParseKVLax for Flexible Key-Value Parsing #4007
Conversation
|
@LaurenceJJones: There are no 'kind' label on this PR. You need a 'kind' label to generate the release automatically.
DetailsI am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository. |
|
@LaurenceJJones: There are no area labels on this PR. You can add as many areas as you see fit.
DetailsI am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #4007 +/- ##
==========================================
+ Coverage 62.67% 62.71% +0.04%
==========================================
Files 410 410
Lines 32943 33003 +60
==========================================
+ Hits 20646 20698 +52
- Misses 10173 10177 +4
- Partials 2124 2128 +4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
- Replace regex-based parsing with scanner approach for better handling of complex key-value pairs - Add support for unquoted values containing spaces (e.g., UNIFIhost=Express 7) - Maintain backward compatibility with existing quoted and simple unquoted values - Add robust filtering to prevent false positives from invalid key patterns - Improve quote handling and escaping for quoted values - Add comprehensive test cases covering edge cases and mixed scenarios Fixes parsing issues with CEF logs and other formats where values contain spaces without quotes.
4303a62 to
0f4a627
Compare
|
linked too: crowdsecurity/hub#940 |
This PR is mainly aimed at supporting a looser key value parsing for Unifi CEF lines (even though the tests have full log line in practice we will parse out the message so it more aimed). Our current implementation for key value only parses:
foo=barbar="foo bar"Overview
This PR introduces
ParseKVLax, a new key-value parser that complements the existingParseKVfunction. WhileParseKVmaintains strict regex-based parsing for backward compatibility,ParseKVLaxuses a scanner-based approach to support unquoted values with spaces and other complex log formats.Changes
New Function:
ParseKVLaxkey=patterns and determines value boundaries intelligentlyUNIFIhost=Express 7 port=443now correctly parses as{"UNIFIhost": "Express 7", "port": "443"}key=patterns inside quoted values\") and backslashes (\\)Key Features
ParseKVunchanged - all existing code continues to workParseKVLaxonly where needed for loose parsingUse Cases
msg=User login successful→ captures full phraseRES=0x00 SYN URGP=0→ captures flags as part of valuesUNIFIhost=Express 7→ captures version as part of value