http: configures libhtp to allow spaces in uri#7341
Closed
catenacyber wants to merge 1 commit intoOISF:masterfrom
Closed
http: configures libhtp to allow spaces in uri#7341catenacyber wants to merge 1 commit intoOISF:masterfrom
catenacyber wants to merge 1 commit intoOISF:masterfrom
Conversation
Codecov Report
@@ Coverage Diff @@
## master #7341 +/- ##
==========================================
- Coverage 75.85% 75.77% -0.09%
==========================================
Files 656 656
Lines 190065 190066 +1
==========================================
- Hits 144171 144013 -158
- Misses 45894 46053 +159
Flags with carried forward coverage won't be shown. Click here to find out more. |
catenacyber
force-pushed
the
http-uri-spaces-2881-v8
branch
from
e89d507 to
5a20dc9
Compare
|
ERROR: ERROR: QA failed on tlpw1_files_sha256. ERROR: QA failed on tlpr1_alerts_cmp. Pipeline 7217 |
Contributor
Author
|
Replaced by #7352 |
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Oct 23, 2024
Extract a simple function function to a simple header with no circular dependencies. Next we'll use bindgen to generate Rust bindings for this function. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Oct 23, 2024
Use bindgen to generate Rust bindings for "app-layer-ext.h", currently just one function, SCAppLayerStateGetProgressFn. We use bindgen as a build dependency and dynamically (re)build the bindings as needed. The alternative would be to require bindgen-cli as a build tool. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Oct 23, 2024
Extract a simple function function to a simple header with no circular dependencies. Next we'll use bindgen to generate Rust bindings for this function. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Oct 23, 2024
Use bindgen to generate Rust bindings for "app-layer-ext.h", currently just one function, SCAppLayerStateGetProgressFn. We use bindgen as a build dependency and dynamically (re)build the bindings as needed. The alternative would be to require bindgen-cli as a build tool. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Oct 23, 2024
Extract a simple function function to a simple header with no circular dependencies. Next we'll use bindgen to generate Rust bindings for this function. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Oct 23, 2024
Use bindgen to generate Rust bindings for "app-layer-ext.h", currently just one function, SCAppLayerStateGetProgressFn. We use bindgen as a build dependency and dynamically (re)build the bindings as needed. The alternative would be to require bindgen-cli as a build tool. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Oct 24, 2024
Extract a simple function function to a simple header with no circular dependencies. Next we'll use bindgen to generate Rust bindings for this function. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Jan 22, 2025
Add a minimal integration of bindgen to the build. Bindgen is integrated at compile time with a "build.rs" file that for now only generates AppLayerEventType Rust bindings. This required some refactoring of the C so app-layer-events.h did not also include "rust.h", which causes issues for bindgen, probably related to circular references. AppLayerEventType was chosen as the first step as its an argument type some app-layer functions that we may want to use bindgen to export Rust, and one of the requirements of bindgen might be that C functions should only use datatypes defined in C, and not Rust. Following such a rule also prevents circular dependencies between Rust and C code. Bindgen generates the bindings in a file named "bindings.rs" in the target/ direction, which "sys.rs" will statically "include" at compiling name, making these bindings available under package "crate::sys". "build.rs" had to be placed in the non-standard location of "src/build.rs" (its usually alongside Cargo.toml) to satisfy the out-of-tree build requirements of distcheck. Note that bindgen is also available as a command line tool which could be used instead of integrating this at compile time, however the tool requires a newer version of Rust than our MSRV, and may present additional issues with respect to autoconf and distcheck. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Jan 22, 2025
Add a minimal integration of bindgen to the build. This required some refactoring of the C so app-layer-events.h did not also include "rust.h", which causes issues for bindgen, probably related to circular references. AppLayerEventType was chosen as the first step as its an argument type some app-layer functions that we may want to use bindgen to export Rust, and one of the requirements of bindgen might be that C functions should only use datatypes defined in C, and not Rust. Following such a rule also prevents circular dependencies between Rust and C code. As bindgen can only accept one header file, we construct a pseudo-header file of all the headers that need to be consumed. Makefile dependency checking is done to make sure the pseudo-header is only generated as needed to avoid rebuilding every time. Special handling is required for Windows to use the Windows path. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Jan 22, 2025
Add a minimal integration of bindgen to the build. This required some refactoring of the C so app-layer-events.h did not also include "rust.h", which causes issues for bindgen, probably related to circular references. AppLayerEventType was chosen as the first step as its an argument type some app-layer functions that we may want to use bindgen to export Rust, and one of the requirements of bindgen might be that C functions should only use datatypes defined in C, and not Rust. Following such a rule also prevents circular dependencies between Rust and C code. As bindgen can only accept one header file, we construct a pseudo-header file of all the headers that need to be consumed. Makefile dependency checking is done to make sure the pseudo-header is only generated as needed to avoid rebuilding every time. Special handling is required for Windows to use the Windows path. Ticket: OISF#7341 bindgen: pre-generate rust bindings to C This simplifies building, as we don't have to worry about path and such under autoconf/automake. It does however mean when we update C headers that are exposed to Rust, we manually have to re-generate the bindings, but this is a check we can do in CI. It also eliminates the need for everyone who wants to build Suricata to have bindgen and clang tools installed.
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Jan 22, 2025
Add a minimal integration of bindgen to the build. This required some refactoring of the C so app-layer-events.h did not also include "rust.h", which causes issues for bindgen, probably related to circular references. AppLayerEventType was chosen as the first step as its an argument type some app-layer functions that we may want to use bindgen to export Rust, and one of the requirements of bindgen might be that C functions should only use datatypes defined in C, and not Rust. Following such a rule also prevents circular dependencies between Rust and C code. As bindgen can only accept one header file, we construct a pseudo-header file of all the headers that need to be consumed. Makefile dependency checking is done to make sure the pseudo-header is only generated as needed to avoid rebuilding every time. Special handling is required for Windows to use the Windows path. Ticket: OISF#7341 bindgen: pre-generate rust bindings to C This simplifies building, as we don't have to worry about path and such under autoconf/automake. It does however mean when we update C headers that are exposed to Rust, we manually have to re-generate the bindings, but this is a check we can do in CI. It also eliminates the need for everyone who wants to build Suricata to have bindgen and clang tools installed.
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Jan 22, 2025
Add a minimal integration of bindgen to the build. This required some refactoring of the C so app-layer-events.h did not also include "rust.h", which causes issues for bindgen, probably related to circular references. AppLayerEventType was chosen as the first step as its an argument type some app-layer functions that we may want to use bindgen to export Rust, and one of the requirements of bindgen might be that C functions should only use datatypes defined in C, and not Rust. Following such a rule also prevents circular dependencies between Rust and C code. As bindgen can only accept one header file, we construct a pseudo-header file of all the headers that need to be consumed. Makefile dependency checking is done to make sure the pseudo-header is only generated as needed to avoid rebuilding every time. Special handling is required for Windows to use the Windows path. Ticket: OISF#7341 bindgen: pre-generate rust bindings to C This simplifies building, as we don't have to worry about path and such under autoconf/automake. It does however mean when we update C headers that are exposed to Rust, we manually have to re-generate the bindings, but this is a check we can do in CI. It also eliminates the need for everyone who wants to build Suricata to have bindgen and clang tools installed.
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Jan 23, 2025
Add a minimal integration of bindgen to the build. This required some refactoring of the C so app-layer-events.h did not also include "rust.h", which causes issues for bindgen, probably related to circular references. AppLayerEventType was chosen as the first step as its an argument type some app-layer functions that we may want to use bindgen to export Rust, and one of the requirements of bindgen might be that C functions should only use datatypes defined in C, and not Rust. Following such a rule also prevents circular dependencies between Rust and C code. As bindgen can only accept one header file, we construct a pseudo-header file of all the headers that need to be consumed. Makefile dependency checking is done to make sure the pseudo-header is only generated as needed to avoid rebuilding every time. Special handling is required for Windows to use the Windows path. Ticket: OISF#7341 bindgen: pre-generate rust bindings to C This simplifies building, as we don't have to worry about path and such under autoconf/automake. It does however mean when we update C headers that are exposed to Rust, we manually have to re-generate the bindings, but this is a check we can do in CI. It also eliminates the need for everyone who wants to build Suricata to have bindgen and clang tools installed.
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Feb 16, 2025
Require a minimum version of 0.66.0. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Feb 16, 2025
Follow Rust convention of using a "sys" crate for bindings to C functions. The bindings don't exist yet, but will be generated by bindgen and put into this crate. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Feb 16, 2025
Bindgen works by processing a header file which includes all other header files it should generate bindings for. For this I've created bindgen.h which just includes app-layer-protos.h for now as an example. These bindings are then generated and saved in the "suricata-sys" crate and become availale as "suricata_sys::sys". Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Feb 16, 2025
Have bindgen generate bindings for app-layer-protos.h, then use the generated definitions of AppProto/AppProtoEnum instead if defining them ourselves. This header was chosen as its used by Rust, and its a simple header with no circular dependencies. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Feb 16, 2025
Regenerates the `sys.rs` and looks for any difference. Check will fail if there is a difference. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Feb 16, 2025
We don't keep bindgen's autogenerated do not edit line as it contains the bindgen version which could break the CI check for out of date bindings. So add our own do not edit line. Ticket: OISF#7341
jasonish
added a commit
to jasonish/suricata
that referenced
this pull request
Feb 16, 2025
Also disable bindgen's generated layout tests. They are valid for the platform generating the tests, but may not be valid for other platforms. For example, if the tests are generated on a 64 bit platform the tests will not be valid when run on a 32 bit platform as pointers are a different size. However, the generating bindings are valid for both platform. Ticket: OISF#7341
victorjulien
pushed a commit
to victorjulien/suricata
that referenced
this pull request
Feb 17, 2025
Require a minimum version of 0.66.0. Ticket: OISF#7341
victorjulien
pushed a commit
to victorjulien/suricata
that referenced
this pull request
Feb 17, 2025
Follow Rust convention of using a "sys" crate for bindings to C functions. The bindings don't exist yet, but will be generated by bindgen and put into this crate. Ticket: OISF#7341
victorjulien
pushed a commit
to victorjulien/suricata
that referenced
this pull request
Feb 17, 2025
Bindgen works by processing a header file which includes all other header files it should generate bindings for. For this I've created bindgen.h which just includes app-layer-protos.h for now as an example. These bindings are then generated and saved in the "suricata-sys" crate and become availale as "suricata_sys::sys". Ticket: OISF#7341
victorjulien
pushed a commit
to victorjulien/suricata
that referenced
this pull request
Feb 17, 2025
Have bindgen generate bindings for app-layer-protos.h, then use the generated definitions of AppProto/AppProtoEnum instead if defining them ourselves. This header was chosen as its used by Rust, and its a simple header with no circular dependencies. Ticket: OISF#7341
victorjulien
pushed a commit
to victorjulien/suricata
that referenced
this pull request
Feb 17, 2025
Regenerates the `sys.rs` and looks for any difference. Check will fail if there is a difference. Ticket: OISF#7341
victorjulien
pushed a commit
to victorjulien/suricata
that referenced
this pull request
Feb 17, 2025
We don't keep bindgen's autogenerated do not edit line as it contains the bindgen version which could break the CI check for out of date bindings. So add our own do not edit line. Ticket: OISF#7341
victorjulien
pushed a commit
to victorjulien/suricata
that referenced
this pull request
Feb 17, 2025
Also disable bindgen's generated layout tests. They are valid for the platform generating the tests, but may not be valid for other platforms. For example, if the tests are generated on a 64 bit platform the tests will not be valid when run on a 32 bit platform as pointers are a different size. However, the generating bindings are valid for both platform. Ticket: OISF#7341
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to redmine ticket:
https://redmine.openinfosecfoundation.org/issues/2881
Describe changes:
Modifies #7318 with doc addition
suricata-verify-pr: 691
OISF/suricata-verify#691