Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Add support for FTW-tests #159

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

superstes
Copy link

@superstes superstes commented Jan 5, 2025

Greetings!

This PR adds support to run the FTW-Tests as mentioned in #69.

Basically I've implemented the same as seen in the coraza-caddy repository.

When ran manually they work.
But there are ~300 errors. Not quite sure why that is.
With CRS 4.5.0 instead of 4.7.0 there are ~50 fewer.

ftw/Dockerfile.ftw Outdated Show resolved Hide resolved
magefile.go Outdated Show resolved Hide resolved
.github/workflows/test.yaml Show resolved Hide resolved
@fionera
Copy link
Contributor

fionera commented Jan 5, 2025

Looks very nice 👍 Thank you very much! Are the error counts the same as with the caddy connector? If yes, then its fine I would guess

@superstes
Copy link
Author

superstes commented Jan 6, 2025

No.. caddy only has 69 failing/ignored ones.

I've had overlooked that there is an 'albedo' test-backend in use. This could result in errors as the tests may expect a specific response in some cases..

But when I enable it, spoa fails on the first request => Nevermind. Response checking was not enabled

@superstes
Copy link
Author

superstes commented Jan 6, 2025

Now the albedo backend is active - but spoa fails without any error when I try to enable response-checking.

HAProxy Logs

SPOE: [coraza-agent] <GROUP:coraza-req> sid=0 st=0 2 0/1
SPOE: [coraza-agent] <EVENT:on-http-response> sid=0 st=1 501 1/2
172.18.1.5:35708 [06/Jan/2025:12:05:48.911] default test/test 0/0/2/2/0/1/1/505 500 290 - - PH-- 1/1/0/0/0 0/0 "GET / HTTP/1.1" TBCYNHCNZSRZKSBL spoa-error: 1 waf-hit: - waf-action: -

SPOA Logs

�[90m12:05PM�[0m �[32mINF�[0m �[1mStarting coraza-spoa�[0m
�[90m12:06PM�[0m �[32mINF�[0m �[1mReceived SIGTERM, shutting down...�[0m

@superstes
Copy link
Author

superstes commented Jan 8, 2025

Yey - have found the issue. @fionera
With HAProxy versions <3.1 it works.. seems to be a side-effect of the v3.1 SPOE changes to HAProxy.
Will add a separate PR for the multi-version testing that cloud already have catched this issue: #160

@superstes
Copy link
Author

We still have 234 failed tests (in comparison to 69 of caddy).
But most of the tests seem to work fine. Maybe we should address those errors in follow-up issues (?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants