test: mutation testing for Nostr event handling - #849
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
🚧 Files skipped from review as they are similar to previous changes (4)
WalkthroughMutation testing now runs through a shared Makefile target with configurable test ports. Event acceptance checks use private helpers with expanded test coverage. Expiration-tag and global spam-gate behavior receive regression tests. ChangesNostr testing improvements
Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: ⚪ Minimal · up to This change adds mutation-testing coverage and test-environment controls without any identified merge-blocking correctness or production risk; it is merge-ready after normal checks and review. Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Makefile`:
- Around line 69-71: Update the mutation-test target to avoid the Bash-only set
-o pipefail under Make’s default shell, and preserve caller configuration by not
unconditionally overwriting MOSTRO_TEST_LN_PORT. Also prevent parallel mutation
workers from sharing the same fixed port by guarding concurrency or assigning
distinct worker ports while retaining configurable overrides.
In `@src/nip33.rs`:
- Around line 1121-1143: Add a companion test alongside
create_event_does_not_duplicate_a_caller_supplied_expiration_tag that supplies a
custom "expiration" tag through new_order_event, then assert the resulting order
contains no auto-added standard TagKind::Expiration tag. Keep the existing
standard-tag test unchanged and verify the custom branch in create_event's
has_expiration_tag logic.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: b2a6afb7-fe1d-4339-8c3c-1635a3e9717d
📒 Files selected for processing (7)
.github/workflows/mutation.ymlMakefilesrc/app.rssrc/lightning/invoice.rssrc/lnurl.rssrc/nip33.rssrc/spam_gate.rs
Both review comments addressed — one of them turned out to be the opposite of what it looked likePushed
|
`.cargo/mutants.toml` (added in 87b2b6f, this PR) set additional_cargo_test_args = ["--test-threads=4"] cargo-mutants places those args before `cargo test`'s own `--`, so cargo rejects the flag rather than forwarding it to libtest: *** cargo test --verbose --package=mostro@0.18.0 --test-threads=4 error: unexpected argument '--test-threads' found *** result: Failure(1) ERROR cargo test failed in an unmutated tree, so no mutants were tested The baseline never passed, so no mutant was ever tested — via the Makefile target or the CI job, since cargo-mutants reads this file regardless of how it is invoked. Intended as an OOM guard, it silently disabled the thing it was guarding. No config-file or CLI mechanism in cargo-mutants 27.1.0 forwards arguments past that `--`, and `CARGO_MUTANTS_JOBS` is the cap that actually binds. Removing the file restores the baseline: the suite now runs to completion (1021 passed locally, the one failure being the known hardcoded-8080 `AddrInUse` flake that PR MostroP2P#849 fixes).
Uncapped parallel jobs + per-test thread fan-out exhausted RAM and crashed the machine during a local run. Cap via CARGO_MUTANTS_JOBS=2 (Makefile, verified with strace since .cargo/config.toml's [env] does not propagate to third-party subcommands) and --test-threads=4 (.cargo/mutants.toml). Both CI mutation jobs now go through the same `make mutation-test` target.
cargo-mutants requires a fully green baseline before mutating anything, and the lightning-address test path was hardcoded to 127.0.0.1:8080 in both the test server and lnurl.rs's cfg!(test) URL builder. On a machine already using 8080 that baseline never passes. MOSTRO_TEST_LN_PORT (env, default 8080) lets `make mutation-test` point both sides at a free port without changing default `cargo test` behavior. Also drops --test-threads=4 from the mutation-test target: cargo-mutants 27.1.0 has no working way to forward libtest args (config key, CLI flag, and trailing -- args all insert before cargo test's own --), so the flag only broke the baseline. CARGO_MUTANTS_JOBS=2 remains the real OOM cap.
Issue MostroP2P#636 (mutation testing for Nostr event handling) flagged accept_event()'s post-unwrap validation as uncovered: the replay-window timestamp check and the identity/signature check had zero mutants caught. Extracted both into pure functions (is_stale, missing_inner_signature) so cargo-mutants' boundary mutants can be hit directly, and added three accept_event-level tests built on a real wrap_message_with-signed GiftWrap event (happy path, wrong kind, wrong receiver) to cover the surrounding POW/kind/verify gates. Mutation run on src/app.rs + src/nip33.rs (partial, 90/108 mutants tested before this checkpoint): every accept_event mutant now caught except the 3 in the is_v2 spam-gate branch, deliberately deferred since exercising it needs the SpamGate global singleton initialized. Overall score and the rest of app.rs's dispatcher/warning_msg mutants still outstanding — see [[project_july_contribution_plan]] memory for the resume point.
…mutants Targets the remaining surviving mutants on the Nostr event-handling path for issue MostroP2P#636: - accept_event's protocol-v2 spam-gate / PoW-first-contact branch (3 mutants around the `!gate.is_known(..) && !event.check_pow(..)` check) was never exercised by is_v2=true — add tests for the accepted and rejected first-contact cases. - nip33::create_event's NIP-40 expiration-tag dedup check (`||` between the canonical and custom expiration tag kinds) was only exercised via the "no existing tag" paths — add a test that pre-supplies a real expiration tag and asserts it isn't duplicated. spam_gate.rs: fix a latent test-order bug the new accept_event tests exposed — install_global_then_second_install_is_rejected assumed it would always be the first test in the binary to install the process-wide SpamGate OnceLock, which broke once another test raced it there. 10/10 targeted mutants confirmed killed via a `-F`-filtered cargo-mutants run scoped to accept_event/create_event.
`has_expiration_tag` tested two shapes, `TagKind::Expiration` and
`TagKind::Custom("expiration")`. The second is unreachable: nostr
normalises the tag name at construction, so `Tag::custom` built with
the name "expiration" — exactly what `order_to_tags` emits for every
order event — already arrives as `TagKind::Expiration`. Verified by
probing `Tag::kind()` directly before removing it.
Being unreachable, it was also an equivalent mutant: deleting the arm
changes nothing observable, so no test could ever kill it.
Add a test that pins the real production path end to end — a
custom-named "expiration" tag must normalise and suppress the auto-add
— so an sdk upgrade that stopped normalising goes red here instead of
silently double-stamping every order event.
Every cargo-mutants worker runs the full suite in its own temp dir but shares the host's TCP ports, so two workers collide on the LNURL test's fixed listener. That collision fails the test for its own reasons — and under mutation testing a failing test counts as a killed mutant, so the collision silently inflates the score this target exists to measure. Run a single worker, and stop hard-coding the port over a caller's own `MOSTRO_TEST_LN_PORT` so a busy 18080 can be worked around without editing the Makefile. Leaves `set -o pipefail` alone: `SHELL := $(shell which bash)` on line 1 already applies to every recipe, and six existing targets rely on it.
e885508 to
541129f
Compare
Rebased onto
|
`.cargo/mutants.toml` (added in 87b2b6f, this PR) set additional_cargo_test_args = ["--test-threads=4"] cargo-mutants places those args before `cargo test`'s own `--`, so cargo rejects the flag rather than forwarding it to libtest: *** cargo test --verbose --package=mostro@0.18.0 --test-threads=4 error: unexpected argument '--test-threads' found *** result: Failure(1) ERROR cargo test failed in an unmutated tree, so no mutants were tested The baseline never passed, so no mutant was ever tested — via the Makefile target or the CI job, since cargo-mutants reads this file regardless of how it is invoked. Intended as an OOM guard, it silently disabled the thing it was guarding. No config-file or CLI mechanism in cargo-mutants 27.1.0 forwards arguments past that `--`, and `CARGO_MUTANTS_JOBS` is the cap that actually binds. Removing the file restores the baseline: the suite now runs to completion (1021 passed locally, the one failure being the known hardcoded-8080 `AddrInUse` flake that PR MostroP2P#849 fixes).
1 participant
Context
Follow-up from #618. Implements mutation testing for Nostr event handling — the communication layer between Mostro and its clients. Closes #636.
What changed
Makefile/.github/workflows/mutation.yml: added amutation-testtarget (CARGO_MUTANTS_JOBS=1 MOSTRO_TEST_LN_PORT=$${MOSTRO_TEST_LN_PORT:-18080} cargo mutants) so a single knob caps concurrency (avoids OOM on constrained machines) and lets the local LN test port be overridden when 8080 is already taken on the host. Single worker on purpose: workers share the host's TCP ports, and a worker losing the race for the LNURL listener fails that test for its own reasons — which mutation testing scores as a killed mutant, inflating the result.src/lnurl.rs,src/lightning/invoice.rs: threadedMOSTRO_TEST_LN_PORTthrough the local test HTTP server/URL builder socargo-mutantsruns don't collide with something else already bound to 8080.src/app.rs:is_staleandmissing_inner_signatureout ofaccept_eventwith direct boundary tests.accept_event_testscovering the full accept/reject paths (valid gift wrap, wrong kind, wrong receiver) plus the protocol-v2 spam-gate / PoW-first-contact branch (accepted when the bar is cleared, dropped when it isn't) — this branch had 3 surviving mutants with zero coverage.src/nip33.rs: added tests forcreate_event's NIP-40 expiration-tag dedup check — a caller-supplied expiration tag must not be duplicated by the auto-expiration logic, which was the source of a surviving||→&&mutant. Also removed that check'sTagKind::Custom("expiration")arm: nostr normalises the tag name at construction, so it was unreachable — and therefore an equivalent mutant no test could ever kill. The added tests pin the real path instead, including the exactTag::customshapeorder_to_tagsemits.src/spam_gate.rs: fixedinstall_global_then_second_install_is_rejected, which assumed it would always be the first test in the binary to install the process-wideSpamGateOnceLock— the newaccept_eventspam-gate tests expose that the assumption doesn't hold once another test races it there. Now robust to install order, still asserts a second install is always rejected.Verification
9/9mutants confirmed killed viamake mutation-test ARGS="--file src/app.rs --file src/nip33.rs -F 'in accept_event|in create_event'"— 0 missed, 0 timeout, 0 unviable. (7 in the two target functions, plus 2 incheck_trade_indexthatcargo-mutants 27.1.0admits because-Fdoes not filter "delete field from struct expression" mutants.) Reads 9 rather than 10 because dropping the unreachable arm above also drops its||→&&mutant from the set — nothing became uncovered.cargo fmt --check,cargo clippy --all-targets --all-features -- -D warnings: clean.cargo test: 1059 passed, 2 ignored (withMOSTRO_TEST_LN_PORTpointed past a host process already holding 8080).Acceptance Criteria (from #636)
accept_event's spam-gate/PoW branch)create_event)Summary by CodeRabbit
Bug Fixes
Reliability