Skip to content

Conversation

a-nogikh
Copy link
Collaborator

@a-nogikh a-nogikh commented Aug 27, 2025

Closes #5813.

In the presence of numerous program seeds that mount fs images, our html
coverage reports end up being as big as 1GB, which makes it problematic
to render/manipulate them in the web browser.

Adjust coverage report generation: once the total size of program seeds
exceeds 100MB, switch to the compact seed serialization mode.


@sirdarckcat, do you still have workflows that rely on parsing coverage.html files from syzkaller?

Refactor Prog.Serialize() to accept a variadic list of flags.

For now, two are supported:
1) Verbose (equal to SerializeVerbose()).
2) SkipImages (don't serialize fs images).
@a-nogikh a-nogikh requested a review from dvyukov August 27, 2025 10:25
@a-nogikh a-nogikh changed the title pkg/manager: omit fs images from big coverage reports prog, pkg/manager: omit fs images from big coverage reports Aug 27, 2025
@sirdarckcat
Copy link
Member

Oh yea. Maybe just drop an export file (not meant for rendering) somewhere? Also, we don't need the source code, which I assume is most of the size (just the lines).

@a-nogikh
Copy link
Collaborator Author

We have the /coverprogs output added in #5798 that exports per-corpus program coverage (since it's per-program, it's even more detailed that these cover.html reports), and these are also regularly uploaded to GCS. Could that be a replacement?

In the presence of numerous program seeds that mount fs images, our html
coverage reports end up being as big as 1GB, which makes it problematic
to render/manipulate them in the web browser.

Adjust coverage report generation: once the total size of program seeds
exceeds 100MB, switch to the compact seed serialization mode.
@a-nogikh a-nogikh force-pushed the features/omit-fs-images branch from 68fb260 to c21e1dc Compare August 27, 2025 11:12
@sirdarckcat
Copy link
Member

If they are uploaded to GCS for all programs then yeah that's awesome

@dvyukov
Copy link
Collaborator

dvyukov commented Sep 22, 2025

Oh yea. Maybe just drop an export file (not meant for rendering) somewhere? Also, we don't need the source code, which I assume is most of the size (just the lines).
If they are uploaded to GCS for all programs then yeah that's awesome

They are now exported as jsonl w/o source code (#5798)
And uploaded to GCS to URLs like (for each instance):
https://storage.googleapis.com/syzkaller/programs-cover/ci-upstream/ci-upstream-kasan-gce.jsonl

@sirdarckcat is it enough for @a-nogikh to proceed with this change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pkg/cover: cover.html is too big
3 participants