Add report linearization

[Liam-DeVoe]

Part of #3 (comment)

[Liam-DeVoe]

• Linearized-history view is currently the only dashboard view, option to select runs-in-last-n-days coming later
• Big PR; important bits are def linearize_reports and the dataclasses in database.py

[Zac-HD]

looking good!

[Zac-HD]

Let's decompose this: generate a worker_and_run() or something, and then have reports() draw a list of those. We can pass in a list of (db_key, nodeid) pairs for each report to sample from; and have the multi-worker reports strategy add whatever offset we need after the fact if overlap=False (noting that we probably want a smaller upper bound on timestamps).

[Liam-DeVoe]

ok if I come back to this one? I definitely want a strong strategy here, but also want to write the overlapping case first before making it smarter

[Zac-HD]

lgtm

[Zac-HD]

My bad for the hasty review, seriously, but this implementation is wrong once you have a restart - sorting by elapsed time will interleave all the runs, and then crash with assertion errors.

Having played around a bit in a branch (https://github.com/Zac-HD/hypofuzz/compare/zac/linearize), I think the solution is to change REPORTS to be a nested dictionary: nodeid -> worker_uuid -> list[Report] sorted by timestamp

• With that structure, it's easy to derive the diffs just be iterative over the elements of each list
  • then linearize by concatenating all the per-worker-uuid lists and sorting by typestamp, and dropping not-at-the-start replay entries.
  • I actually think that computing diffs in the dashboard server kinda sucks; either we should take the space hit and put that in the database (ie denormalize a bit; our worker-identity mapping is already heavy-ish), or do it in the frontend.
• we need all those individual lists anyway, since we want an option to plot them separately on the per-test pages
• also, in every place we construct a Report or a Metadata loaded from the database, we need to handle 'parsing errors' due to invalid json, or missing/extra keys due to writes from an older or newer fuzzer version.
  • gosh that's going to be annoying, ugh, at least it's not too many places...
  • I think we just skip over those entries; I'm cautious about deleting stuff (imagine you update the fuzzer, have to roll back, and in the meantime we dropped all your metadata...) - we can do something sensible later.