Malware scan/detection in manifest generation process

[taiki-e]

From my comment in #488 (comment).

IMO we should be doing more indepth analysis of binaries during the creation of the manifests , i.e. doing active vulnerability scans on the binaries.

For Rust crates, we could be checking if binaries were built using https://github.com/rust-secure-code/cargo-auditable and adding a column to the tools markdown table to give these binaries a ✔️ if they are auditable, and using the embedded SBOM to check for vulns. This type of functionality is going to be incorporated into cargo itself - currently rust-lang/rfcs#3553 , so encouraging tool creators to use cargo-auditable will help surface problems that should be considered in the RFC .

For Go, iirc osv-scanner already detects vulns in the binaries.

This is a reasonable idea, but we probably need to think how we should evaluate vulnerabilities when they are discovered. Especially with Rust, there is a tendency for advisories to be submitted for completely unimportant issues...

A simple vulnerability scan will likely just burden maintainers with false positives as above ¹. I think the ideal approach is the one commonly used in recent compromise detection: running the tool for a few hours to days in an isolated environment and detecting any suspicious activity. It might be best to do this in a separate repository in #1663 's cooldown period.

Footnotes

1. Is it reasonable to flag only cases of compromise? We clearly don’t have enough resources to analyze whether every reported vulnerability actually has an impact (if you want me to do that, you’ll have to hire me), and besides, it’s rare for other kind of vulnerabilities to affect only the current version... It is not uncommon for all versions to be flagged. ↩