Classify executables by magic bytes instead of the extensionless heuristic

[bobtista]

Problem

Executable classification currently relies on an extensionless-file heuristic. That heuristic will misfire on ordinary extensionless files — README, LICENSE and similar — treating them as executables.

Proposed approach

Replace the heuristic with magic-byte sniffing:

• Mach-O (macOS)
• ELF (Linux)
• MZ (Windows)

Classify on file content rather than on the absence of an extension.

Acceptance criteria

• Extensionless non-executables (README, LICENSE) are not classified as executables.
• Mach-O, ELF and MZ binaries are correctly identified on all platforms.
• Classification does not depend on file extension.

Notes

Worth landing before the current heuristic trips on real content in the wild.

[coderabbitai]

Coding Plan

Summary

• Introduce a cross-platform IExecutableSignatureDetector service (Core interface + shared-services implementation) that classifies files by reading their leading bytes and matching ELF, Mach-O, and MZ/PE magic-number signatures, mirroring the existing Sha256HashProvider pattern and DI registration.
• Replace the extensionless ContentManifestBuilder.IsExecutableFile heuristic with a call to the new detector, so ManifestFile.IsExecutable is driven by content rather than the absence of an extension.
• Add unit tests covering true binaries (all formats/variants) and extensionless non-executables (README/LICENSE), plus an integration assertion through AddFilesFromDirectoryAsync, to lock in the acceptance criteria.
• The WorkspaceStrategyBase .exe fallback and GitHubInferenceHelper (which operates on remote metadata names, not local content) are intentionally left out of scope.

Design Choices

Design Choice 1: Should magic-byte detection be an injectable service or a static helper?

Options Considered:

1. Injectable service (IExecutableSignatureDetector interface + implementation, DI-registered, constructor-injected into ContentManifestBuilder), mirroring the IFileHashProvider/Sha256HashProvider pattern.
2. Static helper method that opens the file and inspects header bytes, keeping the current in-class private-static shape.

Chosen Option: 1

Rationale: Option 1 — it matches the established file-reading service convention (Sha256HashProvider), keeps the byte-signature logic independently testable/mockable, and cleanly separates I/O from the manifest builder.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Introduce content-based executable detection

This phase adds a new, cross-platform capability that determines whether a file is an executable by inspecting its leading bytes against known binary-format signatures. It lives entirely in shared code (no platform-specific projects) since reading header bytes via FileStream is fully portable in .NET. The design separates a pure signature-matching routine (byte content → bool) from the file-reading wrapper so the recognition logic can be tested without disk I/O.

Task 1: Define magic-byte signature constants

Centralize the canonical binary-format signatures so the detector has a single, documented source of truth.

• Add executable-signature definitions alongside the existing shared constants (e.g., extend GenHub.Core/Constants/ with a new ExecutableSignatures constants type, or reuse IoConstants for a small read-buffer size).
• Cover ELF (7F 45 4C 46 at offset 0), MZ/PE (4D 5A at offset 0), and all Mach-O magics at offset 0: FE ED FA CE, CE FA ED FE, FE ED FA CF, CF FA ED FE, plus the fat/universal variants CA FE BA BE, BE BA FE CA, CA FE BA BF, BF BA FE CA.
• Define the minimal number of header bytes to read (4 bytes is sufficient for all signatures; MZ needs only 2).

Task 2: Add the detector interface in Core

Provide a platform-agnostic abstraction consistent with existing GenHub.Core/Interfaces/Common/ conventions.

• Create IExecutableSignatureDetector in GenHub/GenHub.Core/Interfaces/Common/, exposing a method that takes a file path and returns whether the file's content matches a known executable signature (mirror the sync/async style of IFileHashProvider; a synchronous bool IsExecutable(string filePath) is acceptable given only a few bytes are read).
• Keep the contract intent-focused: "classify by content, independent of extension."

Task 3: Implement the detector in shared services

Implement the interface following the Sha256HashProvider file-reading idiom.

• Create the implementation in GenHub/GenHub/Common/Services/ (e.g., MagicByteExecutableDetector).
• Open the file using the established FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize, useAsync: true) pattern, read up to the leading N header bytes into a small buffer, and gracefully handle files shorter than the required bytes (return false) and files that don't exist (return false).
• Delegate the actual signature comparison to a small pure matcher (a static method taking the header bytes / ReadOnlySpan<byte> and returning bool) that checks against the constants from Task 1.1; keep this matcher easily unit-testable.
• Explicitly do not consider file extension in any branch, satisfying the "classification does not depend on file extension" criterion.

Task 4: Register the detector in DI

Wire the new service into the content pipeline alongside the hash provider.

• Register IExecutableSignatureDetector → implementation as a singleton inside AddCoreServices in GenHub/GenHub/Infrastructure/DependencyInjection/ContentPipelineModule.cs, matching the lifetime/placement used for Sha256HashProvider.

🤖 Prompt for AI agents

Implement a new cross-platform, content-based executable detection capability in
shared code (no platform-specific projects needed), separating pure
signature-matching logic from file-reading I/O for testability.

- Add magic-byte signature constants (e.g., a new `ExecutableSignatures`
constants type in `GenHub.Core/Constants/`, or extend `IoConstants` for a
read-buffer size). Cover: ELF (`7F 45 4C 46` at offset 0), MZ/PE (`4D 5A` at
offset 0), and all Mach-O magics at offset 0 (`FE ED FA CE`, `CE FA ED FE`, `FE
ED FA CF`, `CF FA ED FE`) plus fat/universal variants (`CA FE BA BE`, `BE BA FE
CA`, `CA FE BA BF`, `BF BA FE CA`). Define the minimal header byte count needed
(4 bytes covers all; MZ needs only 2).
- Create interface `IExecutableSignatureDetector` in
`GenHub/GenHub.Core/Interfaces/Common/`, exposing a method taking a file path
and returning whether the file's content matches a known executable signature.
Mirror the sync/async style of `IFileHashProvider`; a synchronous `bool
IsExecutable(string filePath)` is acceptable. The contract should be
intent-focused: classify by content, independent of extension.
- Implement the interface in `GenHub/GenHub/Common/Services/` (e.g.,
`MagicByteExecutableDetector`), following the `Sha256HashProvider` file-reading
idiom: open the file via `FileStream(path, FileMode.Open, FileAccess.Read,
FileShare.Read, bufferSize, useAsync: true)`, read up to the leading N header
bytes, and return `false` gracefully for files shorter than required bytes or
files that don't exist.
- Delegate signature comparison to a small pure matcher (static method taking
header bytes / `ReadOnlySpan<byte>`, returning `bool`) checked against the Task
1.1 constants, kept easily unit-testable. Do not consider file extension
anywhere in this logic.
- Register `IExecutableSignatureDetector` → its implementation as a singleton
inside `AddCoreServices` in
`GenHub/GenHub/Infrastructure/DependencyInjection/ContentPipelineModule.cs`,
matching the lifetime/placement used for `Sha256HashProvider`.

Phase 2: Replace the heuristic in manifest generation

This phase swaps the misfiring extensionless heuristic for the new content-based detector at the single place where it is applied, without altering the surrounding ManifestFile creation flow or the public AddFilesFromDirectoryAsync behavior beyond correcting classification.

Task 1: Consume the detector in ContentManifestBuilder

Route executable classification through the new content-based detector.

• Update ContentManifestBuilder (GenHub/GenHub/Features/Manifest/ContentManifestBuilder.cs) constructor to accept IExecutableSignatureDetector via injection (alongside the existing IFileHashProvider/IManifestIdService).
• Remove the private static IsExecutableFile(string filePath) extensionless heuristic.
• In AddFilesFromDirectoryAsync, replace IsExecutable = isExecutable || IsExecutableFile(filePath) with IsExecutable = isExecutable || _executableSignatureDetector.IsExecutable(filePath), preserving the explicit caller-supplied isExecutable override and the file-on-disk gating.
• Keep the UnixPermissions assignment behavior as-is unless the content detector result should drive it; note that the pre-existing inconsistency (permissions derived only from the explicit isExecutable parameter) is out of scope for this ticket.

Task 2: Update DI/consumer wiring as needed

Ensure all construction sites of ContentManifestBuilder receive the new dependency.

• Verify the DI registration/factory for ContentManifestBuilder (and any callers such as ManifestProvider/ManifestGenerationService paths) resolves the newly required IExecutableSignatureDetector; adjust any manual construction accordingly.

🤖 Prompt for AI agents

Replace the extensionless-file heuristic used in manifest generation with the
new content-based detector, at the single application site, without changing
surrounding behavior beyond correcting classification.

- Update `ContentManifestBuilder`
(`GenHub/GenHub/Features/Manifest/ContentManifestBuilder.cs`) constructor to
accept `IExecutableSignatureDetector` via dependency injection, alongside the
existing `IFileHashProvider`/`IManifestIdService`.
- Remove the private static `IsExecutableFile(string filePath)` extensionless
heuristic entirely.
- In `AddFilesFromDirectoryAsync`, replace `IsExecutable = isExecutable ||
IsExecutableFile(filePath)` with `IsExecutable = isExecutable ||
_executableSignatureDetector.IsExecutable(filePath)`, preserving the explicit
caller-supplied `isExecutable` override and the existing file-on-disk gating.
- Leave `UnixPermissions` assignment behavior unchanged unless the content
detector result should drive it; the pre-existing inconsistency where
permissions are derived only from the explicit `isExecutable` parameter is out
of scope for this ticket.
- Verify the DI registration/factory for `ContentManifestBuilder` (and any
callers such as `ManifestProvider`/`ManifestGenerationService` paths) correctly
resolves the newly required `IExecutableSignatureDetector` dependency, adjusting
any manual construction sites as needed.

Phase 3: Tests

This phase pins down the corrected behavior with unit tests covering both true executables and the previously-misclassified extensionless non-executables.

Task 1: Add detector unit tests

Directly validate signature recognition across all target formats and negative cases.

• Create a test class under GenHub/GenHub.Tests/GenHub.Tests.Core/Common/Services/ for the new detector, following the inline temp-file idiom (Path.GetTempFileName() + File.WriteAllBytes + try/finally) already used in DownloadServiceTests.
• Cover positive cases: ELF, MZ/PE, and each Mach-O magic variant (32/64-bit, byte-swapped, fat/universal).
• Cover negative cases: extensionless text files with README/LICENSE-style content, and short files (fewer than the required header bytes), asserting false.
• Optionally test the pure matcher directly with byte arrays to keep recognition assertions independent of disk I/O.

Task 2: Extend ContentManifestBuilder tests

Confirm end-to-end classification through the manifest builder.

• Update ContentManifestBuilderTests construction to supply a (real or mocked) IExecutableSignatureDetector.
• Extend AddFilesFromDirectoryAsync_SetsCorrectInstallTargets (or add a focused test) to include an extensionless README-style file and an extensionless file containing ELF/MZ magic bytes, asserting IsExecutable == false and == true respectively — directly validating the acceptance criteria.

🤖 Prompt for AI agents

Add unit tests to lock in the corrected classification behavior for both the new
detector and its consumption within `ContentManifestBuilder`.

- Create a test class under
`GenHub/GenHub.Tests/GenHub.Tests.Core/Common/Services/` for the new
`IExecutableSignatureDetector` implementation, following the inline temp-file
idiom (`Path.GetTempFileName()` + `File.WriteAllBytes` + `try/finally`) already
used in `DownloadServiceTests`.
- Cover positive cases: ELF, MZ/PE, and each Mach-O magic variant (32/64-bit,
byte-swapped, fat/universal).
- Cover negative cases: extensionless text files with `README`/`LICENSE`-style
content, and short files (fewer than the required header bytes), asserting
`false` is returned.
- Optionally add tests for the pure matcher directly using byte arrays, to keep
recognition assertions independent of disk I/O.
- Update `ContentManifestBuilderTests` construction to supply a (real or mocked)
`IExecutableSignatureDetector`.
- Extend `AddFilesFromDirectoryAsync_SetsCorrectInstallTargets` (or add a
focused new test) to include an extensionless `README`-style file and an
extensionless file containing ELF/MZ magic bytes, asserting `IsExecutable ==
false` for the README-style file and `IsExecutable == true` for the magic-byte
file — directly validating the acceptance criteria.

Research

GenHub is a .NET solution split into a platform-agnostic GenHub.Core library (interfaces, models, constants), a shared GenHub application layer (feature services), platform-specific projects (GenHub.Windows/GenHub.Linux), and a GenHub.Tests suite. Executable classification currently lives in a private static helper ContentManifestBuilder.IsExecutableFile(string filePath) (in GenHub/GenHub/Features/Manifest/ContentManifestBuilder.cs), which treats a file as executable when its extension is .exe/.dll/.so or empty and the file exists on disk. Its result is OR-ed into ManifestFile.IsExecutable inside AddFilesFromDirectoryAsync. Byte-level file reading already has an established pattern in Sha256HashProvider (interface in GenHub.Core/Interfaces/Common/, implementation in GenHub/Common/Services/, registered as a singleton in ContentPipelineModule.AddCoreServices). No magic-byte/file-signature detection exists anywhere today.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Implement a new cross-platform, content-based executable detection capability in
shared code (no platform-specific projects needed), separating pure
signature-matching logic from file-reading I/O for testability.

- Add magic-byte signature constants (e.g., a new `ExecutableSignatures`
constants type in `GenHub.Core/Constants/`, or extend `IoConstants` for a
read-buffer size). Cover: ELF (`7F 45 4C 46` at offset 0), MZ/PE (`4D 5A` at
offset 0), and all Mach-O magics at offset 0 (`FE ED FA CE`, `CE FA ED FE`, `FE
ED FA CF`, `CF FA ED FE`) plus fat/universal variants (`CA FE BA BE`, `BE BA FE
CA`, `CA FE BA BF`, `BF BA FE CA`). Define the minimal header byte count needed
(4 bytes covers all; MZ needs only 2).
- Create interface `IExecutableSignatureDetector` in
`GenHub/GenHub.Core/Interfaces/Common/`, exposing a method taking a file path
and returning whether the file's content matches a known executable signature.
Mirror the sync/async style of `IFileHashProvider`; a synchronous `bool
IsExecutable(string filePath)` is acceptable. The contract should be
intent-focused: classify by content, independent of extension.
- Implement the interface in `GenHub/GenHub/Common/Services/` (e.g.,
`MagicByteExecutableDetector`), following the `Sha256HashProvider` file-reading
idiom: open the file via `FileStream(path, FileMode.Open, FileAccess.Read,
FileShare.Read, bufferSize, useAsync: true)`, read up to the leading N header
bytes, and return `false` gracefully for files shorter than required bytes or
files that don't exist.
- Delegate signature comparison to a small pure matcher (static method taking
header bytes / `ReadOnlySpan<byte>`, returning `bool`) checked against the Task
1.1 constants, kept easily unit-testable. Do not consider file extension
anywhere in this logic.
- Register `IExecutableSignatureDetector` → its implementation as a singleton
inside `AddCoreServices` in
`GenHub/GenHub/Infrastructure/DependencyInjection/ContentPipelineModule.cs`,
matching the lifetime/placement used for `Sha256HashProvider`.
===============================================================================

Task: 2

Replace the extensionless-file heuristic used in manifest generation with the
new content-based detector, at the single application site, without changing
surrounding behavior beyond correcting classification.

- Update `ContentManifestBuilder`
(`GenHub/GenHub/Features/Manifest/ContentManifestBuilder.cs`) constructor to
accept `IExecutableSignatureDetector` via dependency injection, alongside the
existing `IFileHashProvider`/`IManifestIdService`.
- Remove the private static `IsExecutableFile(string filePath)` extensionless
heuristic entirely.
- In `AddFilesFromDirectoryAsync`, replace `IsExecutable = isExecutable ||
IsExecutableFile(filePath)` with `IsExecutable = isExecutable ||
_executableSignatureDetector.IsExecutable(filePath)`, preserving the explicit
caller-supplied `isExecutable` override and the existing file-on-disk gating.
- Leave `UnixPermissions` assignment behavior unchanged unless the content
detector result should drive it; the pre-existing inconsistency where
permissions are derived only from the explicit `isExecutable` parameter is out
of scope for this ticket.
- Verify the DI registration/factory for `ContentManifestBuilder` (and any
callers such as `ManifestProvider`/`ManifestGenerationService` paths) correctly
resolves the newly required `IExecutableSignatureDetector` dependency, adjusting
any manual construction sites as needed.
===============================================================================

Task: 3

Add unit tests to lock in the corrected classification behavior for both the new
detector and its consumption within `ContentManifestBuilder`.

- Create a test class under
`GenHub/GenHub.Tests/GenHub.Tests.Core/Common/Services/` for the new
`IExecutableSignatureDetector` implementation, following the inline temp-file
idiom (`Path.GetTempFileName()` + `File.WriteAllBytes` + `try/finally`) already
used in `DownloadServiceTests`.
- Cover positive cases: ELF, MZ/PE, and each Mach-O magic variant (32/64-bit,
byte-swapped, fat/universal).
- Cover negative cases: extensionless text files with `README`/`LICENSE`-style
content, and short files (fewer than the required header bytes), asserting
`false` is returned.
- Optionally add tests for the pure matcher directly using byte arrays, to keep
recognition assertions independent of disk I/O.
- Update `ContentManifestBuilderTests` construction to supply a (real or mocked)
`IExecutableSignatureDetector`.
- Extend `AddFilesFromDirectoryAsync_SetsCorrectInstallTargets` (or add a
focused new test) to include an extensionless `README`-style file and an
extensionless file containing ELF/MZ magic bytes, asserting `IsExecutable ==
false` for the README-style file and `IsExecutable == true` for the magic-byte
file — directly validating the acceptance criteria.

💡 Iterate on the plan with: `@coderabbitai` <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For design choice 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!