feat: add Hugging Face dataset pull/push provider

[kaaloo]

Problem

eval-transcript keeps benchmark audio and ground-truth transcripts in data/ground_truth/ (post-rename), which is gitignored for artifacts. This blocks two things: sharing benchmark corpora across machines/teammates, and running evaluations from CI/CD or Scaleway batch jobs, where a worker cannot git pull the corpus and has nowhere to push results.

Hugging Face Hub is a good fit: it speaks datasets, gated/private repos, tokens, revisions, and is a natural primitive for a benchmark corpus. The team already uses huggingface_hub as a dependency in the wider Hugging Face ecosystem, and the issue-first standard calls for documenting this before the implementation PR.

Scope

Add a new huggingface provider module and subcommands for data I/O only (no ASR inference — that stays with oMLX, Albert, Scaleway, ElevenLabs). The provider pulls and pushes the benchmark corpus via a single HF Dataset repo.

Subcommands

• eval-transcript huggingface dataset ls — list sample IDs in the configured corpus repo. Anonymous read when the repo is public.
• eval-transcript huggingface dataset pull [--samples a,b,c] [--all] [--force] [--revision REV] — write data/audio/<id>.<ext> and data/ground_truth/<id>.md|txt from the dataset rows; idempotent (skip files already up to date with the pulled revision). --force overwrites local files.
• eval-transcript huggingface dataset push [--message MSG] — upload the local data/audio/ + data/ground_truth/ to the configured dataset repo, creating it with private=True by default. Refuses to push to a public repo with a hard error (no override flag; see Constraints).

Layout (single parquet, with audio as an Audio column)

Single HF Dataset config, one parquet file. Schema (one row per sample):

column	type	notes
sample_id	string	also the parquet filename stem; matches data/audio/<id>.<ext>
audio	Audio	bytes + sample_rate + path metadata; HF Datasets' Audio() feature
ground_truth	string	full reference text
language	string	optional, e.g. "fr"
notes	string	optional free text

The parquet is the single source of truth. Pulls stream rows and decode audio bytes to the on-disk file. Pushes read data/audio/* and data/ground_truth/*, build a single Dataset.from_dict(...) and push_to_hub.

Auth model

• Constructor accepts token: str | None; defaults to os.getenv("HF_TOKEN").
• Reads (ls, pull) work anonymously when the repo is public.
• Writes (push) require HF_TOKEN; the constructor fails fast if it is missing.
• push checks HfApi.repo_info(...).private; raises HuggingFaceError("Refusing to push to public dataset repo: <name>") if False. No override flag. Public benchmark data leakage is a real risk for an institutional benchmark.
• push creates the repo with private=True by default. The constructor accepts visibility: Literal["private"]; only private is accepted in this PR.

Configuration (.env / load_dotenv)

• HF_TOKEN — required for write.
• HF_ORG — optional; defaults to the token user.
• HF_DATASET_REPO — defaults to eval-transcript-corpus.
• All three are overridable via CLI flags (--token, --org, --repo) for CI.

Acceptance criteria

• eval-transcript huggingface dataset ls lists sample IDs in the configured repo without requiring HF_TOKEN (when the repo is public).
• eval-transcript huggingface dataset pull --samples a,b writes data/audio/a.<ext> and data/ground_truth/a.md|txt from the dataset rows; existing files are skipped unless --force is passed.
• eval-transcript huggingface dataset push creates the repo with private=True if it does not exist, then uploads the local data/audio/ + data/ground_truth/.
• push raises a domain error (HuggingFaceError) and exits 1 if the target repo is public.
• Constructor fails fast with a clear message if HF_TOKEN is missing on a write command.
• Unit tests use a mocked huggingface_hub.HfApi and cover: anonymous list, token-required write, public-repo rejection, missing-token error, idempotent pull, --force overwrite.
• README "Hugging Face dataset" section with a 5-line example (.env, pull, push).
• .env.example updated.
• uv run pre-commit run --all-files is clean.
• uv run python -m unittest discover -s tests -v is clean.

Constraints

• No new runtime dependencies beyond huggingface_hub (already available in the ecosystem; if it is not yet a core dep, add it to pyproject.toml).
• Reuse the existing constructor fail-fast pattern from OmlxClient / AlbertClient.
• Reuse the transcription_output_path convention only by analogy; this provider does not write to data/transcriptions/.
• Do not introduce ASR inference through the HF provider. PR Replace Parakeet with Kyutai STT as the local benchmark model #22 dropped HF Parakeet because the Inference Providers path was dead. eval-transcript huggingface is read/write only.
• data/audio/ and data/ground_truth/ content is treated as binary/text by content hash, not by line diff, for idempotency decisions.

Out of scope

• ASR inference through HF Inference Providers.
• Results ledger (separate issue).
• Manifest-aware aggregate commands (separate issue).
• CI workflow (separate issue).

Depends on

• chore: rename source_truth to ground_truth with soft deprecation (issue chore: rename source_truth to ground_truth with soft deprecation #27). This issue references data/ground_truth/, not data/source_truth/.

👾 Generated with Letta Code