Skip to content

Add config-driven pipeline CLI; make the example configs executable#9

Merged
DaoyuanLi2816 merged 1 commit into
mainfrom
feat/pipeline-cli
Jun 11, 2026
Merged

Add config-driven pipeline CLI; make the example configs executable#9
DaoyuanLi2816 merged 1 commit into
mainfrom
feat/pipeline-cli

Conversation

@DaoyuanLi2816

Copy link
Copy Markdown
Owner

Closes the gap where examples/configs/*.yaml had no consumer in the codebase:

  • python -m labelbank.run --cfg <yaml>stage: retriever runs the full loop the README describes (zero-shot eval → rank the whole bank → gold_first pool mining → contrastive retrain → re-evaluate, for mining_rounds rounds), saving the adapter, per-split rankings.parquet and metrics.json. stage: reranker consumes that parquet and trains the listwise reranker (completion-only, gold position shuffled). Rounds continue one adapter; the docstring notes the competition's fresh-per-round protocol (measured separately in the upcoming experiment example).
  • Both example configs rewritten to the flat schema the CLI actually parses (reproduce_competition.yaml keeps the original v0 hyperparameters incl. pool 50, NF4, the Eedi instruct prefix, fold-0 eval); a test asserts the shipped configs load.
  • 10 new CPU tests: config defaults/unknown-key rejection/stage validation, generic-format column remapping, seeded holdout and fold splits. Suite: 73 passed.
  • README: install matrix now lists the existing [rerank] extra; pipeline section shows the two CLI one-liners.
  • competition/README.md: public-LB typo in the intro fixed (0.54 → 0.56, matching the two other mentions in the same write-up).

🤖 Generated with Claude Code

python -m labelbank.run --cfg <yaml> now drives the full retriever loop
(zero-shot eval -> rank bank -> mine gold-first pools -> retrain ->
re-evaluate, for mining_rounds rounds; saves adapter, per-split
rankings.parquet and metrics.json) and a reranker stage that consumes
the rankings parquet. Both example configs are rewritten to the CLI
schema they now actually drive. README install matrix gains the
existing [rerank] extra; competition write-up public-LB typo fixed
(0.54 -> 0.56, matching the two other mentions).
@DaoyuanLi2816 DaoyuanLi2816 merged commit 8708888 into main Jun 11, 2026
2 checks passed
@DaoyuanLi2816 DaoyuanLi2816 deleted the feat/pipeline-cli branch June 11, 2026 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant