Skip to content

feat: CLI interface and unit tests#2

Open
Rakshitha-Ireddi wants to merge 2 commits into
sarvamai:mainfrom
Rakshitha-Ireddi:feat/cli-and-tests
Open

feat: CLI interface and unit tests#2
Rakshitha-Ireddi wants to merge 2 commits into
sarvamai:mainfrom
Rakshitha-Ireddi:feat/cli-and-tests

Conversation

@Rakshitha-Ireddi

Copy link
Copy Markdown

Summary

  • CLI interface (cli.py + pyproject.toml entry point): Users can now run the tool directly from the terminal
    without editing Python source files.
  • Unit tests (tests/): 30 tests across test_utilities.py and test_main.py covering all core logic with zero
    heavy model dependencies.

CLI usage

Gemini (default):

llm-wer \
  --dataset predictions.csv \                                                                                         
  --ref-col transcription \                               
  --pred-col prediction \                                                                                             
  --audio-col audio_filepath \
  --creds-path service_account.json                                                                                   

Any OpenAI-compatible API:

  llm-wer \
    --dataset predictions.csv \
    --ref-col transcription \                                                                                           
    --pred-col prediction \  
    --audio-col audio_filepath \                                                                                        
    --no-gemini \                                           
    --model gpt-4o \
    --api-key sk-... \
    --base-url https://api.openai.com/v1                                                                                

With Google Sheets output:
llm-wer ... --sheet-name "LLM WER Analysis" --worksheet-name "run-1"

Install with: pip install -e ".[dev]" then run llm-wer --help.

Tests

tests/
├── test_utilities.py # wer(), cer() — edge cases, custom weights, clamping
└── test_main.py # get_segments(), build_prompt(), process_llm_responses(),
# reconstruct_and_score(), load_and_validate_dataset()

Run with: pytest (no heavy model downloads required — normalizer/LLM calls are not exercised).

Test plan

  • pip install -e ".[dev]" && pytest — all 30 tests pass
  • llm-wer --help — shows full usage
  • llm-wer --no-gemini --model gpt-4o --api-key ... --base-url ... — routes to OpenAI
  • Omitting --sheet-name — no Sheets upload attempted

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant