Skip to content

Conversation

Copy link

Copilot AI commented Dec 30, 2025

The repository lacked a command-line tool for running benchmarks. Users had no way to benchmark pipelines without the web UI.

Changes

New CLI tool (open_bench.py)

  • list: Enumerate available pipelines with status
  • benchmark: Find max stream count at target FPS with configurable device/model/parameters
  • optimize: Grid search over parameter combinations

Documentation (docs/user-guide/how-to-run-open-bench.md)

  • Setup, usage examples, troubleshooting
  • Docker and scripting workflows

Updated README.md

  • Added CLI benchmarking to key features
  • Linked to documentation

Example Usage

# List pipelines
python open_bench.py list

# Benchmark on CPU
python open_bench.py benchmark \
    --pipeline smartnvr \
    --video /path/to/video.mp4 \
    --device CPU \
    --fps-floor 30

# Optimize parameters on GPU
python open_bench.py optimize \
    --pipeline smartnvr \
    --video /path/to/video.mp4 \
    --device GPU \
    --batch-sizes 1 2 4 8 \
    --nireqs 1 2 4

The tool integrates with existing Benchmark and PipelineOptimizer classes, using pipeline configs from config.yaml for model defaults.

Original prompt

how to run this open_bench.py


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI self-assigned this Dec 30, 2025
Copilot AI changed the title [WIP] Add instructions for running open_bench.py Add open_bench.py CLI for pipeline benchmarking Dec 30, 2025
Copilot AI requested a review from madhuri-rai07 December 30, 2025 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants