Skip to content

[DOC] align-max-concurrent-trials-docs #125

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
thameem-abbas merged 1 commit into from
Mar 2, 2026
Merged

[DOC] align-max-concurrent-trials-docs #125

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ pip install -e .
#### CLI Interface
```bash
# Run optimization study
auto-tune-vllm optimize --config config.yaml --max-concurrent 2
auto-tune-vllm optimize --config config.yaml --max-concurrent-trials 2

# Stream live logs
auto-tune-vllm logs --study-id 42 --trial-number 15
Expand All @@ -59,9 +59,9 @@ All ML dependencies (vLLM, Ray, GuideLLM, BoTorch) are included automatically.

### Ray Cluster Concurrency Validation

**Issue**: The `--max-concurrent` parameter is not validated against available Ray cluster resources.
**Issue**: The `--max-concurrent-trials` parameter is not validated against available Ray cluster resources.

**Details**: When using Ray backend, the system doesn't check if the requested concurrency level is feasible given the cluster's GPU/CPU resources. For example, setting `--max-concurrent 10` on a cluster with only 4 GPUs will not warn the user that only 4 trials can actually run concurrently.
**Details**: When using Ray backend, the system doesn't check if the requested concurrency level is feasible given the cluster's GPU/CPU resources. For example, setting `--max-concurrent-trials 10` on a cluster with only 4 GPUs will not warn the user that only 4 trials can actually run concurrently.

**Reason**: There is not a clear answer if all the trials would use the exact same number of GPUs. For example, we might have different parallelism related tunings for different trials which might result in different number of GPUs being required for the trial.

Expand All @@ -81,7 +81,7 @@ All ML dependencies (vLLM, Ray, GuideLLM, BoTorch) are included automatically.
auto-tune-vllm check-env --ray-cluster

# Set realistic concurrency (e.g., if you have 4 GPUs)
auto-tune-vllm optimize --config study.yaml --max-concurrent 4
auto-tune-vllm optimize --config study.yaml --max-concurrent-trials 4
```

## License
Expand Down
4 changes: 2 additions & 2 deletions docs/quick_start.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ Use an existing Ray cluster (recommended if one is already running):
auto-tune-vllm optimize \
--config examples/study_config_local_exec.yaml \
--venv-path "$(pwd)/venv" \
--max-concurrent <count>
--max-concurrent-trials <count>
```

Start a new Ray head locally (when no cluster is running):
Expand All @@ -157,7 +157,7 @@ Start a new Ray head locally (when no cluster is running):
auto-tune-vllm optimize \
--config examples/study_config_local_exec.yaml \
--venv-path "$(pwd)/venv" \
--max-concurrent <count> \
--max-concurrent-trials <count> \
--start-ray-head
```

Expand Down
Loading