Index out of bounds error in `quantile_duration` in src/results.rs

[cyril23]

Describe the bug
An "index out of bounds" error occurs in src/results.rs during benchmark runs. The error message index out of bounds: the len is 1 but the index is 1 at src/results.rs:252:31 points to an issue in the quantile_duration function.

To Reproduce
Steps to reproduce the behavior:

1. Run a benchmark with settings that might result in only one successful response.
2. Observe the panic.

Expected behavior
The quantile_duration function should handle cases where the data vector has only one element, possibly by returning that single element or a predefined value, instead of panicking.

Console logs

export URL = http://localhost:8000/v1
sudo docker run --network host \
  -e HF_TOKEN=$HF_TOKEN \
  -v ~/inference-benchmarker-results:/opt/inference-benchmarker/results \
  inference_benchmarker inference-benchmarker \
  --prompt-options "num_tokens=200,max_tokens=220,min_tokens=180,variance=10" \
  --decode-options "num_tokens=200,max_tokens=220,min_tokens=180,variance=10"
  --url $URL
...
thread 'tokio-runtime-worker' panicked at /usr/local/cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ratatui-0.29.0/src/terminal/init.rs:52:16:
thread 'tokio-runtime-worker' panicked at src/results.rs:252:31:chmarker/src/event.rs:40:48:cf8c6b5b557f/ratatui-0.29.0/src/terminal/init
index out of bounds: the len is 1 but the index is 1ailed to initialize input reader" }

Possible Cause
The quantile_duration function at src/results.rs:252 accesses data[i as usize + 1] without checking if i as usize + 1 is a valid index. If data has only one element, and i is 0 (which can happen if quantile is such that (data.len() - 1) as f64 * quantile is less than 1), then i as usize + 1 would be 1, leading to an out-of-bounds access.

Suggested Fix
Add a check within the quantile_duration function to handle cases where the length of the data vector is 1, or when i as usize + 1 would be out of bounds. For example, if data.len() is 1, the function could return data[0].as_secs_f64().

Environment:

• OS: Ubuntu 24 LTS as WSL with Windows 11 host
• inference-benchmarker version: build inference_benchmarker:latest from 2025-05-15T18:48:42.880472539+02:00

Remark
this issue was created with the help of Gemini 2.5 Pro (especially the parts about possible cause and suggested fix...)

[cyril23]

Suggested Fix
Add a check within the quantile_duration function to handle cases where the length of the data vector is 1, or when i as usize + 1 would be out of bounds. For example, if data.len() is 1, the function could return data[0].as_secs_f64().

Here is an example implementation from Google Gemini 2.5 Pro. I didn't open a Pullrequest yet because I have no idea how bad it is. But if anyone wants to try out: inference_benchmarker_changes.patch

git clone https://github.com/huggingface/inference-benchmarker.git
cd inference-benchmarker
# important: checkout the following commit which my patch is based on:
git checkout 687e477930b387d3c9c787d4953a266f6469f047
# optionally: create your own branch
git switch -c mybranch
# apply the patch:
git apply inference_benchmarker_changes.patch

If you don't want to apply the patch like that and just copy-paste and replace the whole fn quantile_duration within inference-benchmarker/src/results.rs, here you go:

    /// Calculate the quantile of a given data set using interpolation method
    /// Results are similar to `numpy.percentile`
    fn quantile_duration(&self, mut data: Vec<Duration>, quantile: f64) -> anyhow::Result<f64> {
        if !self.is_ready() {
            // If benchmark results are not ready (e.g., no start/end times),
            // we can't reliably calculate quantiles.
            return Err(anyhow::anyhow!(BenchmarkErrors::NoResponses));
        }

        // Sort the data to prepare for percentile calculation.
        data.sort();

        if data.is_empty() {
            // If there's no data, a quantile cannot be determined.
            return Err(anyhow::anyhow!(BenchmarkErrors::NoResponses));
        }

        if data.len() == 1 {
            // If there's only one data point, it represents all quantiles.
            return Ok(data[0].as_secs_f64());
        }

        // For data.len() > 1, proceed with linear interpolation.
        // Ensure quantile is within a valid range [0.0, 1.0] if necessary,
        // though this example assumes valid input based on typical percentile usage.
        // If quantile can be outside [0,1], additional clamping or error handling might be needed.
        // For this fix, we primarily address the out-of-bounds access.

        let n_minus_1 = (data.len() - 1) as f64;
        let index_float = quantile * n_minus_1;

        let lower_idx = index_float.floor() as usize;
        let fraction = index_float - (lower_idx as f64);

        // Guard against the calculated lower_idx itself being out of bounds.
        // This could happen with extreme/invalid quantile values or if data.len() was mishandled earlier.
        if lower_idx >= data.len() {
             // This implies an issue beyond just hitting the last element, e.g., quantile > 1.0
             // and index_float.floor() goes past the end of the array.
            return Err(anyhow::anyhow!(BenchmarkErrors::NoResponses));
        }
        
        if lower_idx == data.len() - 1 {
            // If the lower_idx is the last element in the sorted array (e.g., quantile is 1.0),
            // the value is simply the last element. No upper element for interpolation.
            Ok(data[lower_idx].as_secs_f64())
        } else {
            // Standard linear interpolation: (1-fraction)*value_at_lower_idx + fraction*value_at_upper_idx
            // Since lower_idx < data.len() - 1, lower_idx + 1 is a valid index.
            let val_lower = data[lower_idx].as_secs_f64();
            let val_upper = data[lower_idx + 1].as_secs_f64(); // Safe: lower_idx + 1 < data.len()
            let result = (1.0 - fraction) * val_lower + fraction * val_upper;
            Ok(result)
        }
    }