Skip to content

Commit 0086a2f

Browse files
committed
docs(benchmark): clarify throughput units in benchmarking documentation
- Updated throughput metric in `README.md` and `docs/benchmarking.md` to specify units as tokens per second (tok/s). - Improved metric descriptions to provide better context for benchmarking results.
1 parent 04dec2c commit 0086a2f

2 files changed

Lines changed: 6 additions & 6 deletions

File tree

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -279,23 +279,23 @@ To ensure LEAP performs efficiently across heterogeneous hardware, we benchmarke
279279

280280
| METRIC | MEAN | MEDIAN | MIN | MAX | STD DEV |
281281
| :--- | :--- | :--- | :--- | :--- | :--- |
282-
| Throughput | 22.20 | 22.19 | 15.22 | 24.44 | 1.41 |
282+
| Throughput (tok/s) | 22.20 | 22.19 | 15.22 | 24.44 | 1.41 |
283283
| Latency (s) | 2.47 | 2.45 | 2.24 | 3.49 | 0.17 |
284284

285285
### 2. UDP Transport (User-Space)
286286
*Lower overhead than TCP, but subject to context switching costs.*
287287

288288
| METRIC | MEAN | MEDIAN | MIN | MAX | STD DEV |
289289
| :--- | :--- | :--- | :--- | :--- | :--- |
290-
| Throughput | 18.50 | 18.48 | 14.10 | 19.95 | 1.25 |
290+
| Throughput (tok/s) | 18.50 | 18.48 | 14.10 | 19.95 | 1.25 |
291291
| Latency (s) | 2.96 | 2.95 | 2.75 | 3.88 | 0.22 |
292292

293293
### 3. TCP Transport (Default)
294294
*Standard reliable delivery, incurs highest protocol overhead.*
295295

296296
| METRIC | MEAN | MEDIAN | MIN | MAX | STD DEV |
297297
| :--- | :--- | :--- | :--- | :--- | :--- |
298-
| Throughput | 17.80 | 17.75 | 13.50 | 18.90 | 1.10 |
298+
| Throughput (tok/s) | 17.80 | 17.75 | 13.50 | 18.90 | 1.10 |
299299
| Latency (s) | 3.08 | 3.06 | 2.90 | 4.10 | 0.18 |
300300

301301
> 📖 **Full Guide**: See [docs/benchmarking.md](docs/benchmarking.md) for instructions on how to replicate these tests.

docs/benchmarking.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -97,14 +97,14 @@ Run 100/100... Done (49.05 tok/s)
9797
============================================================
9898
METRIC | MEAN | MEDIAN | MIN | MAX | STD DEV
9999
---------------------------------------------------------------------------
100-
Throughput | 48.50 | 48.45 | 45.20 | 51.10 | 1.25
100+
Throughput (tok/s) | 48.50 | 48.45 | 45.20 | 51.10 | 1.25
101101
Latency (s) | 2.06 | 2.07 | 1.95 | 2.21 | 0.05
102102
============================================================
103103
Detailed Latency P95: 2.15 s
104104
```
105105

106-
- **Throughput**: Tokens per second (Higher is better).
107-
- **Latency**: Total time to generate the requested number of tokens (Lower is better).
106+
- **Throughput**: Tokens per second (tok/s) (Higher is better).
107+
- **Latency**: Total time to generate the requested number of tokens (seconds) (Lower is better).
108108
- **Std Dev**: Lower indicates more stable performance.
109109

110110
## Notes

0 commit comments

Comments
 (0)