feat: Add reproducible NUMA-aware benchmark script (#45)#55
Closed
Dlove123 wants to merge 5 commits intoScottcjn:mainfrom
Closed
feat: Add reproducible NUMA-aware benchmark script (#45)#55Dlove123 wants to merge 5 commits intoScottcjn:mainfrom
Dlove123 wants to merge 5 commits intoScottcjn:mainfrom
Conversation
## Summary Creates a reproducible benchmark comparing RAM Coffers NUMA-aware inference vs stock llama.cpp. ## Features - ✅ Downloads TinyLlama 1.1B Q4_K_M model automatically - ✅ Runs prefill benchmark (128 tokens) - ✅ Runs text generation benchmark (32 tokens) - ✅ Outputs markdown table for GitHub issue - ✅ Works on any multi-NUMA Linux system - ✅ Compares multiple llama.cpp installations ## Usage ```bash chmod +x benchmark_coffers_vs_llamacpp.sh ./benchmark_coffers_vs_llamacpp.sh ``` ## Output - Console results table - Markdown table for GitHub comments - Bounty claim info included Fixes: Scottcjn#45 Bounty: 15 RTC ## Payment Information **Wallet**: Dlove123 **RTC**: RTCb72a1accd46b9ba9f22dbd4b5c6aad5a5831572b **GitHub**: Dlove123
- benchmark_coffers.py - Reproducible benchmark script - Compare RAM Coffers vs stock llama.cpp - Multiple iterations for accuracy - Generate JSON results and Markdown report - Calculate tokens per second and speedup - Easy to use with command line args Bounty: 15 RTC
- Complete benchmark.py comparing RAM Coffers vs llama.cpp - pp128/tg32 configuration as specified - Markdown table output format - NUMA topology detection and reporting - Wallet name: RTCb72a1accd46b9ba9f22dbd4b5c6aad5a5831572b Payment Information: - PayPal: 979749654@qq.com - ETH: 0x31e323edC293B940695ff04aD1AFdb56d473351D - RTC: RTCb72a1accd46b9ba9f22dbd4b5c6aad5a5831572b - GitHub: Dlove123
- benchmark_numa.py: Complete benchmark comparing coffers vs llama.cpp - Auto-downloads TinyLlama-1.1B-Chat-v1.0-GGUF (Q4_K_M) - Runs pp128/tg32 tests with configurable iterations - Detects NUMA topology using numactl - Outputs detailed markdown report with tables - Works on any multi-NUMA Linux system - Includes visualization and analysis Features: - NUMA-aware execution with numactl --interleave=all - JSON output parsing from llama-bench - Iteration averaging for accurate results - System information and reproducibility instructions Payment: - PayPal: 979749654@qq.com - ETH: 0x31e323edC293B940695ff04aD1AFdb56d473351D - RTC: RTCb72a1accd46b9ba9f22dbd4b5c6aad5a5831572b - GitHub: Dlove123
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Added reproducible benchmark script comparing RAM Coffers NUMA-aware inference vs stock llama.cpp.
Features
Usage
Testing
💰 Payment Information
PayPal: 979749654@qq.com
**ETH **(Ethereum) 0x31e323edC293B940695ff04aD1AFdb56d473351D
**RTC **(RustChain) RTCb72a1accd46b9ba9f22dbd4b5c6aad5a5831572b
GitHub: Dlove123