fix: raise fd limit at startup and fail fast when hard limit is too low#7
Conversation
Add GitHub Actions release workflow triggered by v* tags that builds ckbadger binaries for Linux x86_64, macOS x86_64, and macOS aarch64. Fix build.rs to handle detached HEAD (tag checkout in CI) by making branch_name optional — detached HEAD now produces clean version strings like main branch (e.g. 0.1.0@abcdef123456). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds startup-time file descriptor (fd) limit management to reduce “Too many open files” failures during RocksDB-heavy bulk sync, and to fail fast when the effective limit is insufficient.
Changes:
- Introduces
raise_fd_limit()(best-effort) andcheck_fd_limit_for_indexer()(fail-fast below a minimum threshold). - Displays the effective fd limit in the startup system line (with conditional warning formatting).
- Adds unit tests covering fd-limit helpers and updates existing
print_startup_infocall sites.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
When a SporeCluster ID was passed to GET /spore/objects/{id}, the API
returned it as a spore item because both clusters and spore items share
CF_SPORE_DATA. The frontend then rendered the cluster as a spore detail
page and fetched item activities — which are always empty for clusters
(no outpoints are recorded for clusters).
Now get_spore and list_spore_item_activities reject cluster entries with
404, causing the frontend to fall through to the collection view.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
@claude fix failed checks |
|
Claude finished @janx's task in 1m 53s —— View job Fixed failed checks
Root cause: |
|
@claude check copilot review comments |
|
Claude finished @janx's task in 3m 57s —— View job Fixed Copilot review comments
|
If the process hard fd limit is also low (e.g. macOS with a locked-down launchctl config), the previous soft-limit raise would silently fall back to the hard ceiling and proceed, only to crash mid bulk-sync when RocksDB exhausted the available file descriptors. Changes: - raise_fd_limit() now tries to raise BOTH rlim_max and rlim_cur to 65536; falls back to raising soft limit to current hard ceiling if the hard raise is denied (requires root on Linux, may work for regular users on macOS) - check_fd_limit_for_indexer() fails fast with an actionable error (including OS-specific fix commands) if the effective limit is below FD_LIMIT_MIN=4096 after the raise attempt -- aligns with Fail Fast, Fail Early principle - cmd_run() calls check_fd_limit_for_indexer() before spawning the supervisor when the indexer service is selected - cmd_internal(Indexer) adds the same check as a safety net for direct invocation - print_startup_info() now shows the fd limit in the System line; shows a WARNING in yellow if below FD_LIMIT_MIN - Add tests: raise_fd_limit nonzero, idempotent; check_fd_limit passes/fails Co-authored-by: busyforking <janx@users.noreply.github.com>
…ndexer Co-authored-by: busyforking <janx@users.noreply.github.com>
- raise_fd_limit() now returns Result<u64>; bail! with OS error on getrlimit failure instead of silently returning 0 (misleading message) - services.iter().any(|s| s == "indexer") avoids String allocation - print_startup_info warning threshold changed to FD_LIMIT_TARGET so the banner shows useful signal when limit was raised but below target Co-authored-by: busyforking <janx@users.noreply.github.com>
e860177 to
927c288
Compare
rlim_t is u64 on Linux x86_64 but u32 on some other platforms, so the casts are needed for portability. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixes #4
Goal
Principle Alignment
Result
raise_fd_limit()tries to raise both hard and soft limits to 65536; if the hard-limit raise is denied (EPERM), falls back to raising soft to the current hard ceilingcheck_fd_limit_for_indexer()fails fast with an actionable error if the effective limit is below 4096 after the raise attemptGenerated with Claude Code