Skip to content

refactor: remove MinerJob defaults to prevent resource exhaustion#1604

Merged
glevco merged 1 commit intomasterfrom
refactor/miner-job-defaults
Feb 25, 2026
Merged

refactor: remove MinerJob defaults to prevent resource exhaustion#1604
glevco merged 1 commit intomasterfrom
refactor/miner-job-defaults

Conversation

@glevco
Copy link
Contributor

@glevco glevco commented Feb 24, 2026

Motivation

After running multiple processes that use hathor-core and crash, it's possible that hathor-core will stop working because of leaked resources that are not cleaned up. This happens because of multiprocessing resources that are initialized on module level. The fix is to move them to runtime.

To be clear, this is harmless on most environments, but I caught it while working on integration with Rust.

Acceptance Criteria

  • Move default values of MinerJob from module-level to runtime.

Below is a traceback when the resources are exhausted:

Traceback (most recent call last):
       File "<string>", line 9, in <module>
       File "./.venv/lib/python3.11/site-packages/hathor_tests/unittest.py", line 16, in <module>
         from hathor.builder import BuildArtifacts, Builder
       File "./.venv/lib/python3.11/site-packages/hathor/builder/__init__.py", line 15, in <module>
         from hathor.builder.builder import BuildArtifacts, Builder, SyncSupportLevel
       File "./.venv/lib/python3.11/site-packages/hathor/builder/builder.py", line 35, in <module>
         from hathor.manager import HathorManager
       File "./.venv/lib/python3.11/site-packages/hathor/manager.py", line 57, in <module>
         from hathor.stratum import StratumFactory
       File "./.venv/lib/python3.11/site-packages/hathor/stratum/__init__.py", line 15, in <module>
         from hathor.stratum.stratum import (
       File "./.venv/lib/python3.11/site-packages/hathor/stratum/stratum.py", line 120, in <module>
         class MinerJob(NamedTuple):
       File "./.venv/lib/python3.11/site-packages/hathor/stratum/stratum.py", line 125, in MinerJob
         data: Any = Array('B', 2048)
                     ^^^^^^^^^^^^^^^^
       File "~/.local/share/uv/python/cpython-3.11.13-macos-aarch64-none/lib/python3.11/multiprocessing/sharedctypes.py",
 line 93, in Array
         lock = ctx.RLock()
                ^^^^^^^^^^^
       File "~/.local/share/uv/python/cpython-3.11.13-macos-aarch64-none/lib/python3.11/multiprocessing/context.py", line
 73, in RLock
         return RLock(ctx=self.get_context())
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       File "~/.local/share/uv/python/cpython-3.11.13-macos-aarch64-none/lib/python3.11/multiprocessing/synchronize.py",
 line 194, in __init__
         SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1, ctx=ctx)
       File "~/.local/share/uv/python/cpython-3.11.13-macos-aarch64-none/lib/python3.11/multiprocessing/synchronize.py",
 line 57, in __init__
         sl = self._semlock = _multiprocessing.SemLock(
                              ^^^^^^^^^^^^^^^^^^^^^^^^^
     OSError: [Errno 28] No space left on device

Checklist

  • If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

@glevco glevco self-assigned this Feb 24, 2026
@glevco glevco moved this from Todo to In Progress (WIP) in Hathor Network Feb 24, 2026
@github-actions
Copy link

🐰 Bencher Report

Branchrefactor/miner-job-defaults
Testbedubuntu-22.04
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
minutes (m)
(Result Δ%)
Lower Boundary
minutes (m)
(Limit %)
Upper Boundary
minutes (m)
(Limit %)
sync-v2 (up to 20000 blocks)📈 view plot
🚷 view threshold
1.61 m
(-6.23%)Baseline: 1.71 m
1.54 m
(95.98%)
2.05 m
(78.14%)
🐰 View full continuous benchmarking report in Bencher

@glevco glevco moved this from In Progress (WIP) to In Progress (Done) in Hathor Network Feb 25, 2026
@codecov
Copy link

codecov bot commented Feb 25, 2026

Codecov Report

❌ Patch coverage is 83.33333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 85.65%. Comparing base (735f4e0) to head (697e1de).

Files with missing lines Patch % Lines
hathor/stratum/stratum.py 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1604      +/-   ##
==========================================
- Coverage   85.67%   85.65%   -0.02%     
==========================================
  Files         439      439              
  Lines       33618    33618              
  Branches     5282     5282              
==========================================
- Hits        28802    28796       -6     
- Misses       3801     3806       +5     
- Partials     1015     1016       +1     
Flag Coverage Δ
test-lib 85.65% <83.33%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-project-automation github-project-automation bot moved this from In Progress (Done) to In Review (WIP) in Hathor Network Feb 25, 2026
@glevco glevco merged commit 36fff29 into master Feb 25, 2026
58 of 60 checks passed
@glevco glevco deleted the refactor/miner-job-defaults branch February 25, 2026 14:04
@github-project-automation github-project-automation bot moved this from In Review (WIP) to Waiting to be deployed in Hathor Network Feb 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Waiting to be deployed

Development

Successfully merging this pull request may close these issues.

3 participants