Skip to content

[Bug] gitt miner score --json --log-level warning crashes in _drain_logs()bt.logging.off() raises TransitionNotAllowed from Warning state #1373

Description

@taillred

Scope

One-line guard in _drain_logs() so the cleanup path is tolerant of every --log-level value Click already accepts. Matches the reachable-crash acceptance shape of PR #624.

Description

gitt miner score advertises warning in click.Choice(['warning', 'info', 'debug', 'trace']) (score.py:255), but using that value crashes at shutdown. The command calls _drain_logs() (score.py:325), which unconditionally calls bt.logging.off() (score.py:235). When bittensor's LoggingMachine is in the Warning state, disable_logging is not a defined transition and the command exits with a traceback.

This is the only advertised --log-level value that crashes. info, debug, and trace all complete cleanly.

Bittensor library evidence (deterministic, not flaky)

From bittensor/utils/btlogging/loggingmachine.py:131-137:

disable_logging = (
    Trace.to(Disabled)
    | Debug.to(Disabled)
    | Default.to(Disabled)
    | Disabled.to(Disabled)
    | Info.to(Disabled)
)

Warning is intentionally omitted — bittensor's only Warning shutdown transition is disable_warning = Warning.to(Default) (line 127). Any caller that lands in Warning and then invokes bt.logging.off() raises TransitionNotAllowed. This is a stable bittensor API behavior, not a transient race.

Steps to Reproduce

From the repository root:

uv run --extra dev gitt miner score \
  --pat ghp_invalid_token_for_repro \
  --json \
  --log-level warning

Exit code 1, traceback ends in:

File ".../gittensor/cli/miner_commands/score.py", line 325, in score_command
    _drain_logs()
File ".../gittensor/cli/miner_commands/score.py", line 235, in _drain_logs
    bt.logging.off()
  ...
statemachine.exceptions.TransitionNotAllowed: Can't disable_logging when in Warning.

A/B across all four advertised --log-level values:

for lvl in info debug trace warning; do
  uv run --extra dev gitt miner score --pat ghp_invalid_token_for_repro \
    --json --log-level "$lvl" >/tmp/out 2>/tmp/err
  echo "$lvl exit=$?"
done

Output:

info exit=0
debug exit=0
trace exit=0
warning exit=1

Expected Behavior

Every value Click already accepts in --log-level should complete without a state-machine crash. The command should emit its JSON result or a normal CLI error envelope; it should not raise out of the logging-shutdown path.

Actual Behavior

--log-level warning exits 1 with statemachine.exceptions.TransitionNotAllowed: Can't disable_logging when in Warning.. All other advertised levels complete cleanly. The crash is independent of the scoring pipeline — _apply_log_level('warning') puts bittensor's LoggingMachine into the Warning state, and _drain_logs()'s unconditional bt.logging.off() then fires a transition bittensor does not define.

Proposed Fix

One-line state check in _drain_logs(). Matches bittensor's own internal pattern of comparing current_state_value before transitioning (see set_debug / set_trace / set_info in loggingmachine.py):

 def _drain_logs() -> None:
     ...
     import bittensor as bt

-    bt.logging.off()
+    # bittensor's LoggingMachine has no `disable_logging` transition from the
+    # Warning state (only `disable_warning = Warning.to(Default)` exists).
+    # info/debug/trace all permit the transition, so guard only Warning.
+    if bt.logging.current_state_value != 'Warning':
+        bt.logging.off()
     queue = bt.logging.get_queue()
     ...

State check is preferred over try/except TransitionNotAllowed so we don't use exceptions for normal control flow.

Regression test

Hermetic — doesn't need the full scoring pipeline. Assert that draining after each advertised log level is a no-op exception-wise and still reaches the flush path:

import pytest
import bittensor as bt
from gittensor.cli.miner_commands.score import _apply_log_level, _drain_logs

@pytest.mark.parametrize('level', ['warning', 'info', 'debug', 'trace'])
def test_drain_logs_tolerates_every_advertised_level(level):
    _apply_log_level(level)
    _drain_logs()  # must not raise
    # Idempotent: a second drain in the now-Disabled state must also not raise.
    _drain_logs()

The first call covers the Warning → Disabled crash this issue reports. The second covers Disabled → Disabled, which bittensor does allow but is worth pinning.

Affected Files

Prior art / non-duplication

Distinct from:

3-axis search (_drain_logs × TransitionNotAllowed × --log-level warning) returns no overlapping open issue or PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions