Complete repository health remediation

.github/workflows/tests.yml

@@ -6,15 +6,27 @@ on:
   pull_request:
 
 jobs:
-  pytest:
+  test:
     runs-on: ubuntu-latest
     strategy:
       matrix:
         python-version: ["3.10", "3.12"]
     steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-python@v5
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
         with:
           python-version: ${{ matrix.python-version }}
       - run: pip install pytest
+      - run: python -m compileall -q bin scripts tests
       - run: python -m pytest tests/ -q
+
+  repository-checks:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
+        with:
+          python-version: "3.12"
+      - run: python scripts/check_frontend.py index.html
+      - run: node tests/frontend.test.js
+      - run: python scripts/scan_repo.py .

AGENTS.md

@@ -2,7 +2,7 @@
 
 ## Project
 - **Name:** Usage Tracker
-- **Stack:** Single-file HTML + vanilla JS + inline SVG charts. No CDN dependencies (Google Fonts only). Companion exporter in Python (stdlib only).
+- **Stack:** Single-file HTML + vanilla JS + inline SVG charts. No CDN or runtime network dependencies. Companion exporter in Python (stdlib only).
 - **Run page:** `python3 -m http.server 5200`
 - **Refresh data:** `python3 bin/export_usage.py --since 30d`
 

CHANGELOG.md

@@ -6,6 +6,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ## [Unreleased]
 
+### Added
+- Multi-directory Claude Code and Codex exports, Codex SQLite backfill, machine-readable summaries, and CLI contract tests.
+- Automated frontend smoke, syntax, and repository content checks in CI.
+
+### Changed
+- Reworked the README installation path and examples to match the shipped exporter.
+- Refreshed repository branding and visual assets.
+- Replaced third-party web fonts with offline system font stacks.
+- Consolidated shared frontend rendering helpers across the five visual variants.
+
 ## [0.1.0] - 2026-06-27
 
 ### Added

README.md

@@ -13,7 +13,7 @@
 </p>
 
 <p align="center">
-  Static page plus a tiny Python exporter: OpenClaw, Claude Code, and Codex session spend. API cost vs OAuth subscription value, per model and per agent. No backend.
+  Static page plus a tiny Python exporter: OpenClaw, Claude Code, and Codex session spend. API cost vs OAuth subscription value, per model and per agent. No backend or external runtime assets.
 </p>
 
 <p align="center">
@@ -32,7 +32,8 @@
 git clone https://github.com/escoffier-labs/usage-tracker.git
 cd usage-tracker
 # export local session transcripts to data/usage.json, then open the static page
-python export.py    # see repo for exact entrypoint
+python3 bin/export_usage.py --since 30d
+python3 -m http.server 5200
 ```
 
 ## What it does
@@ -42,7 +43,7 @@ python export.py    # see repo for exact entrypoint
 | **Collect** | Local transcripts | OpenClaw, Claude Code, Codex rollouts |
 | **Split** | API vs OAuth value | What you paid vs what subscriptions absorbed |
 | **Chart** | Models and agents | Daily series, per-model bars, session tables |
-| **Stay local** | Static HTML | No backend, no phone-home |
+| **Stay local** | Static HTML | No backend or third-party requests |
 
 
 ## Quick start
@@ -99,17 +100,19 @@ If `data/usage.json` is missing (e.g., you opened the page on a different machin
 ## Development
 
 ```bash
+python3 -m pip install pytest
 python3 -m pytest tests/   # exporter tests
+node tests/frontend.test.js  # dependency-free frontend behavior checks
 python3 -m http.server 5200  # page
 ```
 
 Example multi-machine export:
 
 ```bash
 python3 bin/export_usage.py \
-  --extra-claude-projects /tmp/gandalf-usage/claude/projects \
-  --extra-codex-sessions /tmp/gandalf-usage/codex \
-  --extra-codex-state-db /tmp/gandalf-usage/codex/state_5.sqlite
+  --extra-claude-projects /tmp/remote-usage/claude/projects \
+  --extra-codex-sessions /tmp/remote-usage/codex \
+  --extra-codex-state-db /tmp/remote-usage/codex/state_5.sqlite
 ```
 
 ## License

bin/export_usage.py

@@ -15,6 +15,7 @@
 import argparse
 import json
 import re
+import sqlite3
 import sys
 from datetime import datetime, timedelta, timezone
 from pathlib import Path
@@ -334,6 +335,89 @@ def walk_codex_sessions(sessions_dir, mtime_cutoff=None):
     return records
 
 
+def walk_codex_state_db(db_path, existing_session_ids, mtime_cutoff=None, warn=False):
+    """Backfill totals for Codex threads without retained rollout usage.
+
+    The state database only has a thread-level token total, so these records
+    deliberately leave the token breakdown and cost unknown.
+    """
+    records = []
+    try:
+        db = sqlite3.connect(f"file:{Path(db_path)}?mode=ro", uri=True)
+    except sqlite3.Error as exc:
+        if warn:
+            print(f"could not read Codex state database {db_path}: {exc}", file=sys.stderr)
+        return records
+    try:
+        rows = db.execute(
+            "SELECT id, updated_at, model_provider, cwd, tokens_used, model "
+            "FROM threads WHERE tokens_used > 0"
+        )
+        for session_id, updated_at, provider, cwd, tokens, model in rows:
+            if session_id in existing_session_ids:
+                continue
+            if mtime_cutoff is not None and updated_at < mtime_cutoff:
+                continue
+            ts = datetime.fromtimestamp(updated_at, timezone.utc).isoformat()
+            label = f"{Path(cwd).name}:{session_id[:8]}" if cwd else None
+            records.append({
+                "ts": ts,
+                "agent": "codex-cli",
+                "sessionId": session_id,
+                "sessionKey": label,
+                "runId": None,
+                "provider": provider or "openai",
+                "modelId": model,
+                "modelApi": "codex-state-db",
+                "billing": "oauth",
+                "workspaceDir": cwd,
+                "input": 0,
+                "output": 0,
+                "cacheRead": 0,
+                "cacheWrite": 0,
+                "totalTokens": tokens,
+                "costUsd": None,
+            })
+    except sqlite3.Error as exc:
+        if warn:
+            print(f"could not read Codex state database {db_path}: {exc}", file=sys.stderr)
+        return []
+    finally:
+        db.close()
+    return records
+
+
+def unique_paths(paths):
+    """Return paths once, resolving aliases without requiring they exist."""
+    seen = set()
+    unique = []
+    for path in paths:
+        resolved = Path(path).expanduser().resolve(strict=False)
+        if resolved in seen:
+            continue
+        seen.add(resolved)
+        unique.append(resolved)
+    return unique
+
+
+def dedupe_records(records):
+    """Collapse the same source call found through overlapping or copied roots."""
+    seen = set()
+    unique = []
+    for record in records:
+        identity = (
+            record.get("_source"), record.get("sessionId"), record.get("ts"),
+            record.get("provider"), record.get("modelId"), record.get("input"),
+            record.get("output"), record.get("cacheRead"), record.get("cacheWrite"),
+            record.get("totalTokens"),
+        )
+        if identity in seen:
+            continue
+        seen.add(identity)
+        unique.append(record)
+    return unique
+
+
 def parse_since(spec, now=None):
     """Turn '7d' / '24h' / '30m' into an ISO cutoff timestamp.
 
@@ -406,6 +490,13 @@ def main(argv=None):
         action="store_true",
         help="Skip Claude Code transcripts entirely",
     )
+    parser.add_argument(
+        "--extra-claude-projects",
+        action="append",
+        default=[],
+        metavar="PATH",
+        help="Additional Claude Code projects directory (repeatable)",
+    )
     parser.add_argument(
         "--codex-sessions",
         default=str(Path.home() / ".codex" / "sessions"),
@@ -419,6 +510,20 @@ def main(argv=None):
         action="store_true",
         help="Skip Codex CLI rollouts entirely",
     )
+    parser.add_argument(
+        "--extra-codex-sessions",
+        action="append",
+        default=[],
+        metavar="PATH",
+        help="Additional Codex sessions or archived sessions directory (repeatable)",
+    )
+    parser.add_argument(
+        "--extra-codex-state-db",
+        action="append",
+        default=[],
+        metavar="PATH",
+        help="Additional Codex state_5.sqlite database for missing-thread backfill (repeatable)",
+    )
     parser.add_argument(
         "--out",
         default=str(Path(__file__).resolve().parent.parent / "data" / "usage.json"),
@@ -438,6 +543,11 @@ def main(argv=None):
             "Example: --oauth-providers openai-codex,claude-cli,acpx,xai"
         ),
     )
+    parser.add_argument(
+        "--summary-json",
+        action="store_true",
+        help="Print a compact machine-readable export summary to stdout",
+    )
     args = parser.parse_args(argv)
 
     oauth_providers = None
@@ -450,26 +560,69 @@ def main(argv=None):
     records = walk_agents_dir(
         args.agents_dir, oauth_providers=oauth_providers, mtime_cutoff=mtime_cutoff
     )
-    openclaw_count = len(records)
+    for record in records:
+        record["_source"] = "openclaw"
 
-    claude_count = 0
-    if not args.no_claude_code and Path(args.claude_projects).is_dir():
-        claude_records = walk_claude_projects(
-            args.claude_projects, mtime_cutoff=mtime_cutoff
-        )
-        claude_count = len(claude_records)
-        records.extend(claude_records)
-
-    codex_count = 0
-    if not args.no_codex and Path(args.codex_sessions).is_dir():
-        codex_records = walk_codex_sessions(
-            args.codex_sessions, mtime_cutoff=mtime_cutoff
-        )
-        codex_count = len(codex_records)
-        records.extend(codex_records)
+    if not args.no_claude_code:
+        for projects_dir in unique_paths([args.claude_projects, *args.extra_claude_projects]):
+            if not projects_dir.is_dir():
+                continue
+            claude_records = walk_claude_projects(
+                projects_dir, mtime_cutoff=mtime_cutoff
+            )
+            for record in claude_records:
+                record["_source"] = "claudeCode"
+            records.extend(claude_records)
+
+    if not args.no_codex:
+        default_sessions = Path.home() / ".codex" / "sessions"
+        codex_dirs = [args.codex_sessions, *args.extra_codex_sessions]
+        state_dbs = list(args.extra_codex_state_db)
+        if Path(args.codex_sessions) == default_sessions:
+            codex_dirs.append(str(Path.home() / ".codex" / "archived_sessions"))
+            state_dbs.insert(0, str(Path.home() / ".codex" / "state_5.sqlite"))
+        for sessions_dir in unique_paths(codex_dirs):
+            if not sessions_dir.is_dir():
+                continue
+            codex_records = walk_codex_sessions(
+                sessions_dir, mtime_cutoff=mtime_cutoff
+            )
+            for record in codex_records:
+                record["_source"] = "codexCli"
+            records.extend(codex_records)
+        existing_session_ids = {
+            r["sessionId"] for r in records
+            if r.get("agent") == "codex-cli" and r.get("sessionId")
+        }
+        explicit_state_dbs = set(unique_paths(args.extra_codex_state_db))
+        for state_db in unique_paths(state_dbs):
+            if not state_db.is_file():
+                if state_db in explicit_state_dbs:
+                    print(
+                        f"could not read Codex state database {state_db}: file not found",
+                        file=sys.stderr,
+                    )
+                continue
+            backfills = walk_codex_state_db(
+                state_db, existing_session_ids, mtime_cutoff=mtime_cutoff,
+                warn=state_db in explicit_state_dbs,
+            )
+            for record in backfills:
+                record["_source"] = "codexCli"
+            records.extend(backfills)
+            existing_session_ids.update(r["sessionId"] for r in backfills)
 
     if cutoff:
         records = filter_since(records, cutoff)
+    records = dedupe_records(records)
+
+    source_counts = {
+        "openclaw": sum(r.get("_source") == "openclaw" for r in records),
+        "claudeCode": sum(r.get("_source") == "claudeCode" for r in records),
+        "codexCli": sum(r.get("_source") == "codexCli" for r in records),
+    }
+    for record in records:
+        record.pop("_source", None)
 
     # Sort newest-first
     records.sort(key=lambda r: r.get("ts") or "", reverse=True)
@@ -497,10 +650,19 @@ def main(argv=None):
     cost_missing = len(records) - cost_known
     print(
         f"exported {len(records)} records to {out_path} "
-        f"({openclaw_count} openclaw, {claude_count} claude-code, {codex_count} codex-cli; "
+        f"({source_counts['openclaw']} openclaw, "
+        f"{source_counts['claudeCode']} claude-code, {source_counts['codexCli']} codex-cli; "
         f"{cost_known} with cost, {cost_missing} missing)",
         file=sys.stderr,
     )
+    if args.summary_json:
+        print(json.dumps({
+            "records": len(records),
+            "sources": source_counts,
+            "costKnown": cost_known,
+            "costMissing": cost_missing,
+            "totalTokens": sum(r.get("totalTokens") or 0 for r in records),
+        }, separators=(",", ":")))
     return 0
 
 

hooks/pre-push

@@ -1,30 +1,13 @@
 #!/usr/bin/env bash
-# pre-push: block push if content-guard finds blocking violations.
-# Scans the working tree against policies/public-repo.json.
+# pre-push: run the same lightweight checks used by CI.
 # Bypass only if you know what you're doing: git push --no-verify
 set -euo pipefail
 
-CONTENT_GUARD_DIR="${CONTENT_GUARD_DIR:-$HOME/repos/content-guard}"
-POLICY="${CONTENT_GUARD_POLICY:-$CONTENT_GUARD_DIR/policies/public-repo.json}"
-
-if [[ ! -d "$CONTENT_GUARD_DIR/src/content_guard" ]]; then
-  echo "pre-push: content-guard not found at $CONTENT_GUARD_DIR" >&2
-  echo "pre-push: clone https://github.com/solomonneas/content-guard or set CONTENT_GUARD_DIR" >&2
-  exit 1
-fi
-
-if [[ ! -f "$POLICY" ]]; then
-  echo "pre-push: policy file not found: $POLICY" >&2
-  exit 1
-fi
-
 REPO_ROOT="$(git rev-parse --show-toplevel)"
-echo "pre-push: scanning $REPO_ROOT against $(basename "$POLICY")"
+cd "$REPO_ROOT"
 
-if ! PYTHONPATH="$CONTENT_GUARD_DIR/src" python3 -m content_guard scan "$REPO_ROOT" --policy "$POLICY"; then
-  echo >&2
-  echo "pre-push: BLOCKED. content-guard found violations." >&2
-  echo "pre-push: fix the leak, or add an inline tag on the offending line:" >&2
-  echo "pre-push:   <!-- content-guard: allow <rule-id> -->" >&2
-  exit 1
-fi
+python3 -m compileall -q bin scripts tests
+python3 scripts/check_frontend.py index.html
+node tests/frontend.test.js
+python3 scripts/scan_repo.py .
+python3 -m pytest tests/ -q