Hermes Context & Token Optimization

A step-by-step guide to audit and massively reduce token waste in Hermes Agent sessions. Based on real production tuning that cut input-to-output ratios from 100:1 to manageable levels.

---

What Problem This Solves

Hermes sessions accumulate massive token overhead:

• Skills auto-injected every turn (~5-15K tokens)
• Full memory dump every turn (~1,500 tokens)
• Unpruned conversation history (grows without bound)
• Verbose tool outputs returned verbatim

Real session data:

Session	Messages	Input Tokens	Output Tokens	Ratio
"Prompt caching"	23	824,916	7,742	106:1
"Whisper install"	86	475,850	4,841	98:1

Most tokens are overhead — not actual generation.

---

Quick Audit

Run this to diagnose your own bloat:

# Find your Hermes state.db (usually ~/.hermes/state.db)
python3 << 'PYEOF'
import sqlite3, sys

db = sys.argv[1] if len(sys.argv) > 1 else "~/.hermes/state.db"
db = db.replace("~", "/home/" + __import__("getpass").getuser())

conn = sqlite3.connect(db)
c = conn.cursor()

# Current session
c.execute('''
  SELECT id, title, message_count, input_tokens, output_tokens
  FROM sessions ORDER BY started_at DESC LIMIT 1
''')
row = c.fetchone()
if row:
    sid, title, msgs, inp, out = row
    ratio = inp / (out or 1)
    print(f"Session: {title}")
    print(f"  Messages: {msgs} | Input: {inp:,} | Output: {out:,} | Ratio: {ratio:.0f}:1")

    # Breakdown by role
    c.execute('''SELECT role, COUNT(*), SUM(token_count) FROM messages
                 WHERE session_id=? GROUP BY role''', (sid,))
    print("  By role:")
    for r in c.fetchall():
        print(f"    {r[0]}: {r[1]} msgs, {r[2] or 0:,} tokens")
conn.close()
PYEOF

Danger signs:

• Ratio > 50:1
• tool messages dominating token count
• Message count > 60 but few user messages

---

The 7 Optimizations (Apply in Order)

1. Prune Unused Skills

Hermes injects ALL matching skills into context every turn. Disable domains you don't use.

# Temporarily hide skill directories (prefix with dot)
cd ~/.hermes/skills
for d in apple creative data-science diagramming dogfood email gaming gifs media note-taking red-teaming smart-home social-media tts-autoplay; do
  if [ -d "$d" ] && [ ! -d ".$d" ]; then
    mv "$d" ".$d"
  fi
done

Tip: Don't delete them — just rename with a . prefix. Re-enable later by removing the dot.

Impact: ~3,000-8,000 tokens saved per turn.

---

2. Cap Context Window

Force Hermes to prune old messages before they bloat the prompt.

# ~/.hermes/config.yaml
model:
  context_length: 32768      # or 65536 for larger models
  ollama_num_ctx: 32768      # Ollama-specific

Impact: Sets a hard ceiling on maximum bloat.

---

3. Reduce max_turns

Rotate sessions before history becomes unwieldy.

# ~/.hermes/config.yaml
agent:
  max_turns: 30              # was 60

Impact: Forces session refresh at 30 turns instead of 60.

---

4. Enable Auto-Compaction

Hermes will auto-summarize old messages into a compressed blob.

hermes config set agent.compact true

Or edit ~/.hermes/config.yaml:

agent:
  compact: true

Impact: Cuts stale history by ~60%.

---

5. Configure Context Compression

Fine-tune how aggressive the pruning is.

# ~/.hermes/config.yaml
compression:
  enabled: true
  threshold: 0.7          # compress when context hits 70% of limit
  target_ratio: 0.2         # keep 20% of threshold as summary
  protect_last_n: 40        # never compress last 40 messages
  protect_first_n: 3        # keep first 3 messages (usually system prompt)
  hygiene_hard_message_limit: 1000

Impact: Automatically manages context without manual intervention.

---

6. Cap Tool Output

Prevent verbose terminal/web search results from flooding the prompt.

# ~/.hermes/config.yaml
tool_output:
  max_bytes: 50000
  max_lines: 2000
  max_line_length: 2000

Also set file read limits:

file_read_max_chars: 100000

Impact: Prevents a single cat or curl from dumping 50K+ tokens.

---

7. Prune Memory Store

User profile + memory notes are injected every turn. Default: ~3,500 chars memory + 2,000 chars profile.

Rules:

• Delete anything session-specific (PR numbers, commit SHAs, "fixed bug X")
• Keep only: preferences, environment facts, conventions, contact methods
• If memory is >80% full, audit and purge stale entries

# Check current memory usage
hermes config show | grep -A2 "memory"

Impact: ~500-1,500 tokens saved per turn.

---

Bonus: Session Rotation with External Memory

Even with all optimizations, long-running sessions eventually bloat. The fix: rotate into an Obsidian vault.

Setup

1. Create an Obsidian vault at ~/Documents/Obsidian Vault
2. Structure:
   • Index.md — master map
   • projects/ — active work
   • tech-stack/ — configs, infra
   • decisions/ — why we chose X over Y
   • meeting-notes/ — dated calls
   • scratch/ — transient notes

Workflow

At ~25 turns, write a summary:

# Session Summary — 2026-06-12

**Topic:** Token optimization for Hermes
**Session ID:** abc123

## What we did
- Disabled 13 unused skill domains
- Set context_length: 32768, max_turns: 30
- Enabled auto-compaction and compression

## Decisions
- Keep Ollama on CT 100 via autossh tunnel (port 11435)
- Use Telegram as primary notification channel

## Files changed
- ~/.hermes/config.yaml
- ~/.hermes/skills/* (disabled via dot-prefix)

## Next steps
- [ ] Test with vLLM for true KV caching
- [ ] Monitor token ratios weekly

Save to ~/Documents/Obsidian Vault/tech-stack/Hermes-Token-Optimization.md.

Resume

Start a new Hermes session with:

Load context from ~/Documents/Obsidian Vault/tech-stack/Hermes-Token-Optimization.md and continue

Result: Read file once (~500 tokens) instead of carrying 25 turns (~12K tokens). Net save: ~11K tokens per rotation.

---

Ollama-Specific Notes

Ollama does NOT persist KV cache across API calls. Each request re-runs the full prefill. keep_alive only keeps model weights in VRAM.

# Still useful — avoids cold starts
ollama_num_ctx: 32768
keep_alive: -1              # keep model loaded indefinitely

For true prompt caching (KV state reuse), use vLLM instead of Ollama.

---

Before vs After

Metric	Before	After
Input/output ratio	100:1	~10-20:1
Baseline overhead/turn	~8K tokens	~3.5K tokens
Session burn (typical)	300-800K tokens	80-200K tokens
Context headroom	~24K	~29K
Auto-prune trigger	60 turns	30 turns

---

Files in This Repo

File	Purpose
README.md	This guide
examples/config-after.yaml	Optimized Hermes config
examples/config-before.yaml	Reference baseline config
scripts/token-audit.sql	SQLite queries for session diagnostics
scripts/session-summary-template.md	Markdown template for Obsidian rotation
scripts/skill-prune.sh	One-liner to disable unused skill domains

---

References

• Hermes Agent Docs
• Ollama API Docs
• vLLM Prefix Caching

---

Last updated: 2026-06-12 | Tested on Hermes Agent with Ollama + OpenRouter backends