Skip to content

Bound LLM classification cost in AdversarialLLMLayer.aevaluate #31

Description

@requie

aevaluate makes one sequential awaited LLM call per (channel, text) target. Multi-agent buffers cap at 1000 entries each (_BUFFER_CAP), and #29 correctly added the shared-memory and broadcast channels, so a busy or adversarial team session can push worst-case evaluation into thousands of sequential API calls. That's a latency tail and a cost amplification lever on attacker-writable channels.

Two candidate fixes, not mutually exclusive:

  1. Bounded-concurrency asyncio.gather over the targets (semaphore, e.g. 8).
  2. Incremental classification: track a per-channel high-water mark and only classify entries added since the last evaluation, since earlier entries were already classified with identical text.

Option 2 is the real fix; option 1 is a cheap immediate win. Either way the layer should log when it truncates or skips, so a capped scan never reads as full coverage.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions