Skip to content

fix: carry the agent's system facts into agent.extract - #112

Open
kevin9327 wants to merge 1 commit into
cactus-compute:mainfrom
kevin9327:fix/agent-extract-system
Open

kevin9327 wants to merge 1 commit into
cactus-compute:mainfrom
kevin9327:fix/agent-extract-system

Conversation

@kevin9327

Copy link
Copy Markdown
Contributor

Problem

agent.extract(...) drops the system facts the agent was built with.

def extract(self, text, schema, max_new_tokens=256, strict=True):
    return extract(text, schema, max_new_tokens=max_new_tokens,
                   weights=self._weights, strict=strict)

weights and strict are forwarded; self._system is not, so the module-level extract builds its
one-shot agent with system=None.

That matters because system facts are what license relative language. From the docs:

Relative language ("tomorrow at 7") resolves only when a date: fact licenses it.

So an agent created the documented way:

agent = needle.Needle(tools=tools, system="date: 2026-07-21 Tue 14:30; locale: en-US")
agent.extract("dinner tomorrow at 7", Reservation)

extracts without the date it was given, while the module-level call one line away keeps it:

needle.extract("dinner tomorrow at 7", Reservation, system="date: 2026-07-21 Tue 14:30")

With strict=True, the default, the loss compounds: _temporal_grounding checks extracted dates
against years found in the source, and an unlicensed relative date has nothing to ground against.

Observed on main by watching what the inner agent is constructed with:

system the agent was built with : 'date: 2026-07-21 Tue 14:30; locale: en-US'
system reaching extract's Needle: None

Fix

Forward self._system, decoded, with an empty string sent as None so an agent built without facts
behaves exactly as before.

Tests

Three in tests/test_weights.py, on the existing engine fixture:

  • test_agent_extract_carries_its_own_system_facts — the facts reach extract. Fails before.
  • test_agent_extract_without_system_facts_sends_none — no facts still means None, not "".
  • test_agent_extract_still_carries_weights_and_strict — the two arguments that already worked keep
    working, alongside the facts. Fails before.

How I tested

Windows 11, Python 3.12.10. pytest -q -m "not slow" gives 67 passed, 9 skipped, up from 64 by the
three new tests; nothing else moves. The 9 skips are the six engine tests and the three worker tests
that want a C compiler.

test_lora.py, test_render.py, test_run.py, test_build.py and test_finetune.py are excluded:
they need flax, whose orbax-checkpoint dependency will not install on Windows because its paths
exceed MAX_PATH. Nothing here touches them, and CI runs them on Ubuntu.

One thing worth a maintainer's opinion: Needle.extract is public but is not in llms.txt or
doc/apis.md, which document only the module-level extract. Happy to add a line for it in either
file if you want it documented rather than quietly available.

Signed-off-by: kevin9327 <kevin9327@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant