Tool cache
Reuses a repeated read's result instead of running the tool again.
Slot: resilience · Ships: off · Enable per agent in agent.toml
When you want it
Agents that re-read the same file, re-run the same search, or re-fetch the same page within or across runs. It costs nothing when nothing repeats.
What it does
When a read tool is called with arguments it was already called with, the cache answers with the stored result and the tool never runs.
The answer is the earlier execution's own output, so a served call is indistinguishable from the call it replaces: the same client event, the same text in the conversation, and an audit line recorded as an executed call. The model is never told that a result came from the cache.
Entries live in the running server until ttl_secs expires them, so a result stored during one run is still available to the next run of the same conversation while it is fresh.
Turn it on
[middleware.tool_cache]
enabled = trueSettings
| Key | Default | What it does |
|---|---|---|
ttl_secs | 300 | How long a cached entry stays fresh. Measured from the execution that produced it, so serving an entry never extends its life. |
max_entries | 1024 | Maximum entries kept; the oldest are evicted first. Bounds the cache by count, not bytes: worst case is about max_entries x max_entry_bytes per agent (~64 MiB at the defaults). Eviction scans the map, so very large values cost more per store. |
max_entry_bytes | 65536 | Largest result that may be cached, measured on its text. Bigger results always re-run. |
A typo in any of these keys fails the boot rather than being silently ignored, and so does a 0 for ttl_secs, max_entries, or max_entry_bytes: each reads like "no limit" but would leave the middleware running on every call and never serving a hit. Disable the middleware instead.
What is never cached
- Writes. Only tools that declare themselves read operations are eligible, in either direction: a write is never stored and never served. A write that succeeds also invalidates every cached read in its scope (this agent, user, and session), because the middleware cannot know which entries that write affected. Without that, an agent could write a file and then be served the pre-write value when it read back to confirm.
- Errors. A failed call is stored nowhere, so a retry really retries.
- Approval-gated calls. A tool your autonomy config puts behind approval reaches its approval card every time, rather than being answered from a result you approved once.
- Results produced by asking the user. A tool that paused mid-call to put a question to the user has that result left out, because a replay would serve the answer-derived result without ever asking the question.
- Results that are more than text. Output carrying images, structured content, a
ui_componentrender hint, or tool reveals is left out: a served result carries text only, so replaying one of these would come back visibly poorer than the execution it stands in for. - Results over
max_entry_bytes.
What counts as the same call
An entry is keyed by the agent, the user, the session, the tool name, and the arguments. Arguments that differ only in key order ({"a":1,"b":2} and {"b":2,"a":1}) are the same call and share one entry.
Nothing is ever served across users or across conversations: a cached result stays inside the session that produced it.
Good to know
- A served call reports a duration of 0 ms in its client event and audit line, because nothing ran.
- A served call still produces a tool span for your observability backend, with the 0 ms duration it reported. The model asked for a tool and acted on what came back, so omitting the span would leave a hole in the trace exactly where a call plainly happened.
- The cache is in memory and per server instance. A restart starts empty, and two instances behind a load balancer keep separate caches; the only effect is a lower hit rate.
- Repeated reads still count toward the loop guard, so an agent stuck re-reading the same thing is still stopped at the usual threshold even when every read is served from cache.
Check it is loaded
axl weave print --config axl-config/agents/<your-agent>/agent.tomltool_cache appears in the resilience band of the printed stack.
Next
- LLM response cache - the same idea for model calls.
- Tool telemetry - find which tool is worth caching.
- All middleware - the full stack and slot ordering.