Caching
How automatic conversation caching keeps replies fast and stretches your token budget.
Last updated on
Every chat is cached automatically. There's nothing to switch on. Caching makes replies faster and helps your token budget go further.
What gets cached
The shared instructions
The large HaloPSA expertise behind every chat is cached once and shared across all users, so it's re-read at only a small fraction of full price instead of re-sent in full each turn.
Your conversation
As a chat grows, the turns so far are cached so the assistant re-reads them cheaply each turn instead of re-paying full price.
What it costs you
- New content: full price. Your message and the assistant's reply, each turn.
- Re-reading the cache: a small fraction. Both your earlier turns and the shared HaloPSA instructions are re-read from cache every turn, billed at only a small fraction of full price. That's the saving: they're never re-sent in full.
The net effect: a long conversation costs far less than re-paying for everything each turn, and answers come back faster.
Caching is automatic and always on; you don't configure anything. It simply makes each chat quicker and cheaper as it grows.
Related
- Understanding tokens: the per-conversation and monthly limits.
- Use fewer tokens: habits that stretch your allowance further.