Under the hood, a model builds internal attention states while reading input during inference. Caching stores those states for the longest identical prefix and skips recomputing them when a later request starts the same way. Even one changed character early in the prompt breaks the match, so serialization must be deterministic and static content must come first.
How do you structure prompts for high cache hit rates?
Put rarely changing content at the start: system instructions, tool definitions, and reference material. Move timestamps, user specific text, and anything dynamic to the end. Multi-turn conversations benefit naturally because earlier messages form the prefix of every later turn, which makes caching especially valuable as the context window fills.
Does prompt caching change model behavior?
No. A cached prefix produces the same internal states as a freshly computed one, so output quality is unaffected. Providers price cached reads below fresh input tokens and may charge a premium to write the cache, so reuse frequency decides whether it pays off. Teams running agentic workflows with many repeated calls see the largest savings.