Production prompts repeat themselves. The same system instructions, the same tool definitions, the same few-shot examples ride along on every request, and only the user's last message changes. Reprocessing that fixed block each time is work the provider has already done.
Prompt caching keeps the attention state for a prefix and reuses it. The next request that starts with the identical prefix skips prefill for that section and begins computing at the first differing token. Providers price cached input well below fresh input and the latency saving on a long prefix is often larger than the cost saving.
The behaviour has one rule that shapes how you write prompts: caching matches from the start of the prompt, so anything variable must come after everything stable. A timestamp at the top of a system prompt invalidates the cache on every call. Move it below the fixed block and the cache holds.
Cache entries expire quickly, typically within minutes of inactivity, so the benefit shows up on sustained traffic rather than occasional requests.