Prompt caching

Most requests repeat the same beginning: a system prompt, tool definitions, a long document, the earlier turns of a conversation. Providers keep that prefix in a cache and charge the tokens read from it at a reduced price, typically 10% of the normal input price. Promptix passes the discount on: the price of a request in your usage log drops as soon as the cache is hit. This page explains how to get those hits.

1. Keep the conversation on one provider#

Prompt caches live at the provider, and many models are served by several providers. If two requests of the same conversation land on different providers, the second one starts from scratch. Promptix therefore pins each conversation to one provider, using the first of these identifiers it finds in the request:

• the x-session-id header;
• metadata.user_id (Anthropic format);
• the user or prompt_cache_key field (OpenAI format);
• otherwise your API key, so a single application still stays together.

Send one stable value per conversation, for example your conversation id. Promptix hashes it before passing it upstream, so the provider never sees your identifiers.

curl https://promptix.tn/api/v1/chat/completions \  -H "Authorization: Bearer $PROMPTIX_API_KEY" \  -H "Content-Type: application/json" \  -H "x-session-id: conversation-8f3a" \  -d '{    "model": "openai/gpt-6-luna",    "messages": [      { "role": "system", "content": "<your long, stable instructions>" },      { "role": "user", "content": "How many days do I have to return a product?" }    ]  }'

2. Put the stable part first#

A cache matches on the prefix of the prompt, byte for byte. Order your messages so that what never changes comes first and what changes comes last:

• system prompt and tool definitions first, then documents, then the conversation history, then the new question;
• do not put a timestamp, a user name or a random id at the top of the system prompt: any change there invalidates everything after it;
• keep the same model: a cache belongs to one model;
• short prompts are not cached. Providers start caching from about 1,024 tokens of prefix (more on some models), so a two-line system prompt will never show cached tokens.

3. Per-provider rules#

GPT (OpenAI), Gemini (Google), DeepSeek and most other providers cache automatically. Nothing to add to the request: with a stable session and a long enough prefix, the second call reports cached tokens.

Claude (Anthropic) caches only where you ask for it. Add a cache_control breakpoint on the last block of the stable part; everything before it is cached for five minutes and refreshed on every hit. It works through both endpoints:

{  "model": "anthropic/claude-sonnet-5.5",  "messages": [    {      "role": "system",      "content": [        {          "type": "text",          "text": "<your long, stable instructions or document>",          "cache_control": { "type": "ephemeral" }        }      ]    },    { "role": "user", "content": "How many days do I have to return a product?" }  ],  "user": "conversation-8f3a"}

{  "model": "anthropic/claude-sonnet-5.5",  "max_tokens": 1024,  "system": [    { "type": "text", "text": "<your long, stable instructions or document>", "cache_control": { "type": "ephemeral" } }  ],  "messages": [{ "role": "user", "content": "How many days do I have to return a product?" }],  "metadata": { "user_id": "conversation-8f3a" }}

4. Check that it works#

Every response carries the provider's usage counts, so you can see the cache in action. In the OpenAI format, look at usage.prompt_tokens_details.cached_tokens. The two responses below come from the same session with a 2,600-token system prompt on openai/gpt-6-luna; the second request cost about a tenth of the first.

// First call: the prefix is written to the cache"usage": {  "prompt_tokens": 2621,  "completion_tokens": 26,  "total_tokens": 2647,  "prompt_tokens_details": { "cached_tokens": 0, "cache_write_tokens": 2618 }} // Second call, same prefix, same session: read from the cache"usage": {  "prompt_tokens": 2617,  "completion_tokens": 17,  "total_tokens": 2634,  "prompt_tokens_details": { "cached_tokens": 2603, "cache_write_tokens": 11 }}

In the Anthropic format, the cached part appears in cache_read_input_tokens and the first write in cache_creation_input_tokens; input_tokens only counts what was not cached:

"usage": {  "input_tokens": 17,  "cache_creation_input_tokens": 0,  "cache_read_input_tokens": 5542,  "output_tokens": 25}

With streaming, the same object arrives in the last chunk before [DONE] (see Streaming). The amount actually charged in TND is in the usage log and in GET /generation.

Claude Code and the agent chat#

Claude Code sends a session id and cache breakpoints on its own, and the agent at chat.promptix.tn does the same for every conversation. Nothing to configure: long agent sessions read most of their context from the cache.