> ## Documentation Index
> Fetch the complete documentation index at: https://docs.primalabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Prompt caching

> How prompt prefixes are reused, who can share them, and how to split your cache further.

Models reuse the work done on a prompt prefix they have recently seen, so a request that
starts like an earlier one (the same system prompt, the same document, the same conversation
so far) gets its first token sooner. Cached input tokens are reported in the response's
`usage` (`prompt_tokens_details.cached_tokens`). Caching is automatic and best-effort: a
prefix stays cached only while there is room for it.

## Your cache is your own

Every account has its **own cache**. A prefix one account sent is never reused for another
account's request, and no other account can observe what yours has cached. All of your API
keys share your account's cache, including keys you create after rotating an old one.

## Splitting your cache further: `cache_salt`

If you serve several end customers through one account, you can keep their prefixes apart too.
Send a `cache_salt` string with the request: requests reuse each other's prefixes only when
they carry the same `cache_salt`. A salt is always scoped to your account, so two accounts
that pick the same value still never share a cache.

* Any non-empty string; an empty string or no `cache_salt` means your account's default cache.
* Use a stable value per partition, such as your own customer id. Changing it starts that
  partition from an empty cache.
* Supported on `/v1/chat/completions`, `/v1/completions`, `/v1/responses` and `/v1/messages`.

```python OpenAI SDK theme={null}
from openai import OpenAI

client = OpenAI(base_url="https://api.primalabs.ai/v1", api_key="YOUR_API_KEY")

client.chat.completions.create(
    model="MODEL_ID",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={"cache_salt": "customer-1234"},
)
```

```python Anthropic SDK theme={null}
from anthropic import Anthropic

client = Anthropic(base_url="https://api.primalabs.ai", api_key="YOUR_API_KEY")

client.messages.create(
    model="MODEL_ID",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={"cache_salt": "customer-1234"},
)
```

```bash curl theme={null}
curl https://api.primalabs.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MODEL_ID",
    "messages": [{"role": "user", "content": "Hello"}],
    "cache_salt": "customer-1234"
  }'
```

A `cache_salt` that is not a string is rejected with a `400`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.