Token

/'toh.kuhn/A single unit of text that language models process. Token counts determine both cost and latency because models bill and compute proportionally to the input and output of tokens.(noun)

“We reduced token usage by shortening the retrieved context.”

Customer example

As Notion's prompts grew from thousands to hundreds of thousands of tokens, they needed to search massive traces for specific tool calls and patterns.

Related Deployment terms

• Brainstore
• Caching
• Fallback
• Function (deployed)
• Gateway
• Hybrid deployment
• Prompt
• Prompt (deployed)
• Slug
• Streaming

From the docs

• Use the Braintrust gateway
• Monitor usage
• Log token metrics to @traced spans manually

Get started with Evals

Braintrust is the observability platform for agents in production. By actively applying intelligence to agent traces and automatically surfacing critical patterns, Braintrust helps teams at Notion, Stripe, Box, OpenAI, and Cloudflare ship quality agents at scale.