API reference - Braintrust

Tracing

Tracing records what your application does as spans you can inspect in Braintrust. The recommended way to capture AI calls is auto-instrumentation: call init_logger() then auto_instrument(), and supported libraries are traced with no further code changes (see Install and instrument). The APIs below set up logging, trace your own code, and flush and link to your traces.

init_logger()

Creates a project logger for production traces and makes it the current logger by default. Call it once on startup.

import braintrust

logger = braintrust.init_logger(project="Support bot")

logger.log(
    input={"question": "How do I reset my password?"},
    output={"answer": "Use the account recovery flow."},
    metadata={"route": "/support"},
)

logger.flush()

Returns: Logger.Arguments (all optional):

auto_instrument()

Patches supported AI and ML libraries so their calls are traced to Braintrust automatically. This is the recommended way to capture AI calls.

import braintrust

logger = braintrust.init_logger(project="Support bot")
brintrust.auto_instrument()

from openai import OpenAI

client = OpenAI()
response = client.responses.create(
    model="gpt-5-mini",
    input=[{"role": "user", "content": "What is Braintrust?"}],
)

logger.flush()

Call auto_instrument() after init_logger() and before creating provider or framework clients. If your app imports provider classes directly, such as from openai import OpenAI, call auto_instrument() before those imports when possible so the SDK can patch the imported symbols.

Returns: dict[str, bool], mapping each integration name to whether it was successfully instrumented. Missing optional dependencies are skipped. Arguments (all optional): each supported integration has a boolean flag that defaults to true. Set a flag to false to skip that integration. For the full list of integration flags, see Disabling specific integrations. For example, disable OpenAI instrumentation while keeping the other integrations enabled:

brintrust.auto_instrument(openai=False)

traced()

Decorates a function so each call creates a span, sets it as the current span while the function runs, logs thrown errors, and ends the span when the call finishes. Logs the function arguments as input and the return value as output. To trace a block and log fields yourself, use start_span() in a with block instead. set_current defaults to True.

import braintrust

@braintrust.traced
def classify_text(text: str) -> str:
    return "positive"

classify_text("Great result")

You can also call it with span arguments:

@braintrust.traced(name="Classify text", type="task")
def classify_text(text: str) -> str:
    return "positive"

Returns: the decorated function’s result. Arguments (all optional):

start_span()

Starts a span manually. See @traced to trace a function automatically. When used as a context manager (with start_span(...) as span), start_span() marks the span current and ends it when the with block exits. Nested spans and traced LLM calls attach to it while it is current. If you start a span outside a with block, call span.end() yourself. set_current defaults to True in a with block.

import braintrust

with braintrust.start_span(name="Retrieve documents") as span:
    docs = retrieve_documents()
    span.log(output={"count": len(docs)})

Returns: Span. Arguments (all optional):

current_logger() and current_span()

Return the current logger or span, so you can log to it without holding a direct reference, for example from a helper called while a @traced function or start_span() block runs.

logger = braintrust.current_logger()
span = braintrust.current_span()

span.log(metadata={"cache_hit": True})

current_span() returns a no-op span when no span is current. Its log() calls are ignored, so current_span().log(...) will not throw.

flush()

Flushes pending rows to Braintrust.

brintrust.flush()

For short-lived scripts, call logger.flush(), span.flush(), or braintrust.flush() before the process exits.

permalink()

Builds a Braintrust app URL for an exported span slug, so you can link straight to a trace from your own logs or app.

url = braintrust.permalink(span.export())

Returns: str.

set_masking_function()

Installs a global masking function that runs over logged data before it leaves your process, so you can redact sensitive values before they reach Braintrust.

def mask_secrets(value):
    if isinstance(value, dict) and "api_key" in value:
        return {**value, "api_key": "***"}
    return value

braintrust.set_masking_function(mask_secrets)

Set the masking function to None to disable masking.

Evaluations

An evaluation runs your task over a set of cases, scores each output, and logs the results to an experiment, which is how you measure quality and catch regressions as you change prompts or models. Eval() is the main entry point. The other APIs here run async evaluations and customize reporting.

Eval()

Runs an evaluation from your data, a task, and scorers: it runs the task over every case, scores the outputs, logs each row to an experiment, and returns a summary you can compare across runs.

from braintrust import Eval

Eval(
    "Support bot",
    data=lambda: [
        {
            "input": "How do I reset my password?",
            "expected": "Use the account recovery flow.",
        }
    ],
    task=lambda input: answer_question(input),
    scores=[exact_match],
)

Returns: EvalResultWithSummary. Arguments:

EvalAsync()

Asynchronous version of Eval(). Use it when your task or scorers perform async I/O.

from braintrust import EvalAsync

await EvalAsync(
    "Support bot",
    data=load_cases,
    task=answer_question_async,
    scores=[factuality_score],
)

Returns: an awaitable EvalResultWithSummary. Accepts the same arguments as Eval(), with async tasks and scorers. For data, use a synchronous callable that returns a list or an async generator. An async function that returns a list is not supported.

Reporter()

Creates a reporter for custom evaluation reporting, such as emitting results to CI.

from braintrust import Reporter

reporter = Reporter(
    name="CI reporter",
    report_eval=report_eval,
    report_run=report_run,
)

Arguments:

Experiments

An experiment is a single evaluation run logged to a project. Use these APIs when you want to create an experiment and log rows yourself, instead of letting Eval() manage one for you.

init() / init_experiment()

Creates or opens an experiment in a project for manual experiment logging.

import braintrust

experiment = braintrust.init(project="Support bot", experiment="retrieval-v2")

experiment.log(
    input={"question": "How do I reset my password?"},
    output={"answer": "Use the account recovery flow."},
    scores={"exact_match": 1},
)

experiment.summarize()

Returns: Experiment. Arguments:

Datasets

A dataset is a versioned collection of cases you manage in Braintrust and reuse across experiments and evals. Use init_dataset() to create a dataset or open an existing one.

init_dataset()

Creates or opens a dataset in a project.

import braintrust

dataset = braintrust.init_dataset(project="Support bot", name="Golden questions")

dataset.insert(
    input={"question": "How do I reset my password?"},
    expected={"answer": "Use the account recovery flow."},
    metadata={"source": "docs"},
)

dataset.flush()

Returns: Dataset. Arguments:

Prompts and functions

In Braintrust, functions are units of logic you define and version in the UI, then load or invoke from your code. A prompt is a function whose job is to call a model with a templated set of messages. Other functions include scorers, tools, and code you deploy. Load and render a saved prompt with load_prompt(), or invoke a deployed function with invoke().

load_prompt()

Loads a saved prompt from a Braintrust project. Use the returned prompt’s build() to render request parameters with runtime variables.

import braintrust
from openai import OpenAI

prompt = braintrust.load_prompt(project="Support bot", slug="answer-question")
built = prompt.build(question="How do I reset my password?")

# build() renders Chat Completions parameters. Make the call on a Braintrust-
# instrumented client (for example after auto_instrument()) so it is traced and
# build()'s span_info is stripped before reaching OpenAI.
client = OpenAI()
response = client.chat.completions.create(**built)

Returns: Prompt. Arguments:

load_prompt_async()

Asynchronously loads a saved prompt from a Braintrust project. Use in async contexts to avoid blocking the event loop during the initial prompt fetch.

import asyncio
import braintrust

async def main():
    prompt = await braintrust.load_prompt_async(project="Support bot", slug="answer-question")
    built = prompt.build(question="How do I reset my password?")
    return built

asyncio.run(main())

Returns: Prompt. Arguments: identical to load_prompt().

invoke()

Invokes a Braintrust function and returns either a plain Python object or a BraintrustStream.

import braintrust

result = braintrust.invoke(
    project_name="Support bot",
    slug="answer-question",
    input={"question": "How do I reset my password?"},
)

Returns: the function’s output as a Python object, or a BraintrustStream when stream=True. Arguments: Specify the function to invoke with one of these:

Control the invocation and its trace:

Set the execution context and authentication:

invoke_async()

Async counterpart to invoke(). Returns either a plain Python object or a BraintrustStream, and can be awaited in an async context.

import asyncio
import braintrust

async def main():
    result = await braintrust.invoke_async(
        project_name="Support bot",
        slug="answer-question",
        input={"question": "How do I reset my password?"},
    )
    print(result)

asyncio.run(main())

Returns: the function’s output as a Python object, or a BraintrustStream when stream=True. When stream=True, consume the stream without blocking the event loop using async for iteration or await stream.final_value_async():

async def main():
    stream = await braintrust.invoke_async(
        project_name="Support bot",
        slug="answer-question",
        input={"question": "How do I reset my password?"},
        stream=True,
    )
    async for chunk in stream:
        print(chunk)
    # Or, to collect the full result: result = await stream.final_value_async()

The synchronous for iteration and final_value() continue to work on the returned stream. Arguments: identical to invoke().

init_function()

Creates a Python callable for a Braintrust function, usable as an eval task or scorer.

import braintrust

answer_question = braintrust.init_function(
    project_name="Support bot",
    slug="answer-question",
)

output = answer_question({"question": "How do I reset my password?"})

Returns: a callable that invokes the function. Arguments:

projects.create() and function builders

Define scorers, classifiers, tools, and prompts in code, then deploy them to Braintrust as versioned functions with bt functions push. braintrust.projects.create() returns a project handle whose builders — scorers, classifiers, tools, prompts, and parameters — each expose a create() method:

import braintrust
from pydantic import BaseModel

project = braintrust.projects.create(name="Support bot")

class EqualityInput(BaseModel):
    output: str
    expected: str

def equality_scorer(output: str, expected: str):
    return {"score": 1 if output == expected else 0}

project.scorers.create(
    name="Equality scorer",
    slug="equality-scorer",
    handler=equality_scorer,
    parameters=EqualityInput,
)

A code scorer’s parameters (its input schema, as a Pydantic model) is required. Classifiers use the same builder pattern with project.classifiers.create(). Calling create() only registers the function in your file; bt functions push bundles and uploads everything the file registers.

Attachments

Attachments let you log files or large payloads without storing the full bytes inline in the span. When you trace AI calls, Braintrust automatically converts base64 attachments in provider messages into uploaded attachments, so you rarely need the APIs below for instrumented calls. Reach for them when you’re attaching binary content to a span yourself.

Attachment

Wraps file data so you can attach it to logged data. The uploaded value is replaced with an attachment reference in Braintrust logs.

from braintrust import Attachment

logger.log(
    input="screenshot",
    metadata={
        "image": Attachment(
            data=open("screenshot.png", "rb").read(),
            filename="screenshot.png",
            content_type="image/png",
        )
    },
)

ReadonlyAttachment

Reads an already-uploaded attachment.

attachment = row["metadata"]["image"]
contents = attachment.data
metadata = attachment.metadata()

Methods:

Configuration

Configure the SDK with environment variables, or pass the equivalent options to init_logger() and login().

set_http_adapter()

Sets a custom requests HTTP adapter for Braintrust network requests. Use it for custom retry policies and timeouts.

from requests.adapters import HTTPAdapter

braintrust.set_http_adapter(HTTPAdapter(max_retries=3))

Environment variables

The following variables tune the background log flusher. The defaults are suitable for most workloads; adjust them only for high-throughput or self-hosted deployments.

The following variables configure the local caches for prompts and parameters.