API reference - Braintrust

Tracing

Tracing records what your application does as spans you can inspect in Braintrust. The recommended way to capture AI calls is auto-instrumentation: use the trace/contrib packages to instrument supported provider libraries, either at build time with Orchestrion or with runtime middleware (see Go SDK integrations). Tracing is built on OpenTelemetry, so you trace your own code with the standard OpenTelemetry API. The APIs below create the client, trace your own code, and link to your traces.

braintrust.New

Creates a Braintrust client and configures the OpenTelemetry pipeline that exports spans to Braintrust. Call it once on startup, passing your TracerProvider and any options.

import (
    "github.com/braintrustdata/braintrust-sdk-go"
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/sdk/trace"
)

tp := trace.NewTracerProvider()
otel.SetTracerProvider(tp)

client, err := braintrust.New(tp, braintrust.WithProject("My project"))
if err != nil {
    log.Fatal(err)
}

Returns: (*braintrust.Client, error).

braintrust.New reads BRAINTRUST_API_KEY from the environment. Configure the rest with functional options or environment variables (see Configuration). Because tracing is built on OpenTelemetry, you trace your own application code with the standard OpenTelemetry API, and traced AI calls nest under your spans.

ctx, span := otel.Tracer("my-app").Start(ctx, "process-request")
defer span.End()

Client.Permalink

Builds a Braintrust UI URL for a span, so you can link straight to a trace from your own logs or app.

url := client.Permalink(span)

Returns: string.

Evaluations

An evaluation runs your task over a set of cases, scores each output, and logs the results to an experiment, which is how you measure quality and catch regressions as you change prompts or models. The recommended pattern is to define an eval once with braintrust.NewEval, then call Run with any dataset. The same definition works for local runs, bt eval <dir> runs from the command line, and remote eval runs triggered from the playground.

braintrust.NewEval

Creates a runnable *eval.Eval by combining a client with an eval definition. Call Run on it to execute the evaluation.

import (
    "context"
    "github.com/braintrustdata/braintrust-sdk-go"
    "github.com/braintrustdata/braintrust-sdk-go/eval"
)

e := braintrust.NewEval(client, &eval.Eval[string, string]{
    Name: "classify",
    Task: eval.T(func(ctx context.Context, input string) (string, error) {
        return classify(input), nil
    }),
    Scorers: []eval.Scorer[string, string]{exactMatch},
    ProjectName: "my-project",
})

_, err := e.Run(ctx, eval.RunOpts[string, string]{
    Dataset: eval.NewDataset([]eval.Case[string, string]{
        {Input: "apple", Expected: "fruit"},
    }),
})

Returns: *eval.Eval[I, R]. Run returns (*eval.Result, error).

eval.Eval[I, R] fields:

eval.RunOpts[I, R] fields (passed to Run):

braintrust.NewEvaluator

Creates an evaluator for input type I and result type R, bound to a client. Call Run on it with the cases, task, and scorers to execute the evaluation and log an experiment.

import (
    "context"
    "github.com/braintrustdata/braintrust-sdk-go"
    "github.com/braintrustdata/braintrust-sdk-go/eval"
)

evaluator := braintrust.NewEvaluator[string, string](client)

_, err := evaluator.Run(context.Background(), eval.Opts[string, string]{
    Experiment: "answers-v1",
    Dataset: eval.NewDataset([]eval.Case[string, string]{
        {Input: "How do I reset my password?", Expected: "Use the account recovery flow."},
        {Input: "How do I export my data?", Expected: "Open Settings and choose Export."},
    }),
    Task: eval.T(answerQuestion),
    Scorers: []eval.Scorer[string, string]{
        eval.NewScorer("exact_match", func(_ context.Context, r eval.TaskResult[string, string]) (eval.Scores, error) {
            v := 0.0
            if r.Output == r.Expected {
                v = 1.0
            }
            return eval.S(v), nil
        }),
    },
})

Returns: *eval.Evaluator[I, R]. Run returns (*eval.Result, error).

eval.Opts[I, R] fields:

eval.NewScorer

Creates a scorer from a function. A scorer measures how good the task’s output is, returning one or more named scores per case.

scorer := eval.NewScorer("exact_match", func(_ context.Context, r eval.TaskResult[string, string]) (eval.Scores, error) {
    if r.Output == r.Expected {
        return eval.S(1.0), nil
    }
    return eval.S(0.0), nil
})

Returns: eval.Scorer[I, R]. The score function receives an eval.TaskResult[I, R] (with Input, Output, Expected, and Metadata) and returns eval.Scores. Use eval.S to build a single score.

eval.NewClassifier

Creates a classifier from a function. Use a classifier to categorize output instead of scoring it numerically.

classifier := eval.NewClassifier("topic", func(_ context.Context, r eval.TaskResult[string, string]) (eval.Classifications, error) {
    return eval.Classifications{{ID: "billing", Label: "Billing"}}, nil
})

Returns: eval.Classifier[I, R].

eval.TaskWithHooks

Creates a task function that receives *eval.TaskHooks, which gives access to parameters, metadata, tags, and the current spans. Use it when your task needs to read parameter values that were configured in the playground or passed via RunOpts.Parameters.

task := eval.TaskWithHooks(func(ctx context.Context, input string, hooks *eval.TaskHooks) (string, error) {
    model := hooks.Parameters.String("model")
    return classify(input, model), nil
})

Returns: eval.TaskFunc[I, R]. The hooks give access to Parameters, Metadata, Tags, TrialIndex, TaskSpan, and EvalSpan. Use eval.T instead when the task doesn’t need hooks.

Parameters

Parameters let you declare configurable options on an eval. When the eval runs from the Braintrust playground via a remote eval, each declared parameter becomes a control in the UI. For a local run, the task receives the declared defaults. Declare parameters with eval.ParameterSchema on the eval.Eval definition:


e := braintrust.NewEval(client, &eval.Eval[string, string]{
    Name: "classify",
    ParameterSchema: eval.ParameterSchema{
        "model": {
            Type: eval.ParameterTypeModel,
            Default: "gpt-5-mini",
            Description: "Model to use for classification",
        },
        "threshold": {
            Type: "number",
            Default: 0.5,
        },
    },
    Task: eval.TaskWithHooks(func(ctx context.Context, input string, hooks *eval.TaskHooks) (string, error) {
        model := hooks.Parameters.String("model")
        threshold := hooks.Parameters.Float64("threshold")
        return classify(input, model, threshold), nil
    }),
    // ...
})

eval.ParameterSchema is map[string]eval.ParameterDef. Each eval.ParameterDef has:

eval.Parameters (the resolved values, available as hooks.Parameters) has typed accessors that never panic on a type mismatch:

Remote evals

Remote evals let the Braintrust playground trigger your Go eval code on your own infrastructure. The evalrunner package turns a Go binary into a target that the bt CLI can drive.

Go remote evals are in public preview and can change before reaching general availability.

evalrunner.New and evalrunner.RegisterEval

Create a runner and register your evals. The runner reads the bt environment variables, dispatches the right eval, and streams results back.

package main

import (
    "context"
    "github.com/braintrustdata/braintrust-sdk-go/eval"
    "github.com/braintrustdata/braintrust-sdk-go/evalrunner"
)

func main() {
    r := evalrunner.New()

evalrunner.RegisterEval(r, &eval.Eval[string, string]{
        Name: "classify",
        Task: eval.TaskWithHooks(func(ctx context.Context, input string, hooks *eval.TaskHooks) (string, error) {
            model := hooks.Parameters.String("model")
            return classify(input, model), nil
        }),
        Scorers: []eval.Scorer[string, string]{exactMatch},
        ParameterSchema: eval.ParameterSchema{
            "model": {Type: eval.ParameterTypeModel, Default: "gpt-5-mini"},
        },
        ProjectName: "my-project",
    })

evalrunner.Main(r)
}

Then run from the command line:

# Run all evals locally
bt eval ./cmd/evals

# Start the dev server so the playground can trigger runs
bt eval --dev --language go ./cmd/evals

evalrunner.New accepts evalrunner.Option values:

evalrunner.RegisterEval[I, R any](r *Runner, ev *eval.Eval[I, R]) registers an eval by its Name. Registering two evals under the same name replaces the first. evalrunner.Main(r *Runner) dispatches and exits. Use evalrunner.Run(ctx, r) instead if you need to handle the error yourself.

Prompts

Prompts saved in Braintrust carry a template, a model, and parameters. The Go SDK loads them from the API, renders their variables, and returns a provider-agnostic result you hand to any LLM client. A prompt is fetched every time. Nothing is cached.

client.LoadPrompt

Loads a prompt from Braintrust by slug and returns it ready to render. The client’s configured project is used when no project is specified in opts.

import "github.com/braintrustdata/braintrust-sdk-go/prompt"

p, err := bt.LoadPrompt(ctx, prompt.LoadOpts{Slug: "summarizer"})
if err != nil {
    return err
}
built, err := p.Build(map[string]any{"input": article})

Returns: (*prompt.Prompt, error).

prompt.LoadOpts fields:

prompt.Definition

Declares a prompt in Go code. Use it as the Default for a prompt eval parameter:

import "github.com/braintrustdata/braintrust-sdk-go/prompt"

eval.ParameterSchema{
    "summary_prompt": {
        Type: eval.ParameterTypePrompt,
        Default: prompt.Definition{
            Model: "gpt-5-mini",
            Messages: []prompt.Message{
                prompt.System("You summarize articles in one sentence."),
                prompt.User("Summarize this:\n\n{{input}}"),
            },
            Params: map[string]any{"temperature": 0},
        },
    },
}

prompt.Definition fields:

prompt.Prompt.Build

Renders the prompt’s template variables and returns a *prompt.Built ready to send to a model.

built, err := p.Build(map[string]any{"input": article})
if err != nil {
    return err
}
// built is provider-agnostic. built.Map() returns an OpenAI-shaped map you can send to any client:
body, err := json.Marshal(built.Map())

Returns: (*prompt.Built, error). Build fails if the prompt has no body or no model, the template is malformed, or the format is not supported ("nunjucks" is not rendered locally). For the OpenAI Go client, traceopenai.ChatCompletionParams(built) (from github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai) converts a chat prompt directly into openai.ChatCompletionNewParams.

built.AnnotateSpan

Records the prompt’s identity and rendered variables on a span, linking the model call back to the prompt in Braintrust. Call it before ending the span that covers the model call.

built, err := p.Build(map[string]any{"input": input})
if err != nil {
    return err
}
built.AnnotateSpan(hooks.TaskSpan)

Does nothing when Metadata is nil (the prompt has no Braintrust identity) or the span is not recording.

Datasets

A dataset is the set of cases an evaluation runs against. Define cases inline in memory, or manage datasets in Braintrust through the API client.

eval.NewDataset

Groups cases into an in-memory dataset you pass to Evaluator.Run, as an alternative to loading one from Braintrust.

dataset := eval.NewDataset([]eval.Case[string, string]{
    {Input: "How do I reset my password?", Expected: "Use the account recovery flow."},
    {Input: "How do I export my data?", Expected: "Open Settings and choose Export."},
})

Returns: eval.Dataset[I, R]. Each eval.Case[I, R] has an Input and optional Expected, Tags, Metadata, and TrialCount.

Attachments

When your traces involve binary content like images or PDFs, log it as an attachment so it appears in Braintrust instead of as an opaque blob. When you trace AI calls, Braintrust automatically converts base64 attachments in provider messages into uploaded attachments, so you rarely need the APIs below for instrumented calls. Reach for them when you’re attaching binary content to a span yourself.

attachment.From*

Creates an attachment from bytes, a file, or a URL.

import "github.com/braintrustdata/braintrust-sdk-go/trace/attachment"

att, err := attachment.FromFile("image/png", "chart.png")

Constructors:

API client

For direct access to the Braintrust REST API, use the api package. Reach for it to manage projects, experiments, datasets, and functions programmatically, beyond what the higher-level APIs above cover.

api.NewClient

Creates a REST API client from an API key.

import "github.com/braintrustdata/braintrust-sdk-go/api"

client := api.NewClient(os.Getenv("BRAINTRUST_API_KEY"))

Returns: *api.API. Namespaces:

Configuration

Configure the client with functional options passed to braintrust.New, or with environment variables.

client, err := braintrust.New(tp,
    braintrust.WithProject("My project"),
    braintrust.WithBlockingLogin(true),
)

Client options:

Environment variables