Hugging Face - Braintrust

Hugging Face Inference

Hugging Face Inference provides a unified interface to LLMs, embeddings, and other models hosted on Hugging Face and routed providers. Braintrust traces chat completions, text generation, and feature extraction calls.

Setup

Install the Braintrust and @huggingface/inference packages, then set your API keys. Requires @huggingface/inference v2.0.0 or later (any 2.x, 3.x, or 4.x release).

  1. Install packages

    pnpm add braintrust @huggingface/inference

    npm install braintrust @huggingface/inference

  2. Set environment variables

   BRAINTRUST_API_KEY=<your-braintrust-api-key>
   HUGGINGFACE_API_KEY=<your-huggingface-token>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
   # For self-hosted deployments, use your data plane URL
   # BRAINTRUST_API_URL=<your-braintrust-api-url>

Auto-instrumentation

To trace Hugging Face Inference SDK calls without modifying your application code, initialize Braintrust normally, then run your app with Braintrust’s import hook to patch the SDK at runtime.

  1. Initialize Braintrust and call Hugging Face
   import { initLogger } from "braintrust";
   import * as huggingFace from "@huggingface/inference";

initLogger({
     projectName: "huggingface-example",
     apiKey: process.env.BRAINTRUST_API_KEY,
   });

const client = new huggingFace.InferenceClient(
     process.env.HUGGINGFACE_API_KEY,
   );
   
   const response = await client.chatCompletion({
     model: "meta-llama/Llama-3.1-8B-Instruct",
     provider: "featherless-ai",
     messages: [
       {
         role: "user",
         content: "Reply with exactly OK.",
       },
     ],
     max_tokens: 16,
     temperature: 0,
   });
   
   console.log(response.choices?.[0]?.message?.content);
  1. Run with the import hook
   node --import braintrust/hook.mjs trace-huggingface-auto.js

The auto-instrumentation example uses plain JavaScript so node --import can run the file directly. The Braintrust APIs work the same in TypeScript projects — compile your TypeScript to JavaScript, then run the compiled file with the import hook.

If you’re using a bundler, see Trace LLM calls for plugin and loader setup.

Manual instrumentation

To trace Hugging Face clients manually, wrap them yourself with wrapHuggingFace(). Use this when you want to trace selected clients, or wrap the module directly before constructing clients.

import { initLogger, wrapHuggingFace } from "braintrust";
import * as huggingFace from "@huggingface/inference";

initLogger({
  projectName: "huggingface-example",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const hf = wrapHuggingFace(huggingFace);
const client = new hf.InferenceClient(process.env.HUGGINGFACE_API_KEY);

const embedding = await client.featureExtraction({
  inputs: "Paris France",
  model: "thenlper/gte-large",
  provider: "hf-inference",
});

console.log(embedding);

wrapHuggingFace() can wrap either:

If you use routed or custom endpoints via client.endpoint(...), Braintrust records the endpoint URL in span metadata.

What Braintrust traces

Braintrust traces these @huggingface/inference SDK calls, capturing:

Resources

Transformers.js

Transformers.js (@huggingface/transformers) runs models locally in Node.js or the browser without a remote API call. Braintrust traces text generation, text-to-text generation, summarization, feature extraction, and question answering pipelines. Requires @huggingface/transformers v3.0.0 or later (any 3.x or 4.x release).

Setup

Install Braintrust alongside @huggingface/transformers, then set your API key.

  1. Install packages

    pnpm add braintrust @huggingface/transformers

    npm install braintrust @huggingface/transformers

  2. Set environment variables

   BRAINTRUST_API_KEY=<your-braintrust-api-key>

### Auto-instrumentation

To trace Transformers.js pipeline calls without modifying your application code, initialize Braintrust normally, then run your app with Braintrust’s import hook to patch the SDK at runtime.

1. Initialize Braintrust and call a pipeline

```javascript
   import { initLogger } from "braintrust";
   import { pipeline } from "@huggingface/transformers";

initLogger({
     projectName: "transformers-example", // Replace with your project name
     apiKey: process.env.BRAINTRUST_API_KEY,
   });

const generator = await pipeline(
     "text-generation",
     "Xenova/distilgpt2",
   );

const result = await generator("Hello, world!", { max_new_tokens: 20 });
   console.log(result);
  1. Run with the import hook
   node --import braintrust/hook.mjs trace-transformers-auto.js

If you’re using a bundler, see Trace LLM calls for plugin and loader setup.

Manual instrumentation

To trace Transformers.js calls manually, wrap the module or a pipeline instance with wrapHuggingFaceTransformers().

import { initLogger, wrapHuggingFaceTransformers } from "braintrust";
import * as transformers from "@huggingface/transformers";

initLogger({
  projectName: "transformers-example", // Replace with your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const tracedTransformers = wrapHuggingFaceTransformers(transformers);

const generator = await tracedTransformers.pipeline(
  "text-generation",
  "Xenova/distilgpt2",
);

const result = await generator("Hello, world!", { max_new_tokens: 20 });
console.log(result);

wrapHuggingFaceTransformers() can wrap either:

What Braintrust traces

Braintrust traces these @huggingface/transformers pipeline calls, capturing:

Resources