## AutoEvals

AutoEvals is a tool to quickly and easily evaluate AI model outputs.

## Installation

```
pip install autoevals
```

## LLM Evaluators

### Battle

Compare if a solution performs better than a reference solution.

### ClosedQA

Evaluate answer correctness using the model’s knowledge.

### Factuality

Check factual accuracy against a reference.

### Humor

Rate the humor level in text.

### LLMClassifier

High-level classifier for evaluating text using LLMs.

### Possible

Evaluate if a solution is feasible and practical.

### Security

Evaluate if a solution has security vulnerabilities.

### Sql

Compare if two SQL queries are equivalent.

### Summary

Evaluate text summarization quality.

### Translation

Evaluate translation quality.

## String Evaluators

### EmbeddingSimilarity

String similarity scorer using embeddings.

### ExactMatch

A scorer that tests for exact equality between values.

### Levenshtein

String similarity scorer using edit distance.

## Numeric Evaluators

### NumericDiff

Numeric similarity scorer using normalized difference.

## JSON Evaluators

### JSONDiff

Compare JSON objects for structural and content similarity.

### ValidJSON

Validate if a string is valid JSON and optionally matches a schema.

## List Evaluators

### ListContains

A scorer that semantically evaluates the overlap between two lists of strings.

## RAGAS Evaluators

### AnswerCorrectness

Evaluates how correct the generated answer is compared to the expected answer.

### AnswerRelevancy

Evaluates how relevant the generated answer is to the input question.

### AnswerSimilarity

Evaluates how semantically similar the generated answer is to the expected answer.

### ContextEntityRecall

Measures how well the context contains the entities mentioned in the expected answer.

### ContextPrecision

Measures how precise and focused the context is for answering the question.

### ContextRecall

Measures how well the context supports the expected answer.

### ContextRelevancy

Evaluates how relevant the context is to the input question.

### Faithfulness

Evaluates if the generated answer is faithful to the given context.

## Moderation

### Moderation

A scorer that evaluates if AI responses contain inappropriate or unsafe content.

## Other

### DefaultModelConfig

Configuration for default models used by Autoevals.

### LLMClient

A client wrapper for LLM operations that supports both OpenAI SDK v0 and v1.

### Source Code

For the complete Python source code and additional examples, visit the [autoevals GitHub repository](https://github.com/braintrustdata/autoevals).