Missing scorers in automation rules editor - Braintrust

Applies to:

Summary

Issue: When editing an automation rule, previously selected scorers do not appear in the scorer dropdown, though they remain applied to the rule and work correctly. Newly added scorers display properly, but existing selections are invisible in the UI.

Cause: The scorer lives in a different project that the user cannot read. The automation rules editor only displays scorers it can fetch, so a previously selected scorer from an unreadable project cannot be resolved and does not appear, even though the rule still applies it at runtime.

Resolution: Grant the user read access to the project where the scorer is defined. Organization-wide read access is not required.

Resolution steps

For organization administrators

Step 1: Identify the scorer’s project

Determine which project the missing scorer is defined in. This is the project you grant read access to.

Step 2: Grant project-level read access

Add the affected user to a permission group with Read access to that project (Settings > Permission groups). The scorer then appears in the automation rules editor and other dropdowns, without elevating the user to organization-wide read. To expose only scorers rather than all project resources, scope the permission group to read only the project’s prompts, where scorers are stored.

For users without admin access

Step 1: Contact your organization administrator

Request read access to the project where the scorer is defined.

Step 2: Verify scorer visibility

After the permission change is applied, refresh the automation rules page and confirm the scorers appear in the dropdown. If the dropdown loads scorers from more than one project, click Refresh in the dropdown to load the latest.