Agent Eval

Test Platform

Dashboard
Providers
Mock Users
Eval Test Cases
Workspaces
Agents
Tool Catalog
Greetings
LLM Judges
Test Runner
Test Suites

Checking...

LLM Judges

0 Judges 0 Templates

Loading judges...

⚖

No LLM Judges Configured

Create your first LLM judge to start evaluating agent conversations.

Test LLM Judge

- Threshold: - Score Range: -

Description: -

Agent Configuration

Endpoint

Customer

Workspace

Agent

Test Conversation

Your prompt is sent to the agent, then the agent's response is evaluated by the LLM Judge.

Create LLM Judge

Identity

Name *

Description

Tags

Comma-separated tags for categorization

Evaluation Instructions

Judge Prompt *

The system prompt that guides the LLM judge's evaluation criteria and scoring approach.

Reference Materials

Brand Guidelines / Company Info

Optional context provided alongside the transcript for evaluation.

Provider Configuration

LLM Provider *

The LLM provider used to run the judge evaluation.

Model Override

Override the provider's default model for this judge.

Temperature: 0.3

Lower values produce more consistent evaluations.

Max Tokens

Maximum tokens for the judge's response.

Scope

Transcript Scope *

Which portion of the transcript the judge evaluates.

Sliding Window Size

Number of recent messages to include in the evaluation window.

Scoring

Min Score

Max Score

Pass / Fail

Mode

Threshold: 0.7

Normalized score (0.0 - 1.0) required to pass.

Save as Template

Make this judge available as a starting template for others.

Delete LLM Judge

Are you sure you want to delete ? This action cannot be undone.

Agent Eval - Test Platform