📊 Analytics & ROI Dashboard
Eval Studio — Benchmark Agents Before Ship
Datasets, eval runs, model comparisons, and export. Fail-closed if schema unavailable. Pro quotas apply.
Sign in required
This guide covers in-app features. Sign in or create an account to follow along in your workspace.
Updated July 19, 2026
6 min read
Eval Studio is separate from Swarm simulation — it measures agent quality on repeatable test sets.
Fail-closed availability
If eval framework tables are missing, APIs return 503 / EVAL_NOT_AVAILABLE instead of an empty green success. That prevents shipping against a silently broken eval surface.
Workflow
1
Create or import a dataset (CSV/JSON items with expected outputs).
2
Select the agent or prompt configuration to test.
3
Run eval and review accuracy, latency, and cost per item.
4
Export results for compliance or regression tracking.
When to Use Eval vs Simulator
- Eval — "Is this agent accurate on known test cases?"
- Simulator — "Does this Swarm workflow route correctly?"
Was this helpful?