Automation

Everything you do in the app can also be done through Duet's HTTP API: launching runs from CI after every deploy, collecting scores in a script, or letting an AI coding agent run a scenario, read the brief and fix your agent. Three things make that possible.

1. Create an API key#

1. Open Settings → API keys and choose Create API key.

   Name it after where it will be used (“Nightly CI”) and pick when it expires.
2. Copy the key right away.

   It is shown only once. Store it as a secret in your CI or in an environment variable.
3. Send it with every request.

   Authorization: Bearer ve_xxxxxxxx_<secret>

2. Explore the API in Swagger#

Open the API reference (Swagger) from the top of this guide or from API docs in the app's sidebar. It is public, always matches the running version, and lists every endpoint. The raw schema is at /api/openapi.json, handy for generating a client.

A few conventions that hold everywhere:

• Field names are camelCase, ids are UUIDs and times are in UTC.
• Lists of runs and batches are paged and newest first.
• Errors are plain: 401 for a missing or wrong key, 404 for an unknown id, 409 for a conflict (like a name already in use) and 422 for invalid input, pointing at the field.

3. A typical flow#

Launch a batch, wait for it, then read the results:

# Keep the key in an environment variable; never paste it into files or logs.
curl -s -X POST "$DUET_URL/api/runs" \
  -H "Authorization: Bearer $DUET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"scenarioIds": ["<scenario id>"], "repetitions": 3, "concurrency": 1, "label": "nightly"}'

The answer is the new batch with its queued runs. Check it every 10 to 15 seconds until its status is completed or cancelled, then read each run's judgement (allow a few minutes more, up to about 20 with Langfuse):

# The batch: its status and every run's status, judgement status and score
curl -s "$DUET_URL/api/runs/batches/<batch id>" -H "Authorization: Bearer $DUET_API_KEY"

# One run's judgement: score, verdicts, suggestions and the brief
curl -s "$DUET_URL/api/runs/<run id>/judgement" -H "Authorization: Bearer $DUET_API_KEY"

The judgement holds the overall score, whether a must criterion failed, every criterion's verdict and the brief. What each field means is in Scoring. To follow a run turn by turn instead of polling, the API also offers a live WebSocket stream; the Swagger reference and the agent skill describe it.

4. Give an AI agent the skill#

The agent skill teaches an AI coding agent (such as Claude Code) how to use Duet: what scenarios, runs and judgements are, the flows above with exact requests, and how to read results. It always matches the running version, and anyone can fetch it without a key.

Get it

• In the app: Settings → API keys, the Agent skill card. Copy the URLs, copy a ready-made command to save it, or Download skill (.zip).
• From a terminal:

curl -s "$DUET_URL/api/skill" -o SKILL.md          # the guide as one Markdown file
curl -s "$DUET_URL/api/skill.zip" -o voice-evaluator.zip   # a skill folder with the full API reference

Unzip the folder into your agent's skills directory and give the agent an API key through an environment variable. The skill tells the agent to ask you before anything that costs money, like launching runs.

Measuring judge stability#

For the curious: whoever runs the server can judge one finished run several times, without saving anything, to see how much each criterion's band moves. Each repetition costs 3 judge model calls.

docker compose exec backend voice-eval judge-stability <run id> --times 3