Skip to content
DuetAgent evals
Section · Getting started

Getting started

From an empty workspace to your first judged run, voice or text, step by step.

This page takes you from nothing to your first judged run. It follows the order the app expects: credentials first, then the agent, then a scenario, then a launch. Plan for about 20 minutes, most of it writing your first scenario.

Before you start

  • A voice agent that speaks over a WebSocket (the same connection your web or phone client uses), or a text agent reachable over Vivi's channel API.
  • An OpenAI or Azure OpenAI account: Duet uses it for the caller's voice, the evaluator's brain and the judge (a text agent needs no voice).
  • Optional: a Langfuse project where your agent writes its traces, so the judge can also check tool calls and errors.

1. Run Duet

If someone already runs Duet for your team, skip to the next step. Otherwise, whoever manages the server starts it with Docker from the project folder:

cp .env.example .env          # then fill in POSTGRES_PASSWORD and VOICE_EVAL_MASTER_KEY
docker compose up --build -d  # starts the database, backend, web app and proxy
docker compose exec backend voice-eval create-user --email [email protected] --display-name "You"

The last command creates the first user (there is no public sign-up) and asks for a password of at least 12 characters. The web app is then at http://localhost:3000.

2. Sign in

Sign in with the email and password your administrator created for you. You land on Overview, which stays empty until the first run.

3. Add your connections

A connection holds the credentials Duet uses to reach a provider. Secrets are stored encrypted and are never shown again; only their last 4 characters appear as a hint.

  1. Open Settings → Connections and choose Add connection.
    Pick the Provider: OpenAI, Azure OpenAI, Azure AI Speech or Langfuse.
  2. Give it a name and fill in its fields.
    For OpenAI, the API key is enough; the defaults for the voice model and voice are fine. For Azure OpenAI, add the endpoint, the API version and your deployments (a TTS deployment for the voice, and a chat deployment if it will also be the caller's brain).
  3. Press Test connection, then Create connection.
    The test makes one small request to the provider and tells you right away if a key or address is wrong.

4. Add your agent

The agent is the assistant you want to test. Go to Agents, choose Add agent and keep the Voice kind for a voice agent (for a chat agent, see the next section).

  1. Import from cURL (the quickest way).
    In your browser, open your agent's web client, open the developer tools, find its WebSocket request in the Network tab, right-click it and copy it as cURL. Paste it into Import from cURL and press Import: the WebSocket URL, Channel ID, User ID and Origin are filled in for you.
  2. Name it and describe what it does.
    The description (What this agent does) is shared by every scenario for this agent: the evaluator and the judge read it before each call.
  3. Leave Protocol on Auto-detect.
    Duet works out on each run which voice protocol your agent speaks. Pick one only to skip that check.
  4. Optionally link a Langfuse connection, then Create agent.
    With Langfuse linked, the judge also reads the agent's traces for each call.
  5. Press Run connection test.
    Duet dials the agent like a real run, waits until the session is ready (at most 8 seconds) and hangs up without sending audio. You see each step and how long the connection took.

Adding a text agent

A text agent is reached over Vivi's channel API: Duet sends each message as an HTTP request to {base URL}/channelsApi/{channelId}/stream (or /invoke) and reads the reply. In Add agent, choose the Text kind.

  1. Import from cURL, or fill in the fields.
    Paste a cURL command of a request to /channelsApi/…/invoke or /channelsApi/…/stream: the base URL, Channel ID, mode, Origin and, if the command carries a vivi-api-key header, the API key are filled in. Otherwise enter the Base URL (https; plain http only for local hosts) and the channel's UUID as Channel ID.
  2. Pick the mode.
    Stream gets the reply as it is written, and latency is the time to the first text. Invoke gets the whole reply in one response, and latency is the time to the full reply. Agents with a response schema only work with Invoke.
  3. Pick the authentication.
    None for a WEB channel, such as the portal's testing channel of each agent: it needs no credentials (if the channel checks the Origin, set Origin to an allowed domain). API key for an API channel: the key is sent as the vivi-api-key header, stored encrypted and never shown again.
  4. Create the agent and press Run connection test.
    The test sends one short message in a fresh conversation and waits for the whole reply. That message does reach your agent.

5. Write your first scenario

A scenario is one kind of call. Go to Scenarios and choose New scenario. The editor has seven short sections:

  1. Target agent and context
    Choose the agent and, if needed, add context that only this scenario needs.
  2. Persona the evaluator plays
    A name, a locale such as en-US or es-MX (the caller speaks that language), a temperament such as polite, hurried or angry, and notes on habits.
  3. Goal and instructions
    What the caller wants (“Get my lost card blocked”), how they behave, and the facts they know, like the last four digits of the card. Facts are only revealed when the agent asks.
  4. Success criteria for the judge
    What a good call looks like, one sentence each, with a weight. Weights must add up to exactly 100. Mark the essential ones as Must.
  5. When the call ends
    The maximum duration, the maximum silence and when the caller hangs up. For a text agent: the maximum duration, the maximum turns and when the evaluator ends the conversation.
  6. Interruptions
    Optional, voice agents only: make the caller cut the agent off mid-reply.
  7. Evaluator
    How the caller speaks. Keep Pipeline, choose your OpenAI or Azure OpenAI connection as the Evaluator voice, and leave the evaluator model on Same as voice connection.

Press Create scenario. You can come back and edit it any time; past runs keep the version they were launched with.

6. Launch

From the scenario (or from the list, with Launch), the launch panel asks how many runs to make and how many may run at the same time. It shows how many calls that is and a rough wall-time estimate.

Example launch panel for one scenario: 3 runs, one at a time.

7. Watch it live

After launching, open Live sessions. Each run in progress has a card; open one to see the live transcript, the evaluator's intent (what it is about to say, and why), and its metrics. For a call, press Listen to hear it, and choose the agent, the evaluator or both; End call stops it early. For a text agent the chat streams in as the agent writes, with the tools it runs; End conversation stops it early.

8. Read the report

When the run ends, the judge scores it automatically. It usually takes a few minutes, and up to about 20 with Langfuse (traces take a while to become available). Open the run from Runs & reports to see:

  • The overall score, how many goals were met and the judge's summary.
  • The call recording (or, for a text agent, the conversation metrics), the transcript and the scorecard, one row per criterion with evidence.
  • The agent's Langfuse trace, the suggestions, and the handoff brief for a coding agent.

Learn to read every part in Scoring.