Skip to content

agent simulate: read simulator flow nodes from scenarios.yaml - #1031

Open
chenghao-mou wants to merge 7 commits into
mainfrom
chenghao/feat/AGT-3725-public-simulator-flow-schema
Open

chenghao-mou wants to merge 7 commits into
mainfrom
chenghao/feat/AGT-3725-public-simulator-flow-schema

Conversation

@chenghao-mou

@chenghao-mou chenghao-mou commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

lk agent simulate reads simulator flows from scenarios.yaml: a top-level nodes map and start on a scenario. They go to the server in the ScenarioGroup added by livekit/protocol#1856.

nodes:
  # a menu: single-key routes, checked in order
  menu:
    say: "Thanks for calling Acme Dental. For appointments, press 1. To pay a bill, press 2."
    speaker: MACHINE
    routes:
      - { match: "1", to: hold }
      - { match: "2", to: card }
      - { match: "[0-9*#]", to: invalid }  # any other key
    timeout: { after: 8s, to: menu }       # no key: play again, at most 3 times in all
  invalid:
    say: "Sorry, that is not a valid option."
    speaker: MACHINE
    then: menu                             # when say ends
  # a multi-key entry: keys collect until #
  card:
    say: "Enter your 16-digit card number, then press pound."
    speaker: MACHINE
    routes:
      - { match: '\d{16}', to: paid }
      - { match: '[0-9*]+', to: card }     # anything else: ask again
    timeout: { after: 15s, to: livekit/hangup }
  paid:
    say: "Thank you, your payment is complete. Goodbye."
    speaker: MACHINE
    then: livekit/hangup
  # silence: only the clock moves on
  hold:
    say: "Please hold while I connect you."
    speaker: MACHINE
    timeout: { after: 4s, to: screener }
  # converses in a machine voice; `then` once the agent replies, `timeout` if it never does
  screener:
    say: "This is Sam's assistant. May I ask who is calling?"
    instructions: You screen calls for Sam. Once the caller says who they are and why, thank them.
    speaker: MACHINE
    then: sam
    timeout: { after: 10s, to: livekit/hangup }
  # a person, in the scenario's voice; hangs up after 15s without the agent speaking
  sam:
    say: "Hi, this is Sam."
    instructions: You are Sam Lee. Confirm tomorrow's 10am cleaning, then say goodbye.
    timeout: { after: 15s, to: livekit/hangup }
  # a person's recorded greeting, then the built-in voicemail box
  greeting:
    say: "Hi, you've reached Sam. Leave a message after the beep."
    then: livekit/voicemail
scenarios:
  - label: menu-to-sam
    start: menu
    agent_expectations: Presses 1, stays silent on hold, tells the screener who is calling, and confirms the appointment with Sam.
  - label: pay-bill
    start: menu
    userdata: { card_number: "4242424242424242" }
    agent_expectations: Presses 2, enters the card number followed by #, and ends the call after the payment.
  - label: voicemail
    start: greeting
    agent_expectations: Waits for the greeting, then leaves one short message.
  - label: number-not-in-service
    start: livekit/unavailable
    agent_expectations: Does not leave a message and ends the call.
  • Each node is decoded with protojson, so field names, enum names, durations, and unknown keys behave exactly as on the server. An error names the node, for example node "menu": ....
  • The CLI checks two things locally: a scenario does not set both start and instructions (one with neither is a plain persona, as on main), and start names a node or a built-in (livekit/hangup, livekit/voicemail, livekit/unavailable). The server compiles and validates the rest at CreateSimulationRun.
  • The YAML export writes nodes and start back, and adding scenario ids leaves nodes untouched. Files without flows are unchanged.

Warning

Depends on livekit/protocol#1856; go.mod points at its branch until it ships.

Not in this PR: the public OpenAPI client in pkg/public/oapi, which regenerates after the protocol change ships. Copying a flow scenario from a run in the TUI still yields a stub without its nodes.

Addresses AGT-3725

@chenghao-mou chenghao-mou changed the title agent simulate: read simulator flow steps from scenarios.yaml agent simulate: read simulator flow nodes from scenarios.yaml Oct 13, 2026
@chenghao-mou
chenghao-mou added this pull request to stack #1068 October 21, 2026 11:50
@chenghao-mou
chenghao-mou marked this pull request as ready for review October 21, 2026 11:50
Read the top-level `steps` map and the per-scenario `start` and
`max_duration` keys, and send them in the ScenarioGroup. Each step is
decoded with protojson, so field names, enum names, durations, and
unknown-key errors match the server's parser.

Check only clear user errors locally: a scenario must set exactly one
of `start` and `instructions`, and `start` must name a step or a
built-in. The server validates the rest at CreateSimulationRun.

Bump github.com/livekit/protocol to b440b893 for the new fields.

Addresses AGT-3725
Follows the protocol rename: scenarios.yaml takes a top-level `nodes` map,
and errors say node. A node can no longer be a voicemail box; scenarios
reach one through the built-in livekit/voicemail.
Menus now repeat at most three times inside the simulator, so the
schema no longer has max_visits.
@chenghao-mou
chenghao-mou force-pushed the chenghao/feat/AGT-3725-public-simulator-flow-schema branch from abb92b6 to 62f05ec Compare October 21, 2026 11:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant