Skip to main content

Verify and monitor production

A successful draft test proves only the draft test path. Every publication needs a new production-path smoke test.

Voice-agent smoke test​

  1. Confirm the intended version is marked Live.
  2. Confirm the inbound phone number is assigned to the correct agent.
  3. Confirm the agent is Active.
  4. Call the assigned number from an external phone.
  5. Test the greeting, one core task, one high-risk action, and the safe ending or transfer.
  6. End the call and open its exact record in Call Logs.

Verify:

  • routing and answer delay;
  • recognition and audio in both directions;
  • prompt and knowledge behavior;
  • function input, result, and external side effect;
  • transfer or hangup behavior;
  • transcript and duration;
  • outcome, summary, and contact enrichment when processing completes; and
  • credits charged against the published rate.

Version pinning​

A live voice call loads the published configuration when the call starts. Publishing while it is in progress does not rewrite that call. Start a new call to verify the new version.

Website Chat is Beta. Its public sessions similarly pin the published version selected at session creation; later turns in an existing session do not switch versions after a publication.

Monitor after release​

Compare a defined post-release window with the baseline:

  • connection and failure rate;
  • task completion and classified outcomes;
  • transfer frequency and destination success;
  • unknown-answer and escalation patterns;
  • function errors and duplicate actions;
  • call duration and latency;
  • transcript recognition errors; and
  • credits per call and total usage.

Post-call analysis and actions are asynchronous. Allow processing time before treating an immediately missing summary or contact update as a final failure.

Decide whether to continue or roll back​

Continue when critical scenarios pass and metrics remain within the release threshold. Pause delivery or restore a known version when the release causes unsafe actions, material misinformation, broken routing, uncontrolled cost, or a significant outcome regression.

Record the decision, affected version, evidence, and follow-up work.