Build a Two-Phase Tool Boundary for a Tencent RTC Voice Companion

Build a Two-Phase Tool Boundary for a Tencent RTC Voice Companion

Tópico: Build a Two-Phase Tool Boundary for a Tencent RTC Voice Companion
Categoria: Tutoriais | Programação & Tecnologia
Idioma Principal: Português (Conteúdo de Tecnologia)

Descrição do Conteúdo / Informações:
-------------------------------------------------------------------------
A voice companion becomes genuinely useful when it can do something: post a room message, add an item to a queue, update a profile, or call another service.

That is also where a convincing demo can become an unreliable product.

Suppose a user says:

Tell the room I'll leave at eight—actually, don't send that.

The model may have produced the correct tool arguments before the interruption arrived. If tool execution is coupled directly to model output, the message can be published while the companion is still saying, "Sure." A better prompt might reduce the frequency, but it cannot create a transaction boundary.

The uncomfortable engineering reality is not that you are "bad at prompting." The system is missing an enforceable state between the model proposing an action and the application committing it.

In this tutorial, we will build that boundary for a Tencent RTC conversational AI scenario. The companion may use an OpenAI-compatible model or an agent platform such as Dify, but neither provider receives direct authority to commit the action.

The invariant we want

Our application will enforce one rule:

Model output may prepare an action, but only a fresh, explicit user confirmation may commit it.

The resulting path is:

RTC audio
-> speech recognition
-> application turn coordinator
-> LLM or Dify
-> validated action proposal
-> server-side prepared ticket
-> spoken preview
-> explicit user confirmation
-> permission recheck
-> idempotent commit

Tencent RTC documents its Conversational AI scenario as real-time voice interaction that can connect with multiple LLM providers. Its LLM configuration documentation covers OpenAI-compatible models and agent platforms including Dify, as well as request identifiers useful for routing and observability:

• Tencent Conversational AI overview

• Large Language Model configuration

The media, speech, model, and application authorization layers remain distinct. The code below lives in the application orchestration layer; it does not invent a new RTC or messaging API.

Decide which tools need this boundary

Not every model operation needs spoken confirmation. Classify each application-defined tool before exposing it to the model:

Tool effect
Example
Default policy

Read-only
Search a public catalog
Allow with normal validation

Local and reversible
Change the companion's temporary voice style
Allow or provide Undo

Shared and reversible
Add a track to a room queue
Confirm when social impact is meaningful

External or audience-visible
Publish a room message
Require explicit confirmation

Sensitive or difficult to reverse
Purchase, delete, invite, disclose private data
Strong confirmation or do not expose to the agent

This tutorial uses an application-defined publishRoomMessage action. Tencent RTC's social entertainment material includes AI companions, voice rooms, communities, and character dialogue as relevant scenarios, but the tool and authorization policy remain ours: Social Entertainment solution.

Create the TypeScript project

mkdir voice-action-boundary
cd voice-action-boundary
npm init -y
npm install --save-dev typescript tsx @types/node
npx tsc --init
mkdir src

Add scripts to package.json:

{
"scripts": {
"start": "tsx src/demo.ts",
"test": "tsx --test src/*.test.ts"
}
}

Start with explicit action state

Do not represent the whole interaction with booleans such as isLoading, isTalking, and isConfirmed. Their invalid combinations multiply quickly.

// src/types.ts
export type MessageProposal = {
kind: 'publishRoomMessage';
roomId: string;
text: string;
};

export type PreparedAction = {
ticket: string;
turnId: string;
proposal: MessageProposal;
preview: string;
expiresAt: number;
};

export type VoiceActionState =
| { kind: 'idle' }
| { kind: 'requesting-model'; turnId: string }
| { kind: 'previewing'; action: PreparedAction }
| { kind: 'awaiting-confirmation'; action: PreparedAction }
| { kind: 'executing'; action: PreparedAction }
| { kind: 'completed'; turnId: string }
| {
kind: 'recovery';
turnId: string;
reason: 'expired' | 'denied' | 'provider-error' | 'commit-uncertain';
};

This union makes several forbidden states unrepresentable. An action cannot simultaneously be awaiting confirmation and completed, for example.

The ticket is important. It identifies one prepared action rather than giving the model a reusable tool credential.

Validate proposals outside the model

A prompt can tell the model to return JSON, but the application still has to treat that JSON as untrusted input.

// src/proposal.ts
import type { MessageProposal } from './types.js';

export function parseProposal(value: unknown): MessageProposal {
if (!value || typeof value !== 'object') {
throw new Error('Proposal must be an object');
}

const candidate = value as Record<string, unknown>;

if (candidate.kind !== 'publishRoomMessage') {
throw new Error('Unsupported action kind');
}

if (typeof candidate.roomId !== 'string' || !candidate.roomId.trim()) {
throw new Error('Invalid room ID');
}

if (typeof candidate.text !== 'string') {
throw new Error('Message text is required');
}

const text = candidate.text.trim();
if (text.length === 0 || text.length > 280) {
throw new Error('Message must contain between 1 and 280 characters');
}

return {
kind: 'publishRoomMessage',
roomId: candidate.roomId,
text
};
}

The room ID should not normally come from model imagination. Compare it with trusted session context before preparing the action.

Put the commit capability behind a broker

The broker issues short-lived tickets, rechecks permissions at commit time, and prevents the same ticket from being committed concurrently.

// src/broker.ts
import { randomUUID } from 'node:crypto';
import type { MessageProposal, PreparedAction } from './types.js';

type SessionContext = {
userId: string;
roomId: string;
};

type RecordState =
| 'prepared'
| 'executing'
| 'committed'
| 'cancelled'
| 'uncertain';

type StoredAction = {
ownerId: string;
action: PreparedAction;
state: RecordState;
};

export interface RoomPublisher {
publish(
roomId: string,
text: string,
options: { idempotencyKey: string }
): Promise<void>;
}

export class ActionBroker {
private records = new Map<string, StoredAction>();

constructor(
private readonly publisher: RoomPublisher,
private readonly canPublish: (context: SessionContext) => Promise<boolean>,
private readonly now: () => number = Date.now
) {}

prepare(
proposal: MessageProposal,
turnId: string,
context: SessionContext
): PreparedAction {
if (proposal.roomId !== context.roomId) {
throw new Error('Proposal targeted a different room');
}

const ticket = randomUUID();
const action: PreparedAction = {
ticket,
turnId,
proposal,
preview: `Post this message to the room: ${proposal.text}`,
expiresAt: this.now() + 30_000
};

this.records.set(ticket, {
ownerId: context.userId,
action,
state: 'prepared'
});

return action;
}

cancel(ticket: string): void {
const record = this.records.get(ticket);
if (record?.state === 'prepared') record.state = 'cancelled';
}

async commit(ticket: string, context: SessionContext): Promise<void> {
const record = this.records.get(ticket);
if (!record) throw new Error('Unknown action ticket');

if (record.ownerId !== context.userId) {
throw new Error('Ticket belongs to another user');
}

if (record.action.proposal.roomId !== context.roomId) {
throw new Error('Room context changed');
}

if (record.state === 'committed') return;
if (record.state !== 'prepared') {
throw new Error(`Action cannot commit from ${record.state}`);
}

if (this.now() >= record.action.expiresAt) {
record.state = 'cancelled';
throw new Error('Action ticket expired');
}

if (!(await this.canPublish(context))) {
record.state = 'cancelled';
throw new Error('Permission denied at commit time');
}

record.state = 'executing';

try {
await this.publisher.publish(
context.roomId,
record.action.proposal.text,
{ idempotencyKey: ticket }
);
record.state = 'committed';
} catch (error) {
record.state = 'uncertain';
throw error;
}
}
}

The Map keeps the tutorial easy to run. In production, prepared tickets and transitions should use durable storage or another atomic coordination mechanism. Otherwise, a process restart can erase whether an external operation succeeded.

The publishing adapter must also honor the idempotency key if automatic retries are allowed. If the downstream service times out and provides no idempotency guarantee, the correct state is uncertain, not "failed." Retrying blindly could publish twice.

Keep confirmation deliberately boring

Do not ask the LLM whether the user confirmed its own proposal. Use a narrow recognizer for the authorization event.

// src/confirmation.ts
export type Confirmation = 'yes' | 'no' | 'ambiguous';

export function classifyConfirmation(transcript: string): Confirmation {
const normalized = transcript
.toLowerCase()
.replace(/[^a-z\s]/g, '')
.replace(/\s+/g, ' ')
.trim();

if (['yes', 'yes post it', 'confirm', 'send it'].includes(normalized)) {
return 'yes';
}

if (['no', 'cancel', 'dont send it', 'do not send it'].includes(normalized)) {
return 'no';
}

return 'ambiguous';
}

A strict vocabulary adds conversational friction, especially when speech recognition is uncertain. That is an intentional trade-off for consequential actions. A visible Confirm/Cancel control is a useful fallback and should feed the same state machine rather than bypassing it.

Coordinate preview, interruption, and commit

The voice controller receives transcripts and speech lifecycle events from adapters. Exact SDK wiring depends on the client platform, so the interfaces below mark the integration seams without inventing product API names.

// src/controller.ts
import { classifyConfirmation } from './confirmation.js';
import type { ActionBroker } from './broker.js';
import type { PreparedAction, VoiceActionState } from './types.js';

type Context = { userId: string; roomId: string };

type SpeechOutput = {
speak(text: string): Promise<void>;
stop(): void;
};

export class VoiceActionController {
private state: VoiceActionState = { kind: 'idle' };

constructor(
private readonly broker: ActionBroker,
private readonly speech: SpeechOutput,
private readonly context: Context
) {}

snapshot(): VoiceActionState {
return this.state;
}

async present(action: PreparedAction): Promise<void> {
this.state = { kind: 'previewing', action };
await this.speech.speak(`${action.preview}. Say yes to confirm or no to cancel.`);

if (this.state.kind === 'previewing' &&
this.state.action.ticket === action.ticket) {
this.state = { kind: 'awaiting-confirmation', action };
}
}

onUserSpeechStarted(): void {
if (this.state.kind === 'previewing') {
this.speech.stop();
this.state = {
kind: 'awaiting-confirmation',
action: this.state.action
};
}
}

async onTranscript(transcript: string): Promise<void> {
if (this.state.kind !== 'awaiting-confirmation') return;

const action = this.state.action;
const answer = classifyConfirmation(transcript);

if (answer === 'no') {
this.broker.cancel(action.ticket);
this.state = { kind: 'idle' };
await this.speech.speak('Cancelled. Nothing was posted.');
return;
}

if (answer === 'ambiguous') {
await this.speech.speak('I did not get a clear yes or no. The action is still waiting.');
return;
}

this.state = { kind: 'executing', action };

try {
await this.broker.commit(action.ticket, this.context);
this.state = { kind: 'completed', turnId: action.turnId };
await this.speech.speak('Posted.');
} catch {
this.state = {
kind: 'recovery',
turnId: action.turnId,
reason: 'commit-uncertain'
};
await this.speech.speak(
'I could not verify whether that completed. I will not retry it automatically.'
);
}
}
}

Notice what interruption means here. If the user starts speaking during the preview, playback stops immediately, but the action does not execute. The resulting transcript still has to contain an accepted confirmation.

Once an external commit has started, interruption cannot magically roll it back. For tools that support compensation, model that as a separate authorized action. Do not tell the user "cancelled" merely because local audio stopped.

Connect OpenAI-compatible models or Dify safely

The LLM adapter needs only enough authority to return a proposal:

export interface LlmAdapter {
propose(input: {
requestId: string;
transcript: string;
roomId: string;
allowedActions: readonly ['publishRoomMessage'];
}): Promise<unknown>;
}

Use one application-generated request ID per voice turn and carry it through model routing and logs. The Tencent RTC LLM configuration documentation should be the implementation reference when connecting the selected OpenAI-compatible provider or Dify workflow: Large Language Model configuration.

A suitable model instruction can request structured output, but it is not the security control:

Return either a conversational response or a proposal for one allowed action.
Never claim an action completed. The application will validate, confirm, and execute it.

The demonstrated AI capability is interpreting the user's language and proposing structured arguments. The unsupported leap is assuming that linguistic confidence equals authorization. Human control resides in the ticket, confirmation event, permission check, and commit path—not in the model's wording.

For Dify, apply the same rule: a workflow may produce the proposal, but it should not receive the application credential that publishes the message. For an OpenAI-compatible model, do not expose commit(ticket) as another model-selected tool. The application coordinator owns that transition.

Verify behavior with failure-oriented tests

A happy-path voice conversation proves very little. Start with the cases where callbacks overlap.

// src/controller.test.ts
import test from 'node:test';
import assert from 'node:assert/strict';
import { ActionBroker } from './broker.js';
import { VoiceActionController } from './controller.js';

const context = { userId: 'user-1', roomId: 'room-1' };

function fixture() {
const published: string[] = [];
const spoken: string[] = [];

const broker = new ActionBroker(
{
async publish(_roomId, text) {
published.push(text);
}
},
async () => true
);

const controller = new VoiceActionController(
broker,
{
async speak(text) { spoken.push(text); },
stop() {}
},
context
);

const action = broker.prepare(
{
kind: 'publishRoomMessage',
roomId: 'room-1',
text: 'I will leave at eight.'
},
'turn-1',
context
);

return { broker, controller, action, published, spoken };
}

test('an interruption followed by no never publishes', async () => {
const f = fixture();
const presenting = f.controller.present(f.action);

f.controller.onUserSpeechStarted();
await f.controller.onTranscript('No, do not send it');
await presenting;

assert.deepEqual(f.published, []);
assert.equal(f.controller.snapshot().kind, 'idle');
});

test('ambiguous speech does not become consent', async () => {
const f = fixture();
await f.controller.present(f.action);
await f.controller.onTranscript('Maybe change eight to nine');

assert.deepEqual(f.published, []);
assert.equal(f.controller.snapshot().kind, 'awaiting-confirmation');
});

test('explicit confirmation commits once', async () => {
const f = fixture();
await f.controller.present(f.action);
await f.controller.onTranscript('Yes, post it');
await f.controller.onTranscript('Yes, post it');

assert.deepEqual(f.published, ['I will leave at eight.']);
assert.equal(f.controller.snapshot().kind, 'completed');
});

Run them with:

npm test

Then add integration tests around your actual speech and publishing adapters.

Failure drills to run before release

The LLM returns a different room ID

Reject the proposal during preparation. Never let model output select an authorization scope that disagrees with the authenticated RTC session.

The user loses permission after hearing the preview

Recheck authorization at commit time. Preparation is not a permanent permission grant.

Speech recognition produces "yes" from background audio

Require a narrow confirmation phrase, correlate it with the active ticket, and offer a button fallback. For higher-risk actions, voice-only confirmation may be insufficient.

The ticket expires while the user is thinking

Cancel it and generate a new preview if the user still wants the action. Do not silently extend old authority.

The model provider times out

Return to a recoverable conversational state. No ticket exists, so there is nothing to execute. A retry should keep the same turn correlation while avoiding duplicate prepared actions.

The publish request times out after reaching the server

Mark the result uncertain. Query by idempotency key if the downstream system supports that operation. Otherwise, escalate to a visible recovery choice instead of automatically publishing again.

The RTC connection drops during confirmation

Allow the ticket to expire. Reconnecting the media session must not reinterpret an old transcript or delayed callback as fresh consent.

A prompt injection asks the model to skip confirmation

It cannot. The model has proposal authority only; it has no commit capability. This is the difference between describing a boundary in a prompt and enforcing one in code.

A practical release checklist

Before enabling a consequential companion tool, verify that:

• [ ] Model output is parsed and validated as untrusted data.

• [ ] Trusted session context determines user and room scope.

• [ ] The model can propose but cannot commit the action.

• [ ] Prepared tickets are single-purpose, short-lived, and user-bound.

• [ ] The exact effect is previewed before confirmation.

• [ ] Ambiguous speech never counts as consent.

• [ ] Barge-in stops playback without committing the action.

• [ ] Permissions are checked again immediately before execution.

• [ ] Duplicate confirmation callbacks cannot duplicate the effect.

• [ ] Timeouts can represent an uncertain result without inventing success or failure.

• [ ] Logs correlate RTC turn, model request, prepared ticket, and commit result without storing unnecessary private audio or text.

• [ ] The interface provides a visible stop, cancel, or confirmation control.

The skill that remains valuable

As models become better at selecting tools, the developer's role does not shrink to prompt polishing. The harder and more durable work is deciding what authority exists, when it becomes valid, how it expires, and what the user sees when certainty is impossible.

A smooth voice is a presentation layer. Trust comes from the state machine behind it.

Relationship disclosure: I'm writing this article in connection with Tencent RTC, and I used the official Tencent RTC documentation linked above as the implementation reference.