Benchmark the Moderator, Not the Demo: A Safer AI Voice Room with Tencent RTC

A moderation bot can look remarkably capable in a controlled voice-room demo. The difficult question is what happens when its recommendation is late, the reported speaker has already left the mic, or the bot attempts to speak without a seat.

That creates a real tension for developers: you may want AI to reduce a moderator’s workload, but you do not want model confidence to become room authority.

The useful reframe is to stop evaluating the system as one intelligent character. Evaluate it as a protocol with separate responsibilities:

In this tutorial, we will build the application-owned control layer around that protocol. The result is a small moderation benchmark you can run repeatedly before connecting a model or live room.

Our example room has human speakers and an optional AI moderator participant. The AI can submit a recommendation such as “mute this occupied seat,” but it cannot approve the recommendation—even if its participant role is labeled moderator.

If the AI needs to read a warning aloud, it follows the same floor rules as everyone else:

This separates two commonly conflated permissions:

Tencent RTC’s Voice Chat Room material describes the scenario in terms of rooms, mic seats, audio streams, moderation, and related room capabilities. Its implementation guide builds room and mic-seat behavior around LiveListStore and LiveSeatStore. Those documented concepts are the integration boundary used here:

We will not invent SDK methods. Instead, the tutorial creates a testable domain layer that an adapter can connect to the documented stores and operations used by your application.

Use a recent Node.js installation, then create a TypeScript project with Vitest:

mkdir voice-room-moderation-bench cd voice-room-moderation-bench npm init -y npm install --save-dev typescript vitest @types/node npm pkg set scripts.test = "vitest run" npx tsc --init mkdir src Enter fullscreen mode Exit fullscreen mode Add this minimal compiler configuration to tsconfig.json:

{ "compilerOptions": { "target": "ES2022", "module": "ESNext", "moduleResolution": "Bundler", "strict": true, "noUncheckedIndexedAccess": true, "skipLibCheck": true }, "include": [ "src" ] } Enter fullscreen mode Exit fullscreen mode Represent authority and seat state separately Create src/moderation.ts:

export type ParticipantKind = " human " | " ai "; export type ParticipantRole = " host " | " moderator " | " member "; export type RoomPhase = " open " | " closing " | " closed "; export type ModerationAction = " mute " | " removeFromSeat "; export interface Participant { id: string; kind: ParticipantKind; role: ParticipantRole; } export interface Seat { id: string; occupantId: string | null; muted: boolean; // Application-owned revision assigned whenever a fresh seat // snapshot is accepted. It is not presented as an SDK field. revision: number; } export type ProposalStatus = | " pending " | " applying " | " awaitingObservation " | " verified " | " failed " | " uncertain " | " dismissed "; export interface ModerationProposal { id: string; sourceId: string; targetId: string; action: ModerationAction; reason: string; evidenceRefs: string []; expectedSeatId: string; expectedSeatRevision: number; expiresAt: number; status: ProposalStatus; operationId?: string; reviewedBy?: string; failureReason?: string; } export interface RoomState { phase: RoomPhase; participants: Record < string, Participant >; seats: Record < string, Seat >; proposals: Record < string, ModerationProposal >; } export interface ApplyModerationEffect { type: " APPLY_MODERATION "; operationId: string; proposalId: string; action: ModerationAction; targetId: string; seatId: string; expectedSeatRevision: number; } export interface Transition { state: RoomState; effect: ApplyModerationEffect; } export function canPublishAudio ( state: RoomState, participantId: string ): boolean { if ( state. phase!== " open " ) return false; return Object. values ( state. seats ). some ( ( seat ) => seat. occupantId === participantId && seat. muted === false ); } export function approveProposal ( current: RoomState, proposalId: string, reviewerId: string, operationId: string, now: number ): Transition { const state = structuredClone ( current ); const proposal = state. proposals [ proposalId ]; const reviewer = state. participants [ reviewerId ]; if (! proposal ) throw new Error ( " proposal_not_found " ); if (! reviewer ) throw new Error ( " reviewer_not_found " ); if ( state. phase!== " open " ) throw new Error ( " room_not_open " ); if ( proposal. status!== " pending " ) throw new Error ( " proposal_not_pending " ); if ( now >= proposal. expiresAt ) throw new Error ( " proposal_expired " ); // An AI participant can recommend, but cannot commit an action. if ( reviewer. kind!== " human " ) throw new Error ( " human_review_required " ); if ( reviewer. role!== " host " && reviewer. role!== " moderator " ) { throw new Error ( " reviewer_not_authorized " ); } const seat = state. seats [ proposal. expectedSeatId ]; if (! seat ) throw new Error ( " seat_not_found " ); if ( seat. occupantId!== proposal. targetId ) { throw new Error ( " target_no_longer_on_expected_seat " ); } if ( seat. revision!== proposal. expectedSeatRevision ) { throw new Error ( " stale_seat_revision " ); } proposal. status = " applying "; proposal. operationId = operationId; proposal. reviewedBy = reviewerId; return { state, effect: { type: " APPLY_MODERATION ", operationId, proposalId, action: proposal. action, targetId: proposal. targetId, seatId: proposal. expectedSeatId, expectedSeatRevision: proposal. expectedSeatRevision } }; } export type ApplyResult = " accepted " | " rejected " | " unknown "; export function recordApplyResult ( current: RoomState, proposalId: string, operationId: string, result: ApplyResult ): RoomState { const state = structuredClone ( current ); const proposal = state. proposals [ proposalId ]; if (! proposal ) throw new Error ( " proposal_not_found " ); if ( proposal. status!== " applying " ) throw new Error ( " proposal_not_applying " ); if ( proposal. operationId!== operationId ) throw new Error ( " stale_operation " ); if ( result === " accepted " ) { // An accepted request is not yet proof that observed room state changed. proposal. status = " awaitingObservation "; } else if ( result === " rejected " ) { proposal. status = " failed "; proposal. failureReason = " adapter_rejected_action "; } else { proposal. status = " uncertain "; proposal. failureReason = " action_result_unknown "; } return state; } export function verifyObservedState ( current: RoomState, proposalId: string, observedSeat: Seat ): RoomState { const state = structuredClone ( current ); const proposal = state. proposals [ proposalId ]; if (! proposal ) throw new Error ( " proposal_not_found " ); if ( proposal. status!== " awaitingObservation " ) { throw new Error ( " proposal_not_awaiting_observation " ); } const conditionMet = proposal. action === " mute "? observedSeat. occupantId === proposal. targetId && observedSeat. muted: observedSeat. occupantId!== proposal. targetId; if ( conditionMet ) { proposal. status = " verified "; } state. seats [ observedSeat. id ] = observedSeat; return state; } Enter fullscreen mode Exit fullscreen mode There are three deliberate constraints in this model.

First, kind and role are separate. Labeling an AI participant as a moderator does not make it human or grant approval authority.

Second, a proposal refers to a specific seat revision. A recommendation about seat 2 should not be applied after seat 2 has been vacated and occupied by someone else.

Third, an accepted adapter request is not displayed as a completed action. Completion requires an observed seat state that matches the requested result.

A generic “moderation accuracy” score will not reveal authority or race-condition failures. Build cases from situations your room must survive.

import { describe, expect, it } from " vitest "; import { approveProposal, canPublishAudio, recordApplyResult, verifyObservedState, type RoomState } from "./moderation "; function room (): RoomState { return { phase: " open ", participants: { host: { id: " host ", kind: " human ", role: " host " }, speaker: { id: " speaker ", kind: " human ", role: " member " }, bot: { id: " bot ", kind: " ai ", role: " moderator " } }, seats: { s1: { id: " s1 ", occupantId: " speaker ", muted: false, revision: 7 }, s2: { id: " s2 ", occupantId: null, muted: false, revision: 3 } }, proposals: { p1: { id: " p1 ", sourceId: " bot ", targetId: " speaker ", action: " mute ", reason: " Possible repeated interruption ", evidenceRefs: [ " transcript-window-42 " ], expectedSeatId: " s1 ", expectedSeatRevision: 7, expiresAt: 20 _000, status: " pending " } } }; } describe ( " voice-room moderation protocol ", () => { it ( " does not let an AI approve its own proposal ", () => { expect (() => approveProposal ( room (), " p1 ", " bot ", " op-1 ", 10 _000 ) ). toThrow ( " human_review_required " ); }); it ( " does not let an off-seat AI participant publish audio ", () => { expect ( canPublishAudio ( room (), " bot " )). toBe ( false ); }); it ( " uses the same seat rule when the AI is allowed to speak ", () => { const state = room (); state. seats. s2 = { id: " s2 ", occupantId: " bot ", muted: false, revision: 4 }; expect ( canPublishAudio ( state, " bot " )). toBe ( true ); }); it ( " rejects approval when the seat changed after the proposal ", () => { const state = room (); state. seats. s1 = { id: " s1 ", occupantId: " speaker ", muted: false, revision: 8 }; expect (() => approveProposal ( state, " p1 ", " host ", " op-2 ", 10 _000 ) ). toThrow ( " stale_seat_revision " ); }); it ( " waits for observed seat state before claiming success ", () => { const approved = approveProposal ( room (), " p1 ", " host ", " op-3 ", 10 _000 ); expect ( approved. state. proposals. p1. status ). toBe ( " applying " ); const accepted = recordApplyResult ( approved. state, " p1 ", " op-3 ", " accepted " ); expect ( accepted. proposals. p1. status ). toBe ( " awaitingObservation " ); const verified = verifyObservedState ( accepted, " p1 ", { id: " s1 ", occupantId: " speaker ", muted: true, revision: 8 }); expect ( verified. proposals. p1. status ). toBe ( " verified " ); }); it ( " shows an ambiguous timeout instead of pretending success ", () => { const approved = approveProposal ( room (), " p1 ", " host ", " op-4 ", 10 _000 ); const uncertain = recordApplyResult ( approved. state, " p1 ", " op-4 ", " unknown " ); expect ( uncertain. proposals. p1. status ). toBe ( " uncertain " ); expect ( uncertain. proposals. p1. failureReason ). toBe ( " action_result_unknown " ); }); }); Enter fullscreen mode Exit fullscreen mode Run the benchmark:

The official implementation guide uses LiveListStore for room lifecycle and LiveSeatStore for mic-seat state. Keep those live concerns behind an adapter rather than importing SDK behavior into your policy functions.

Define the application boundary like this:

import type { ApplyModerationEffect, RoomPhase, Seat } from "./moderation "; export interface VoiceRoomAdapter { readRoomPhase (): Promise < RoomPhase >; readSeats (): Promise < Seat [] >; applyModeration ( effect: ApplyModerationEffect ): Promise < " accepted " | " rejected " | " unknown " >; } Enter fullscreen mode Exit fullscreen mode The concrete adapter in your application should:

Do not let a client-provided role or an AI response directly invoke the adapter. Approval should be validated on a trusted application service using fresh room and seat state.

seat snapshot ↓ application assigns seat revision ↓ AI or human submits recommendation ↓ moderation queue displays evidence and expiration ↓ human host/moderator approves ↓ server revalidates room, reviewer, occupant, and revision ↓ adapter requests the room action ↓ fresh seat snapshot verifies the resulting condition Enter fullscreen mode Exit fullscreen mode The UI should display the actual states rather than collapsing them into a spinner:

An AI participant can be useful for narrowing a busy review queue, grouping similar reports, or pointing a human toward a short evidence window. None of those capabilities proves that it understands humor, reclaimed language, speaker relationships, or the room’s social context.

approved warning text → AI participant receives a mic seat → seat permission is verified → speech synthesis → audio publication Enter fullscreen mode Exit fullscreen mode The model should not be able to produce a special response token that bypasses either path.

Start with a small case matrix before evaluating any model:

For each case, record three separate results:

This prevents a high classification score from hiding a dangerous control path.

The seat occupant or revision no longer matches. Reject the approval and label the proposal stale. Do not redirect the action to whoever occupies that seat next.

Do not solve this with a prompt alone. Define protected-target policy in deterministic server code, or require an elevated review path appropriate to your room model.

A timeout does not reveal whether the remote operation happened. Mark the result uncertain, refresh authoritative seat state, and use an operation identifier or your persistence layer’s idempotency mechanism before allowing a retry.

Reject new approvals after the room leaves open. Closing and moderation should not race to create unexplained final actions.

Stop its speech path and remove or mute its speaking capability according to your room policy. Cancelling generated text is insufficient if synthesized audio is already queued.

Keep human moderation usable. The queue should accept human reports and direct decisions without requiring the AI pipeline to recover first.

Run the unit benchmark, then perform a live-room verification pass with non-production accounts:

Adding AI does not reduce moderation engineering to choosing a model. The durable work is defining authority, identifying stale evidence, deciding which actions are reversible, and making uncertainty visible.

A model may help sort the queue. It cannot decide what your community promises its members or who is accountable when context is unclear. Encoding those decisions as executable cases gives both developers and moderators something more useful than a convincing demo: a system whose boundaries can be inspected and rerun.

Relationship disclosure: I have a content collaboration relationship with Tencent RTC. Official Tencent RTC documentation was used as the implementation reference for this article.