fix(telnyx): warm up the turn detector in the voice_agent_call example

[a692570]

Fix: warm up the turn detector in the all-Telnyx voice agent example

Problem

plugins/telnyx/examples/voice_agent_call.py builds the agent directly and calls agent.join(). It never goes through AgentLauncher, so _warmup_agent() never runs and no component is warmed up.

This example is the only one in the repo that needs a local turn detector. inbound_call.py uses llm=gemini.Realtime(), which does turn detection server-side, so it is unaffected. voice_agent_call.py uses telnyx.STT(sample_rate=8000), and the comment above it notes why:

# Telnyx's transcription endpoint sends no VAD signals, so turns are
# detected locally rather than by the STT.
turn_detection=smart_turn.TurnDetection(),

SmartTurnDetection is a Warmable. Until warmup() runs, process_audio raises on every chunk (smart_turn_detection.py:222):

if self._vad_session is None:
    raise ValueError("VAD model is not initialized, call warmup() first")

The turn detector is the one component standing in for the VAD signals Telnyx does not send, and it never starts. Turn boundaries are therefore never detected.

Impact

The caller can talk, and the agent transcribes turn boundaries as speech/silence, but it never learns the caller stopped speaking. The agent speaks its opening greeting and then does not reply again no matter what is said.

Reproduction

1. Start a tunnel to port 8000 and route a Telnyx number to the example.
2. Run it (inbound, with Telnyx STT/LLM/TTS):

   NGROK_URL=<host> uv run plugins/telnyx/examples/voice_agent_call.py --setup-telnyx --phone-number +1...
   

3. Call the number. The agent greets the caller and then stops responding.

Before

Turn detector raises on every audio chunk. Counted from a single real inbound call:

$ grep -c "VAD model is not initialized" agent.log
1257

The agent produced its opening line and one Finalizing LLM turn, then stopped. Zero turn-boundary events for the caller:

$ grep -c "finished speaking\|barged-in" agent.log
0

Reporter's description of the call: the agent spoke its greeting and then stayed silent no matter what was said.

After

Same example, same number, same pipeline, one line of warmup added:

$ grep -c "VAD model is not initialized" agent.log
0

$ grep -c "finished speaking\|barged-in" agent.log
6

Turn boundaries are detected now:

👉 Participant phone-<id> barged-in, interrupting the agent
👉 Participant phone-<id> finished speaking

The agent opens with its greeting, the caller's speech is segmented, and barge-in interrupts the agent as expected.

Change

from vision_agents.core.warmup import WarmupCache

# in media_stream(), before agent.join()
await agent.turn_detection.warmup(WarmupCache())

This mirrors what AgentLauncher._warmup_agent() does for the turn detector (agent_launcher.py:449).

Checks

Run against current main (ba2fbf1d):

• ruff check and ruff format --check pass
• pytest plugins/telnyx/tests/ -m "not integration": 67 passed
• Live inbound call with the Telnyx STT/LLM/TTS pipeline, before and after as above

Note, not in scope for this PR

A follow-up observation on transcripts, since the turn detector was masking it before: in one verification call after this fix, the turn boundaries fired but no transcript text arrived, so the agent did not learn the caller's words. A later call with identical configuration transcribed correctly and the agent responded, so this looks intermittent rather than systematic. I could not reproduce the silent call and am not filing it as a bug without a repro. For reference, interim_results defaults to False (stt.py:63) and is honored per engine rather than per endpoint (stt.py:14), but the default Telnyx engine does emit finals, which is what the later call showed.

[coderabbitai]

📝 Walkthrough

Walkthrough

The Telnyx voice agent imports WarmupCache and creates one module-level instance. Before joining the Stream call, the media handler warms the agent's turn detector with this shared cache. Models therefore load once across inbound calls instead of once per call.

Priority: ⬇️ Low

Merge Risk: 🔵 Low · up to 2c178

The example remains functionally usable, but the new shared cache should include its required type annotation before merging.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: ef8f4adb-f08c-4951-903f-e140d12ef6be

📥 Commits

Reviewing files that changed from the base of the PR and between 839bd43 and 2c17804.

📒 Files selected for processing (1)

• plugins/telnyx/examples/voice_agent_call.py

Included review availability: Your plan provides up to 8 included reviews per hour; 6 remain after this review.

[coderabbitai]

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Add the required type annotation.

The Python guidelines require type annotations everywhere. Change this declaration to warmup_cache: WarmupCache = WarmupCache().

Source: Coding guidelines

[a692570]

Follow-up observation from live-testing this fix. The PR fixes the symptom at the example level, but the reproduction exposed something worth a maintainer decision.

Before the fix, one call logged the same error 1,257 times while the call continued with broken turn-taking:

ERROR ...smart_turn_detection: Error processing audio: VAD model is not initialized, call warmup() first

The background loop catches Exception and logs it (smart_turn_detection.py:198-199), so a turn detector that raises on every chunk degrades the call silently. For anyone not tailing logs, a broken turn detector is indistinguishable from a working one.

Two questions this raises, both outside this PR's scope:

1. Should a component whose warmup failed abort the call (fail fast), or should the failure surface as a one-time prominent error event instead of a repeating log line? The current behavior let a fully broken turn detector run an entire call.
2. The loop uses logger.error(f"...") with no traceback, which made the original cause harder to trace. Repo convention (CLAUDE.md) prefers logger.exception for errors with tracebacks. Minor, but it compounds issue 1 when diagnosing.

[DaemonLoki]

Nice find, and appreciate the before/after log counts!

And yeah, by calling agent.join() yourself, it never goes through the warmup agent phase. So the matching to what the launcher does was an important addition

One thought: warmup runs after the caller has already been answered, so on a cold start the Silero VAD, Smart Turn ONNX, and Whisper extractor downloads happen while someone is on the line. You COULD warm a throwaway smart_turn.TurnDetection() against the same cache at startup (or moving the warmup into prepare_call), which would hide that. But since this is just an example, it should also be fine as-is.

[DaemonLoki]

1. Should a component whose warmup failed abort the call (fail fast), or should the failure surface as a one-time prominent error event instead of a repeating log line? The current behavior let a fully broken turn detector run an entire call.

Agent._start_components() already aborts the join when a component's start() raises. The gap is that SmartTurnDetection.start() never checks whether warmup has happened. I'll see that I can condense this into one hard failure at join using the approach we already have.

2. The loop uses logger.error(f"...") with no traceback, which made the original cause harder to trace. Repo convention (CLAUDE.md) prefers logger.exception for errors with tracebacks. Minor, but it compounds issue 1 when diagnosing.

Agreed, logger.exception is the way to go.

Both points are valid, and I'll address them in a separate PR.