Skip to content

fix timeout and audio sync#154

Draft
raghavm243512 wants to merge 1 commit into
mainfrom
pr/elevenlabs_fix
Draft

fix timeout and audio sync#154
raghavm243512 wants to merge 1 commit into
mainfrom
pr/elevenlabs_fix

Conversation

@raghavm243512

@raghavm243512 raghavm243512 commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

Series of small/medium changes to fix timeout and keep audio in sync:

ElevenLabs' Conversational AI changed its end-of-utterance audio delivery: it
no longer stops the user-agent stream in a way the user simulator's partial-chunk
heuristic could detect, so the user sim never emitted end-of-turn / trailing
silence and S2S assistants (ElevenLabs, OpenAI realtime) deadlocked waiting on
each other → inactivity timeout.

  • user sim: derive end-of-turn from ElevenLabs' agent_response event (arm) +
    audio-buffer drain (fire), independent of stream shape; emit trailing silence
    so the assistant VAD can close the turn
  • elevenlabs bridge: forward µ-law 8kHz in 250ms chunks (was batching 1s chunks)
  • elevenlabs server: wall-clock-anchored recording (both tracks, assistant at
    playback time) → correct mixed audio; model_response latency keyed off the
    user_transcript event
  • bump elevenlabs SDK to 2.53.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant