Error from deepgram STT causes agent to freeze

[martin-purplefish]

We saw an error from the recognize task while the LLM was streaming, which in turn caused our agent to freeze/get stuck. When this happens, it seems that a coroutine never yielded as our background timers never went off after this.

{"message": "Error in _recognize_task\nTraceback (most recent call last):\n File "/app/.venv/lib/python3.12/site-packages/livekit/agents/utils/log.py", line 16, in async_fn_logs\n return await fn(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/app/.venv/lib/python3.12/site-packages/livekit/agents/pipeline/human_input.py", line 150, in _recognize_task\n await asyncio.gather(*tasks)\n File "/app/.venv/lib/python3.12/site-packages/livekit/agents/pipeline/human_input.py", line 136, in _stt_stream_co\n async for ev in stt_stream:\n File "/app/.venv/lib/python3.12/site-packages/livekit/agents/stt/stt.py", line 315, in anext\n raise exc from None\n File "/app/.venv/lib/python3.12/site-packages/livekit/agents/stt/stt.py", line 215, in _main_task\n return await self._run()\n ^^^^^^^^^^^^^^^^^\n File "/app/.venv/lib/python3.12/site-packages/livekit/plugins/deepgram/stt.py", line 489, in _run\n ws = await self._connect_ws()\n ^^^^^^^^^^^^^^^^^^^^^^^^\n File "/app/.venv/lib/python3.12/site-packages/livekit/plugins/deepgram/stt.py", line 533, in _connect_ws\n ws = await asyncio.wait_for(\n ^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/python3.12/asyncio/tasks.py", line 520, in wait_for\n return await fut\n ^^^^^^^^^\n File "/app/.venv/lib/python3.12/site-packages/aiohttp/client.py", line 1019, in _ws_connect\n raise WSServerHandshakeError(\naiohttp.client_exceptions.WSServerHandshakeError: 402, message='Invalid response status', url='wss://api.deepgram.com/v1/listen?model=nova-2-general&punctuate=true&smart_format=true&no_delay=true&interim_results=true&encoding=linear16&vad_events=true&sample_rate=16000&channels=1&endpointing=25&profanity_filter=false&language=en-US'", "level": "ERROR", "name": "livekit.agents.pipeline", "pid": 87, "job_id": "AJ_4FqHxRhbBWUc", "timestamp": "2024-12-18T14:08:29.478321+00:00"}

[davidzhao]

this has been fixed in #1291.