Agent joining very late after the session have been started

[AdarshRai3]

Bug Description

Agent was not able to join the session immediately but after 3 minutes due to which users are leaving the sessions. These are logs of the session.

DEFAULT 2026-01-09T12:48:05.575106Z E0000 00:00:1767962885.572991 1 process_state.cc:771] RAW: Raising signal 15 with default behavior
DEFAULT 2026-01-09T12:51:19.537705Z Building livekit-agent @ file:///app
DEFAULT 2026-01-09T12:51:23.519114Z Built livekit-agent @ file:///app
DEFAULT 2026-01-09T12:51:23.526247Z Uninstalled 1 package in 4ms
DEFAULT 2026-01-09T12:51:23.527607Z Installed 1 package in 1ms
DEFAULT 2026-01-09T12:51:40.534599Z starting worker
DEFAULT 2026-01-09T12:51:40.534626Z preloading plugins
DEFAULT 2026-01-09T12:51:40.564281Z HTTP server listening on :8081
DEFAULT 2026-01-09T12:51:51.347128Z initializing process
DEFAULT 2026-01-09T12:51:52.112317Z VAD prewarmed successfully
DEFAULT 2026-01-09T12:51:52.112499Z process initialized
DEFAULT 2026-01-09T12:51:52.118696Z initializing process
DEFAULT 2026-01-09T12:51:52.874635Z process initialized
DEFAULT 2026-01-09T12:51:52.875066Z VAD prewarmed successfully
DEFAULT 2026-01-09T12:51:52.996911Z registered worker
DEFAULT 2026-01-09T12:51:53.041785Z received job request
DEFAULT 2026-01-09T12:51:53.098409Z Agent startup initiated
DEFAULT 2026-01-09T12:51:53.103002Z initializing process
DEFAULT 2026-01-09T12:51:53.589006Z Loaded configuration for profile
DEFAULT 2026-01-09T12:51:54.285128Z process initialized
DEFAULT 2026-01-09T12:51:54.285309Z VAD prewarmed successfully

Expected Behavior

It should join the session immediately.

Reproduction Steps

It don't know the root cause and it is only in production , 1/10 interviews

Operating System

Windows 11

Models Used

Open AI

Package Versions

# -------- LiveKit (VERSION-ALIGNED) --------
    "livekit==1.0.23",
    "livekit-agents==1.3.10",
    "livekit-api==1.1.0",

    # -------- LiveKit Plugins --------
    "livekit-plugins-aws==1.3.10",
    "livekit-plugins-azure==1.3.10",
    "livekit-plugins-cartesia==1.3.10",
    "livekit-plugins-deepgram==1.3.10",
    "livekit-plugins-google==1.3.10",
    "livekit-plugins-openai==1.3.10",
    "livekit-plugins-silero==1.3.10",
    "livekit-plugins-noise-cancellation==0.2.5",

Session/Room/Call IDs

No response

Proposed Solution

Additional Context

No response

Screenshots and Recordings