Degraded Connectivity in US Central

What to do

• Check whether you use LiveKit Cloud US Central - SIP in us. Check your stack

  Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

• Confirm your retries, queues and scheduled jobs caught up after the 1h 12m it lasted.

  Anything that failed between 2026-06-02 14:05 UTC and 2026-06-02 15:18 UTC may need a replay.

• Subscribe to LiveKit Cloud's feed to hear about the next one first. RSS feed

  https://down.guide/feed?provider=livekit works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from LiveKit Cloud 5

1. investigatingJun 2, 2026, 2:05 PM UTC

   We're investigating elevated SIP call connection failures in our US Central region beginning at ~12:06 UTC. We are working to mitigate this issue.

   provider

2. monitoringJun 2, 2026, 2:17 PM UTC

   We have routed the traffic away from the US Central region at 14:00 UTC and are seeing the connection failures returning to normal levels. We are continuing to monitor the issue.

   provider

3. monitoringJun 2, 2026, 2:58 PM UTC

   We've identified the root cause as a single problematic media node in US Central, which has been suspended at 14:38 UTC. We are continuing to monitor before marking this resolved.

   provider

4. resolvedJun 2, 2026, 3:18 PM UTC

   This incident has been resolved.

   provider

5. postmortemJun 4, 2026, 2:01 PM UTC

   **Summary** On June 2, 2026, a small subset of users connected to LiveKit Cloud US Central region experienced elevated connection failures between 12:06 and 14:38 UTC. A single node in the region entered a degraded state in which it continued to accept new connections but could not reliably complete the real-time media connection those calls depend on. As a result, a portion of participant connections, including inbound and outbound SIP calls routed through Chicago failed to connect or dropped shortly after starting. We mitigated this by routing SIP traffic away from the affected server, suspending and removing the faulty node, and restoring normal service to the region. **Timeline \(UTC\)** * 12:06 — Start of measurable customer impact * 14:04 — Chicago SIP traffic drained as mitigation * 14:38 — Problematic server suspended; customer impact ends * 15:07 — Chicago SIP service fully restored **Root cause** The incident was caused by a "gray failure" of a single node in the Chicago region, a partial failure in which a server appears healthy to automated systems but is not actually functioning correctly. The server continued to be assigned new calls and reported itself as available, but could not reliably bring the underlying real-time media connections to a fully active state. **Resolution** We first drained SIP traffic out of the Chicago region to route the traffic to other regions, and once the failing node was identified, we suspended and removed the faulty node. After confirming error rates had returned to normal, we restored full SIP service to the region. **Corrective actions and prevention** We are actively addressing the two gaps that allowed a single failing node to impact the region: * **Error aversion in load balancing:** configuring our load balancing servers to automatically detect and steer traffic away from nodes exhibiting this class of failure. * **Monitoring:** closing the alerting gap so that "gray failure" servers which appear healthy but cannot complete connections are detected and alerted promptly.

   provider

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.