# 2026-September-28 Service Incident

- Provider: [Sauce Labs](https://down.guide/saucelabs)
- Level: confirmed
- Impact: Partial outage
- lasted 1h 15m
- started 2026-09-28 16:04 UTC
- US-West
- us, us-west-1
- id: DWN-8a2d32bc38
- Official page: https://stspg.io/551m8dbsyvqd

## Root cause
> The root cause of the elevated error rates for Real Device tests in the US-West-1 data center has been identified.

Category: Data center or power. [Postmortem](https://stspg.io/551m8dbsyvqd)

## Updates from Sauce Labs
### 2026-09-28 16:04 UTC: investigating
We are currently seeing elevated error rates for automated real device tests in US-West-1 data center. We are currently investigating.

### 2026-09-28 16:56 UTC: monitoring
The root cause of the elevated error rates for Real Device tests in the US-West-1 data center has been identified. Remediation has been applied, and we are actively monitoring the situation.

### 2026-09-28 17:19 UTC: resolved
After taking remedial action, Real Device test error rates have returned to normal in the US-West-1 data center. All services are fully operational.

### 2026-10-02 19:15 UTC: postmortem
### \*\*Dates:\*\*
Monday, September 28th 2026, 14:19 UTC - 16:52 UTC.
### \*\*What happened:\*\*
Real device tests in US West hung and failed to output test artifacts due to slow uploads.
### \*\*Why it happened:\*\*
A degraded interconnect link suffered high packet loss from optical errors, but BGP stayed up, preventing automatic failover to healthy paths.
### \*\*How we fixed it:\*\*
Manually disabled the degraded port on the edge router, forcing traffic onto healthy alternate interconnects.
### \*\*What we are doing to prevent it from happening again:\*\*
Automating link draining based on CRC error thresholds, improving protocols for isolating flapping vendor links, and hardening upload resilience.

---
down.guide reads providers' official status pages every minute and adds its own evidence, always labeled. Free, no account. Agents: MCP at https://down.guide/mcp, REST at https://down.guide/api/v1, feeds at https://down.guide/feed. Built by Sazabi (https://sazabi.com/?utm_source=down-guide&utm_medium=markdown).