2026-July-16 Resolved Service Incident - Error Reporting (Backtrace)

What to do

• Check whether your product depends on Sauce Labs. Check your stack

  Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

• Confirm your retries, queues and scheduled jobs caught up after the 0m it lasted.

  Anything that failed between 2026-07-16 20:33 UTC and 2026-07-16 13:00 UTC may need a replay.

• Subscribe to Sauce Labs's feed to hear about the next one first. RSS feed

  https://down.guide/feed?provider=saucelabs works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Sauce Labs 2

1. resolvedJul 16, 2026, 1:00 PM UTC

   Between July 16th 13:03 UTC and July 16th 17:33 UTC, we experienced a service disruption where symbol archives utilizing multi-part upload protocols failed to process. Incident has been resolved and all services are operational.

   provider

2. postmortemAug 14, 2026, 4:48 PM UTC

   ### **Dates:** Thursday July 16th 2026, 13:03 – 17:33 UTC ### **What happened:** Symbol archives uploaded in multiple parts failed to process. All regions were affected. ### **Why it happened:** Our symbol processing service sent an upload-verification field that our cloud storage provider's API does not accept for multi-part uploads, so those uploads were rejected. A fix for this had already been developed, but it had not yet been included in a released build and the service was not configured to use it. As in the first incident, rejected uploads were retried and accumulated on local disk. ### **How we fixed it:** We deployed a build containing the fix and corrected the service configuration on the affected workers. A large backlog of uploads then processed, which briefly re-filled the disks before draining completely. ### **What we are doing to prevent it from happening again:** We are releasing the fix formally through our build pipeline and persisting the corrected configuration in our configuration management, so it can't be lost. The disk-utilization alerting added after the first incident also covers the disk-exhaustion pattern common to both.

   provider

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.