Increased latency on read and analytics endpoints

What to do

• Check whether you use Orb Reads, Analytics. Check your stack

  Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

• Confirm your retries, queues and scheduled jobs caught up after the 6h 3m it lasted.

  Anything that failed between 2025-05-16 08:51 UTC and 2025-05-16 14:54 UTC may need a replay.

• Subscribe to Orb's feed to hear about the next one first. RSS feed

  https://down.guide/feed?provider=orb works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Orb 3

1. investigatingMay 16, 2025, 8:51 AM UTC

   We are experiencing periods of elevated API response times specifically on read and analytics requests. Our oncall team is currently investigating. Event ingestion, invoice finalization, and write requests are unaffected.

   provider

2. monitoringMay 16, 2025, 9:42 AM UTC

   We've identified a failure mode tied to a degenerate workload that degrades query performance for one of our usage datastores. A temporary resolution has been put in place to ensure system stability, our oncall team will continue to monitor the error rates for the API and adjust workloads as needed to ensure API availability.

   provider

3. resolvedMay 16, 2025, 2:54 PM UTC

   After ongoing monitoring, this issue is now fully resolved. Endpoints have been recovered for the last several hours; the last period of errors occurred before 05/16/25 08:30 UTC.

   provider

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.