Skip to main content
Trigger.dev

us-east-1 region dequeue degradation

Partial outage Lasted 4d 19h · started · Trigger.dev's page

Level
confirmed
Worst impact
Partial outage
Regions
eu, eu-central-1, us, us-east-1
Down Index
49of 100

What to do

  • Check whether you use Trigger.dev in eu, eu-central-1, us, us-east-1. Check your stack

    Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.
  • Confirm your retries, queues and scheduled jobs caught up after the 4d 19h it lasted.

    Anything that failed between 2026-06-22 19:05 UTC and 2026-06-27 14:54 UTC may need a replay.
  • Subscribe to Trigger.dev's feed to hear about the next one first. RSS feed

    https://down.guide/feed?provider=trigger-dev works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Trigger.dev 19

  1. update

    This predominantly affects larger machine sizes. AWS is having capacity issues.

    provider

  2. update

    We managed to secure additional capacity, but large machines still disproportionately affected.

    provider

  3. update

    Runs have stopped executing in us-east-1, we're investigating. The eu-central-1 region is still executing runs, consider switching to that region as a temporary workaround.

    provider

  4. update

    We are continuing to investigate the issue and with us-east-1 but have no hard timeline on when we might be able to recover. The root cause of the issue is still not clear to us, and the nature of the issue is making it very difficult to ascertain the underlying cause. We're focusing on mitigation. eu-central-1 continues to operate under normal conditions.

    provider

  5. update

    Just a note about switching your default region to the eu-central-1 cluster, and the behavior of how the switching of regions effects existing queued runs: already enqueued runs keep their original region (which is set at trigger time). Switching your default region effects new runs only, immediately. It is possible to replay an individual runs on a different region, so one temporary workaround: - Cancel existing queued runs in the us-east-1 region (using the Region filter) - Go through each run cancelled individually and click through to the run detail page, and find the "Replay run" button in the top right - Select eu-central-1 from the drop down We're working on being able to streamline this process, and again we apologize for this outage.

    provider

  6. update

    For users who have switched from the us-east-1 region to the eu-central-1 region, and are seeing slow to minimal run executions for newly created runs, one way to unlock these new run executions is to cancel any of your us-east-1 queued runs, by using the Region filter in the runs list and selecting the Bulk Action -> Cancel option. This us-east-1 outage has discovered a bug in our concurrency accounting that can cause queued runs in degraded regions to be counted against your concurrency limit. We're continuing to make progress on restoring the us-east-1 region, and will update again shortly.

    provider

  7. update

    The control plane has stabilized and core services are coming back online. We’re carefully restoring capacity in a controlled way. EU Central 1 remains fully operational, so you can continue running workloads there in the meantime. No action needed on your end. We’re close to full resolution and will share our next update shortly.

    provider

  8. update

    We are now processing a high number of us-east-1 runs, but there is a large backlog to get through. The backlog is reducing and we're looking to see if we can do this faster in a controlled way. EU Central 1 remains fully operational, so you can continue running workloads there in the meantime.

    provider

  9. update

    We've scaled up our services to handle the increased workload, and throughput is improving. There's still a significant backlog of runs to work through, so you may continue to see delays while we catch up. We're monitoring closely and processing through the queue as fast as we safely can. EU Central 1 remains fully operational, so you can continue running workloads there in the meantime.

    provider

  10. update

    As our us-east-1 region is recovering the eu-central-1 region is now experiencing the same degradation that occurred last night to us-east-1. We're continuing to investigate and attempting to bring the dequeues back online for eu-central-1, while also working through the us-east-1 backlog, which is continuing to clear but won't be fully clear for 1+ hr at the current rate as we battle to get server capacity from our upstream provider.

    provider

  11. update

    us-east-1 is almost completely recovered although still with slower dequeue times than normal. We're continuing to investigate how to bring eu-central-1 back online.

    provider

  12. update

    Our advice remains the same: switch your default region to us-east-1 if you can. More details below. **us-east-1** Operating normally with global queue times better than normal thresholds. If you have excessive runs queued in us-east-1: 1. Check your concurrency limits, increase as needed including at the account level from the Concurrency page (let us know via the in-app Help and Feedback and we will credit your account) 2. If you were on eu-central-1 they could be blocking your us-east-1 runs. Go to the Runs page and filter for "Queued" status and "eu-central-1" region. Then use the Bulk action on the right to cancel those runs. 3. When eu-central-1 is back you can replay these runs there. **eu-central-1** The cluster is being spun back up with more resources and the same settings we used to fix us-east-1. Currently no runs are executing here. The same thundering herd issue that happened in us-east-1 caused this issue as everyone moved their traffic over. This cluster was smaller so it took less to cause this problem. -- After this incident is over we will do a full post mortem with what happened along with a long list of things we need to do better. I've been noting them down and we've recorded this entire process. We will also be offering credits to all impacted customers. I know this doesn't cover the cost to your businesses. This is far from the bar we set for reliability. Sorry everyone, Matt

    provider

  13. update

    Our EU Central 1 is back online and processing the backlog of runs in that region as fast as we can. We will update again when we have an expected clearance time to get back to normal dequeue operations. We're also working on adding the ability to the Bulk Action feature to be able to replay runs in a different region from the replayed runs. US East 1 continues to operate normally. Full postmortem to follow

    provider

  14. update

    The EU Central 1 region continues to drain and it looks like will be back to normal operation within the next hour at current drain rates. We have deployed the Bulk Action -> Replay -> Region override feature now which allows replaying runs into a different region. US East 1 continues to operate normally.

    provider

  15. update

    US East 1 continues to operate normally. EU Central 1 scheduled jobs are also back to normal operation. The remaining impact is EU on-demand workers still catching up on the backlog, so you may see delayed dequeuing of queued runs for a little while longer. One thing to be aware of: if you have a large number of queued runs in EU Central 1, they can consume your concurrency and block runs in other regions (including US East 1) from executing. If your US East 1 runs aren't progressing despite that region being healthy, this is likely the cause. You can unblock yourself by cancelling the queued EU runs and replaying them in your normal region using the Bulk Action → Replay → Region override feature we just deployed.

    provider

  16. update

    US East 1 dequeues are currently degraded, as we work through the large backlog and deal with capacity issues with our upstream provider.

    provider

  17. update

    US East 1 dequeue performance degradation has cleared but we're continuing to monitor the situation

    provider

  18. update

    US East 1 and EU Central 1 dequeue performance issues have been resolved. We're continuing to monitor the situation and a full post-mortem will follow.

    provider

  19. resolved

    The full postmortem has been posted here: https://trigger.dev/blog/incident-report-jun-22-2026

    provider

Reports count in aggregate; nothing about you is shown.

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.

Questions about DWN-db1a0f99d4

Is Trigger.dev still down?

No, this incident was resolved 2026-06-27 14:54 UTC. For Trigger.dev's live state, see https://down.guide/trigger-dev.

How long did it last?

4d 19h, from 2026-06-22 19:05 UTC to 2026-06-27 14:54 UTC, by the times Trigger.dev posted.

What was affected?

Trigger.dev named in eu, eu-central-1, us, us-east-1.

What caused it?

Trigger.dev hasn't published a root cause. If it does, it shows here.

Where does this come from?

Trigger.dev's official status page, https://status.trigger.dev, quoted verbatim. down.guide's own probes, reports and social counts are labeled as ours.