- Level
- confirmed
- Worst impact
- Degraded
- Components
- US-East-1 (N. Virginia), US-East-2 (N. Virginia)
- Regions
- us, us-east-1, us-east-2
- Down Index
- 34of 100
What to do
Check whether you use Wasabi US-East-1 (N. Virginia), US-East-2 (N. Virginia) in us, us-east-1, us-east-2. Check your stack
Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.Confirm your retries, queues and scheduled jobs caught up after the 8h 14m it lasted.
Anything that failed between 2024-12-07 16:14 UTC and 2024-12-08 00:28 UTC may need a replay.Subscribe to Wasabi's feed to hear about the next one first. RSS feed
https://down.guide/feed?provider=wasabi works in Slack (/feed subscribe), Discord feed bots and any RSS reader.
Updates from Wasabi 9
- investigating
We are currently investigating an increase in 500 level HTTP responses on customer traffic to the us-east-1 and us-east-2 regions.
provider
- investigating
We are continuing to investigate the system errors in the us-east-1 and us-east-2 regions. We will update this page as we have more information.
provider
- identified
We have identified a power issue at the data center which is in the process of being restored. We will update this page as we have more information.
provider
- identified
We are continuing the process of restoring power and bringing systems back up. We will update this page as we have more information.
provider
- identified
Power has been fully restored to the us-east-2 region. We are continuing the process of restoration to the us-east-1 region and will update this page as we have more information.
provider
- identified
Power has been fully restored to the us-east-1 region and we are continuing to work on bringing all systems back online. We will update this page as we have more information.
provider
- monitoring
All systems are now back online and fully operational. We are continuing to monitor the regions and will update this page as we have more information.
provider
- resolved
Services in both regions have been restored. Please reach out to support@wasabi.com if you see any issues related to this incident.
provider
- postmortem
On 7 December 2024 from 15:22 UTC to 21:30 UTC, Wasabi experienced a loss of power event in our US-EAST-1 and US-EAST-2 data centers. At 15:22, our Operations Team noticed that infrastructure within the US-EAST-1 and US-EAST-2 regions failed to respond to standard smoke tests and monitoring tools, and reviewing the activity for the regions indicated a full loss of power to all server racks and infrastructure within the building. At 16:00 UTC, Wasabi received confirmation from Iron Mountain that power loss for the entire building has occurred. By 16:15 UTC, Iron Mountain Operations begins to restore power to an incremental number of racks for Wasabi’s infrastructure, allowing our Operations Team to run systematic health checks across all server nodes, and by 21:30 UTC we have confirmation that all systems are running optimally and both US-EAST-1 and US-EAST-2 regions were fully operational.
provider
Reports count in aggregate; nothing about you is shown.
Corrections
None so far.
Something wrong on this page? Report an error or send us a note. Every fix is listed here.
Questions about DWN-a1f3ec9b52
Is Wasabi still down?
No, this incident was resolved 2024-12-08 00:28 UTC. For Wasabi's live state, see https://down.guide/wasabi.
How long did it last?
8h 14m, from 2024-12-07 16:14 UTC to 2024-12-08 00:28 UTC, by the times Wasabi posted.
What was affected?
Wasabi named US-East-1 (N. Virginia), US-East-2 (N. Virginia) in us, us-east-1, us-east-2.
What caused it?
Wasabi hasn't published a root cause. If it does, it shows here.
Where does this come from?
Wasabi's official status page, https://status.wasabi.com, quoted verbatim. down.guide's own probes, reports and social counts are labeled as ours.