System Errors in EU-SOUTH-1 Region

What to do

• Check whether you use Wasabi EU-South-1 (Milan) in eu, eu-south-1. Check your stack

  Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

• Confirm your retries, queues and scheduled jobs caught up after the 1h 33m it lasted.

  Anything that failed between 2025-01-10 16:30 UTC and 2025-01-10 18:03 UTC may need a replay.

• Subscribe to Wasabi's feed to hear about the next one first. RSS feed

  https://down.guide/feed?provider=wasabi works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Wasabi 4

1. investigatingJan 10, 2025, 4:30 PM UTC

   We are currently investigating issues in our EU-SOUTH-1 (Milan) region.

   provider

2. monitoringJan 10, 2025, 4:51 PM UTC

   We have identified and corrected the issue, and our EU-SOUTH-1 (Milan) region is now back to a fully operational state. We will continue to monitor the region.

   provider

3. resolvedJan 10, 2025, 6:03 PM UTC

   This incident has been resolved.

   provider

4. postmortemJan 15, 2025, 8:01 PM UTC

   On 10 January 2024 from 15:57 UTC to 16:45 UTC, we experienced an issue where clients received 4XX level errors while accessing the S3 buckets in the eu-south-1 \(Milan\) region.   During an internal configuration change, a software bug caused the invalidation of TLS certificates used by internal servers to establish mutual TLS handshakes with load balancers. Consequently, the load balancers could not successfully connect to the internal servers, leading to 4XX errors returned to client requests. Wasabi’s Engineering and Operations teams identified the issue and manually reconfigured the TLS certificates to restore functionality. A fix was deployed to the eu-south-1 region to address the underlying bug and to prevent this problem from happening again.

   provider

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.