High response times on CMA. Admin interface not loading. CDA partially impacted.

What to do

• Check whether you use DatoCMS in eu. Check your stack

  Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

• Confirm your retries, queues and scheduled jobs caught up after the 0m it lasted.

  Anything that failed between 2024-10-23 14:22 UTC and 2024-10-23 14:22 UTC may need a replay.

• Subscribe to DatoCMS's feed to hear about the next one first. RSS feed

  https://down.guide/feed?provider=datocms works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from DatoCMS 1

1. resolvedOct 23, 2024, 2:22 PM UTC

   Resolved — Our upstream partner published their post-mortem (Root Cause Analysis (RCA)) about this downtime. You can read the full post here: https://status.heroku.com/incidents/2684 Summary: On October 8, 2024 from 16:27 Coordinated Universal Time (UTC) to 22:10 UTC, and again on October 9, 2024 from 13:06 UTC to 16:36 UTC, a Distributed Denial of Service (DDoS) caused routing failures in a single partition of Heroku’s Common Runtime infrastructure in the EU Region. This resulted in increased error rates and latencies for some clients connecting to some customer applications hosted in that partition. The Salesforce Technology team worked with our upstream infrastructure provider to mitigate the immediate impact of this event, and to put additional network-level protections in place to improve resilience. Resolved — Heroku resolved the issue: https://status.heroku.com/incidents/2685 We are still waiting for their post-mortem message about these two incidents. Identified — The upstream provider is working on a fix Investigating — The upstream provider is currently investigating the issue. This is their new status page issue: https://status.heroku.com/incidents/2685 Investigating — It looks like Heroku is having the same problem again. We have already contacted them. Resolved — We are waiting for Heroku's post mortem in order to collect more information about the downtime Resolved — The upstream issue has been resolved and services are fully restored. Their last update: Starting at 4:27 PM UTC on October 8th , 2024, customers experienced increased error rates and latencies on customer applications hosted on common runtime in EU Region. Heroku engineers investigated and mitigated the impact at 9:52 PM UTC on October 8th , 2024. All application have fully recovered by 10:10 PM UTC. Monitoring — Services (CMA, UI, CDA) appear to be coming back up and are accessible at normal performance again. However, we still have not yet received an official confirmation of resolution from the upstream provider. We will continue to monitor until an official resolution is announced. Identified — The upstream provider is still investigating the issue. We will continue to post updates here, or you can see their status directly at https://status.heroku.com/incidents/2684 Informational note: Already-cached Content Delivery API (GraphQL) requests do not seem to be affected. Uncached CDA queries may or may not be impacted, depending on the specific query and region. We are still awaiting a resolution and will continue to monitor the situation. Identified — The upstream provider identified an issue on their infrastructure: https://status.heroku.com/incidents/2684 Investigating — We are still investigating the issue, and have contacted an upstream provider. We will update again once we have more information. Investigating — We are currently investigating the issue

   provider

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.