Skip to main content
Hasura Cloud

A small set of free tier tenants suffered downtime

Degraded Lasted 0m · started · Hasura Cloud's page

Level
confirmed
Worst impact
Degraded
Regions
us
Down Index
12of 100

What to do

  • Check whether you use Hasura Cloud in us. Check your stack

    Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.
  • Confirm your retries, queues and scheduled jobs caught up after the 0m it lasted.

    Anything that failed between 2024-11-26 20:25 UTC and 2021-02-01 06:30 UTC may need a replay.
  • Subscribe to Hasura Cloud's feed to hear about the next one first. RSS feed

    https://down.guide/feed?provider=hasura works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Hasura Cloud 1

  1. resolved

    Between 21.30PM GMT(Sun) to 6AM GMT(Mon) Hasura Cloud experienced an issue that impacted approximately 5% of free-tier projects and caused those projects to suffer downtime. One of our worker nodes hosting a portion of free-tier tenants failed an internal health check and was marked inactive. Unfortunately, the process for auto-healing that worker did not happen as expected. Additionally, the monitoring that was set up to alert on health check failures missed this worker node. This meant that the projects hosted on that worker experienced over 8 hours of downtime. We acknowledge that this is an unacceptable length of downtime for customers depending on Hasura Cloud. Once we noticed this issue we were able to immediately reallocate the affected projects to another active worker node and restore functionality to all projects by 6.15AM GMT. As part of our remediation of this event, we are enhancing our monitoring setup, defining more rigorous playbooks, and making changes to our internal health check system to increase robustness. We've improved our proactive monitoring to help alert us to any issues affecting projects so that downtime is prevented. We have also validated that our reallocation process works as expected when worker nodes are marked as inactive.

    provider

Reports count in aggregate; nothing about you is shown.

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.

Questions about DWN-5627bbd05c

Is Hasura Cloud still down?

No, this incident was resolved 2021-02-01 06:30 UTC. For Hasura Cloud's live state, see https://down.guide/hasura.

How long did it last?

0m, from 2024-11-26 20:25 UTC to 2021-02-01 06:30 UTC, by the times Hasura Cloud posted.

What was affected?

Hasura Cloud named in us.

What caused it?

Hasura Cloud hasn't published a root cause. If it does, it shows here.

Where does this come from?

Hasura Cloud's official status page, https://hasura-status.com, quoted verbatim. down.guide's own probes, reports and social counts are labeled as ours.