Skip to main content
Apify

Major systems outage

Major outage Lasted 1h 26m · started · Apify's page

Level
confirmed
Worst impact
Major outage
Components
Web (apify.com), Datacenter, Dataset, Console (console.apify.com), Request queue, Residential, API (api.apify.com), Key-value store, SERP, Actors, Scheduler, Webhooks
Down Index
60of 100

What to do

  • Check whether you use Apify Web (apify.com), Datacenter, Dataset, Console (console.apify.com) and 8 more. Check your stack

    Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.
  • Confirm your retries, queues and scheduled jobs caught up after the 1h 26m it lasted.

    Anything that failed between 2025-02-22 22:12 UTC and 2025-02-22 23:38 UTC may need a replay.
  • Subscribe to Apify's feed to hear about the next one first. RSS feed

    https://down.guide/feed?provider=apify works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Apify 7

  1. investigating

    We are investigating the issue.

    provider

  2. investigating

    We are continuing to investigate this issue.

    provider

  3. identified

    We've identified the cause of the issue and are taking steps to resolve it.

    provider

  4. monitoring

    A fix has been implemented and we're monitoring the results. It might take some time to allocate all jobs, until all services have fully recovered. We're sorry for the inconvenience. We will implement new measures based on this critical incident to prevent similar incidents in the future.

    provider

  5. monitoring

    We are continuing to monitor for any further issues.

    provider

  6. resolved

    This incident has been resolved.

    provider

  7. postmortem

    Saturday outage post-mortem Date of incident: 2025-02-22 Impact: * Complete downtime of the Apify platform from 22.00 UTC to 23.00 UTC. * No Actors could be started. * Those already running were paused and continued running after the incident was resolved. * There was no data loss, but the workloads were disrupted. What happened: * Our user base is growing fast, and the number of new signups is increasing exponentially. The exponentially increased load revealed suboptimalities that had not manifested before. * One misconfigured query then caused a peak in memory due to a load of a suboptimal index, which caused the cluster to crash. What we did: * The cluster size was increased to ensure enough memory until the issue was fixed. * Then, the cluster and all the dependent systems were restarted. Next steps: * We are already re-architecting our primary database clusters to optimize systems for future growth. * As part of this, we aim to decrease the size of all our clusters to ensure performant restarts in case of a problem. * We are currently optimizing indexing and querying among heavy-load components. * In addition, we identified missing metrics that would indicate similar problems ahead of time. We sincerely apologize for the disruption and appreciate your patience. If you have any questions, please reach out to [support@apify.com](mailto:support@apify.com). Sincerely, Apify engineering team

    provider

Reports count in aggregate; nothing about you is shown.

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.

Questions about DWN-59679cb928

Is Apify still down?

No, this incident was resolved 2025-02-22 23:38 UTC. For Apify's live state, see https://down.guide/apify.

How long did it last?

1h 26m, from 2025-02-22 22:12 UTC to 2025-02-22 23:38 UTC, by the times Apify posted.

What was affected?

Apify named Web (apify.com), Datacenter, Dataset, Console (console.apify.com) and 8 more.

What caused it?

Apify hasn't published a root cause. If it does, it shows here.

Where does this come from?

Apify's official status page, https://status.apify.com, quoted verbatim. down.guide's own probes, reports and social counts are labeled as ours.