Skip to main content
down.guideby Sazabi
GitHub
DWN-e4b74b02eaGitHubResolved

Incident across several services

Degraded Lasted 18h 44m · started · GitHub's page

Level
confirmed
Worst impact
Degraded
Components
API Requests
Regions
us
Down Index
27of 100

What to do

  • Check whether you use GitHub API Requests in us. Check your stack

    Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

  • Confirm your retries, queues and scheduled jobs caught up after the 18h 44m it lasted.

    Anything that failed between 2026-09-23 10:11 UTC and 2026-09-24 04:55 UTC may need a replay.

  • Subscribe to GitHub's feed to hear about the next one first. RSS feed

    https://down.guide/feed?provider=github works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from GitHub 16

  1. investigating

    We are investigating reports of degraded performance for API Requests

    provider

  2. investigating

    Database replicas have detached. We're working to restore the database replicas. Users may experience issues beyond creating organizations and a degraded experience with the GitHub API and Projects.

    provider

  3. investigating

    Database replicas have been restored. Org creation and the GitHub API are no longer degraded.

    provider

  4. investigating

    The degradation affecting API Requests has been mitigated. We are monitoring to ensure stability.

    provider

  5. investigating

    We are seeing recovery for Projects. Users may experience stale search results for Projects while indexing catches up.

    provider

  6. investigating

    Users may experience stale Project search results. We are working to increase indexing speed. All other services are available.

    provider

  7. investigating

    Updates to issue labels may be delayed in being reflected in Projects by about ~10 minutes. We have added some capacity to work through the backlog more quickly, but it'll likely be a few hours to complete processing the full backlog of messages. All other services are available.

    provider

  8. investigating

    We will post another update in approximately one hour to share our progress.

    provider

  9. investigating

    Continuing to investigate the lag that may be experienced in issue labels being accurately reflected in Projects. We are working on alternate solutions to process the backlog of label updates.

    provider

  10. investigating

    We are preparing to deploy a change that will mitigate the impact.

    provider

  11. investigating

    Updates to issue labels may be delayed in being reflected in Projects. We are continuing to deploy a change that will accelerate processing of the backlog of label updates. All other services are available. We will provide another update within the next hour.

    provider

  12. investigating

    We've deployed a change intended to accelerate processing of the backlog of issue label updates in Projects. A sizable backlog still remains and we continue working through it. All other services are operating normally. We will provide another update within the next hour.

    provider

  13. investigating

    We are continuing to process the backlog of issue label updates for Projects. Users may still see delays before label changes are reflected in Projects. All other services are operating normally.

    provider

  14. investigating

    We are continuing to process the backlog of issue label updates for Projects. Label changes may still be delayed. All other services are operating normally.

    provider

  15. monitoring

    The degradation has been mitigated. We are monitoring to ensure stability.

    provider

  16. resolved

    Starting at 07:57 UTC on September 23, GitHub experienced elevated 500 and 404 responses across several application pages. This caused failures when installing GitHub Apps, creating organizations, and making some organization membership changes. Customers also experienced delayed label updates and stale search results in Projects. The infrastructure failure was isolated to our Azure Central US region. The API errors were mitigated by 10:58 UTC on September 23. Projects' processing continued to recover while an accumulated backlog was drained, and full service was restored at 04:55 UTC on September 24. The incident was caused by a failed planned maintenance operation on a primary database. Automated recovery initiated an emergency database failover, after which several replicas in the affected region were unable to resume replication correctly. This reduced available database capacity and caused the API errors and downstream Projects processing delays. We have mitigated the immediate failure mode. We are also improving maintenance safety checks, database failover handling, post-failover replica validation, and downstream processing resilience to reduce the likelihood and impact of similar incidents.

    provider

Reports count in aggregate; nothing about you is shown.

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.

Questions about DWN-e4b74b02ea

Is GitHub still down?

No, this incident was resolved 2026-09-24 04:55 UTC. For GitHub's live state, see https://down.guide/github.

How long did it last?

18h 44m, from 2026-09-23 10:11 UTC to 2026-09-24 04:55 UTC, by the times GitHub posted.

What was affected?

GitHub named API Requests in us.

What caused it?

GitHub hasn't published a root cause. If it does, it shows here.

Where does this come from?

GitHub's official status page, https://www.githubstatus.com, quoted verbatim. down.guide's own probes, reports and social counts are labeled as ours.

Sources

More from GitHub

Feed