Skip to main content
GitHub

Delayed visibility of newly added issues on project boards

Degraded Lasted 6h 44m · started · GitHub's page

Level
confirmed
Worst impact
Degraded
Components
Issues
Down Index
24of 100

What to do

  • Check whether you use GitHub Issues. Check your stack

    Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.
  • Confirm your retries, queues and scheduled jobs caught up after the 6h 44m it lasted.

    Anything that failed between 2026-03-02 23:10 UTC and 2026-03-03 05:54 UTC may need a replay.
  • Subscribe to GitHub's feed to hear about the next one first. RSS feed

    https://down.guide/feed?provider=github works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from GitHub 13

  1. investigating

    We are investigating reports of impacted performance for some GitHub services.

    provider

  2. investigating

    Issues is experiencing degraded performance. We are continuing to investigate.

    provider

  3. investigating

    Newly added issues can take up to 30 minutes to appear on project boards. We're investigating the cause of this delay.

    provider

  4. investigating

    Newly added issues are taking 30–60 minutes to appear on project boards, compared to the normal near-real-time behavior. We're investigating the root cause and possible mitigations.

    provider

  5. investigating

    The impact extends beyond adding issues to project boards. Adding pull requests and updating fields such as "Status" may also be affected. We're continuing to investigate the root cause.

    provider

  6. investigating

    Project board updates — including adding issues, pull requests, and changing fields such as "Status" — are currently delayed by 1–2 hours. Normal behavior is near-real-time. We're actively investigating the root cause.

    provider

  7. investigating

    The delay for project board updates has increased to up to 3 hours. We've identified a potential cause and are working on remediation.

    provider

  8. investigating

    We're deploying a fix targeting the increased delay in GitHub Projects updates. The rollout should complete within 60 minutes. If successful, the current delay of up to 4 hours should begin to decrease.

    provider

  9. investigating

    The fix is still building and is expected to deploy within 60 minutes. The current delay for GitHub Projects updates has increased to up to 5 hours.

    provider

  10. investigating

    The fix has been deployed and processing speeds have returned to normal. There is a backlog of delayed updates that will continue to be worked through — we're estimating how long that will take and will provide an update in the next 60 minutes.

    provider

  11. investigating

    The backlog of delayed updates is expected to fully clear within approximately 1 hour, after which project board updates will return to near-real-time.

    provider

  12. investigating

    This incident has been resolved. Project board updates are now processing in near-real-time.

    provider

  13. resolved

    Between March 2, 21:42 UTC and March 3, 05:54 UTC project board updates, including adding new issues, PRs, and draft items to boards, were delayed from 30 minutes to over 2 hours, as a large backlog of messages accumulated in the Projects data denormalization pipeline. The incident was caused by an anomalously large event that required longer processing time than expected. Processing this message exceeded the Kafka consumer heartbeat timeout, triggering repeated consumer group rebalances. As a result, the consumer group was unable to make forward progress, creating head-of-line blocking that delayed processing of subsequent project board updates. We mitigated the issue by deploying a targeted fix that safely bypassed the offending message and allowed normal message consumption to resume. Consumer group stability recovered at 04:10 UTC, after which the backlog began draining. All queued messages were fully processed by 05:53 UTC, returning project board updates to normal processing latency. We have identified several follow-up improvements to reduce the likelihood and impact of similar incidents in the future, including improved monitoring and alerting, as well as introducing limits for unusually large project events.

    provider

Reports count in aggregate; nothing about you is shown.

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.

Questions about DWN-e8797f1634

Is GitHub still down?

No, this incident was resolved 2026-03-03 05:54 UTC. For GitHub's live state, see https://down.guide/github.

How long did it last?

6h 44m, from 2026-03-02 23:10 UTC to 2026-03-03 05:54 UTC, by the times GitHub posted.

What was affected?

GitHub named Issues.

What caused it?

GitHub hasn't published a root cause. If it does, it shows here.

Where does this come from?

GitHub's official status page, https://www.githubstatus.com, quoted verbatim. down.guide's own probes, reports and social counts are labeled as ours.