Skip to main content
crates.io

Intermittent timeouts on API endpoints

Partial outage Lasted 7h 13m · started · crates.io's page

Level
confirmed
Worst impact
Partial outage
Components
crates.io
Down Index
42of 100

What to do

  • Check whether you use crates.io crates.io. Check your stack

    Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.
  • Confirm your retries, queues and scheduled jobs caught up after the 7h 13m it lasted.

    Anything that failed between 2024-02-14 19:30 UTC and 2024-02-15 02:43 UTC may need a replay.
  • Subscribe to crates.io's feed to hear about the next one first. RSS feed

    https://down.guide/feed?provider=crates-io works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from crates.io 6

  1. investigating

    Some crates.io endpoints are timing out at present, including the summary route that drives the crates.io home page. We are investigating.

    provider

  2. investigating

    (If this looks suspiciously similar to https://status.crates.io/incidents/t49v2pfpv0vl, the same issue reappeared on the summary endpoint literally within seconds of resolving that incident. C'est la vie.)

    provider

  3. identified

    We believe this issue is being caused by excess database load related to a bug fix deployed earlier today around download counting. This bug fix has caused the normal background processing of per-crate download count totals to take significantly longer and require more resources than usual. We will shortly be temporarily disabling the summary endpoint to alleviate some of the load on the database.

    provider

  4. monitoring

    The long running background job has completed, and response times for the summary endpoint have returned to normal. The next invocation of the relevant background job will be at 00:30 UTC (so in just under 20 minutes); we will be monitoring that closely to see if any problems resurface at that point.

    provider

  5. monitoring

    We are continuing to monitor for any further issues.

    provider

  6. resolved

    crates.io has returned to normal service with the processing of the download count backlog from earlier and the completion of the long running background jobs. Investigations will continue during normal hours for the crates.io team to ascertain what is causing elevated database load.

    provider

Reports count in aggregate; nothing about you is shown.

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.

Questions about DWN-7545240542

Is crates.io still down?

No, this incident was resolved 2024-02-15 02:43 UTC. For crates.io's live state, see https://down.guide/crates-io.

How long did it last?

7h 13m, from 2024-02-14 19:30 UTC to 2024-02-15 02:43 UTC, by the times crates.io posted.

What was affected?

crates.io named crates.io.

What caused it?

crates.io hasn't published a root cause. If it does, it shows here.

Where does this come from?

crates.io's official status page, https://status.crates.io, quoted verbatim. down.guide's own probes, reports and social counts are labeled as ours.