Send Pipeline and Event Status Slowdowns

What to do

• Check whether you use Courier API, Observability. Check your stack

  Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

• Confirm your retries, queues and scheduled jobs caught up after the 23m it lasted.

  Anything that failed between 2024-09-06 16:35 UTC and 2024-09-06 16:59 UTC may need a replay.

• Subscribe to Courier's feed to hear about the next one first. RSS feed

  https://down.guide/feed?provider=courier works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Courier 5

1. identifiedSep 6, 2024, 4:35 PM UTC

   The Courier team has identified an issue with the send pipeline and event status updates causing queued messages and delayed webhook events. The Courier team will monitor the bottleneck and adjust message batching necessary to flow messages normally.

   provider

2. identifiedSep 6, 2024, 4:39 PM UTC

   The team has increased our processing to help with the bottleneck.

   provider

3. monitoringSep 6, 2024, 4:44 PM UTC

   The team has increased the send pipeline worker and messages are clearing the bottleneck.

   provider

4. monitoringSep 6, 2024, 4:52 PM UTC

   Corrective actions have cleared the bottleneck and messages and events should be flowing normally.

   provider

5. resolvedSep 6, 2024, 4:59 PM UTC

   Bottleneck has cleared for messages and event statuses.

   provider

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.