Inaccurate day and month stats aggregates since 8th September

What to do

• Check whether you use Ably Ably Platform, Ably Control API, Ably Website in us. Check your stack

  Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

• Confirm your retries, queues and scheduled jobs caught up after the 1d 1h it lasted.

  Anything that failed between 2024-09-24 17:35 UTC and 2024-09-25 18:49 UTC may need a replay.

• Subscribe to Ably's feed to hear about the next one first. RSS feed

  https://down.guide/feed?provider=ably works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Ably 2

1. identifiedSep 24, 2024, 5:35 PM UTC

   We have been alerted to an issue with stats aggregation (app stats through our stats api and account stats through the website dashboard) since 8th September that can lead to messages being double-counted in day and month aggregates. We have identified the problem and are in the process of remediating it. The issue only affects day and month aggregates; minute and hour aggregates are correct. We will be remediating day and month aggregates for all accounts and apps from the known-good data (the hour aggregates). We hope to have this done by the end of the 25th. Since billing is done at the end of the month, we do not expect any customers to be billed incorrectly as a result of this. In the meantime, we advise anyone using the stats api to query for hourly stats instead of daily or monthly. We realise this is inconvenient, and deeply apologise, especially for how long it took us to notice the issue. If you have any further questions or concerns, please contact our support team at https://ably.com/support .

   provider

2. resolvedSep 25, 2024, 6:49 PM UTC

   The remediation is complete; app and account day and month aggregates should now be correct again. Apologies again for the inconvenience. (For nearly all customers at least; for customers on enterprise dedicated clusters it might take a few more hours before the month aggregrate for the current month shows correctly).

   provider

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.