Service Outage

What to do

• Check whether you use Frontegg in eu. Check your stack

  Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

• Confirm your retries, queues and scheduled jobs caught up after the 0m it lasted.

  Anything that failed between 2025-05-23 17:16 UTC and 2025-05-23 17:16 UTC may need a replay.

• Subscribe to Frontegg's feed to hear about the next one first. RSS feed

  https://down.guide/feed?provider=frontegg works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Frontegg 2

1. resolvedMay 23, 2025, 5:16 PM UTC

   This incident has been resolved. Timeline of Events: 16:53 UTC: DDOS attack begins 16:54 UTC: Monitoring system detects service degradation and alerts the on-call team. 17:03 UTC: On-call team identifies that a DDOS attack is ongoing. 17:10 UTC: On-call team scopes the characteristics of the attack (volume, source IPs, and traffic patterns). 17:15 UTC: The on-call team applies blocking and rate limit rules on Cloudflare to mitigate the attack. 17:16 UTC: System recovers and service is restored.

   provider

2. postmortemMay 28, 2025, 7:14 AM UTC

   Root Cause Analysis \(RCA\): DDOS Attack Incident ‌ **Incident Summary** On May 23, 2025, between 16:53 to 17:16 UTC, our service in the Europe region experienced a temporary outage due to a sophisticated DDOS attack. Despite mitigation efforts by Cloudflare, the scale and speed of the attack overwhelmed our system's autoscaling capabilities, leading to service unavailability for a short period. ‌ **Timeline of Events:** * **16:53 UTC: DDOS attack begins.** * **16:54 UTC: Monitoring system alerts the on-call team.** * **17:03 UTC: On-call team identifies the DDOS attack.** * **17:10 UTC: Attack characteristics scoped.** * **17:15 UTC: Blocking and rate limit rules applied.** * **17:16 UTC: Service recovers.** ‌ **Root Cause** The attack's high volume and rapid escalation exceeded our system's ability to scale automatically in time, causing service disruption. ‌ **Incident Resolution & Next Steps:** To resolve the incident, we took the following actions: * We successfully blocked malicious traffic and hardened our defenses. * Preventive measures are being implemented, including enhancing CDN, infrastructure autoscaling, automated tools to identify attacks faster, and DDOS protection in collaboration with Cloudflare.

   provider

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.