Skip to main content
Frontegg

Api are having Performance degradation

Degraded Lasted 45m · started · Frontegg's page

Level
confirmed
Worst impact
Degraded
Down Index
17of 100

What to do

  • Check whether your product depends on Frontegg. Check your stack

    Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.
  • Confirm your retries, queues and scheduled jobs caught up after the 45m it lasted.

    Anything that failed between 2022-08-14 23:54 UTC and 2022-08-15 00:39 UTC may need a replay.
  • Subscribe to Frontegg's feed to hear about the next one first. RSS feed

    https://down.guide/feed?provider=frontegg works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Frontegg 6

  1. investigating

    We are currently investigating this issue.

    provider

  2. investigating

    We are continuing to investigate this issue.

    provider

  3. investigating

    We are continuing to investigate this issue.

    provider

  4. monitoring

    A fix has been implemented and we are monitoring the results.

    provider

  5. resolved

    This incident has been resolved.

    provider

  6. postmortem

    ### **Executive summary:** On August 15th, 2022 at 02:01 IST \(UTC \+2\) Frontegg underwent a sophisticated DDOS subdomain organized attack. The attackers used multiple servers spread across a variety of Digital Ocean IPs. Each Server executed a low number of requests per second so our WAF did not trigger rate-limiting rules, yet it was recognized that many of the paths were related to WordPress engine's known weakness.  By 03:21 the attack had been successfully mitigated. At 04:46 a second organized attack began. The restrictions put in place by the previous attack were helpful in mitigating the second attack. By 05:30 all traffic returned to normal ### **Affect:** The incident caused a degraded performance to our API gateway. As a result, our API returned 504 and 524 errors to partial traffic over the course of the incident. The majority of these errors were returned between 02:01 IST and 02:30 IST, when our mitigation efforts began to have an effect. A majority of traffic was still able to go through without error during this time. ### **Mitigation and resolution:** Our initial response to the attack was to increase our rate limiting and WAF constraints. This initial step was implemented at 02:30 IST. Once we understood the level of sophistication and distribution of the attack, we implemented changes on the application level, including a different routing mechanism and added more specific WAF constraints based on origins of the attacking traffic, which took effect by 03:21 IST. ### **Preventive steps:** In order to prevent attacks like this in the future, we are implementing a more sophisticated route blocking mechanism to our API-gateway. Additionally we have reported the incident to the cloud provider which hosted a majority of the attacking traffic, and we are consulting with our WAF provider for further guidance on preventing such attacks.

    provider

Reports count in aggregate; nothing about you is shown.

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.

Questions about DWN-1138ea1ef1

Is Frontegg still down?

No, this incident was resolved 2022-08-15 00:39 UTC. For Frontegg's live state, see https://down.guide/frontegg.

How long did it last?

45m, from 2022-08-14 23:54 UTC to 2022-08-15 00:39 UTC, by the times Frontegg posted.

What caused it?

Frontegg hasn't published a root cause. If it does, it shows here.

Where does this come from?

Frontegg's official status page, https://status.frontegg.com, quoted verbatim. down.guide's own probes, reports and social counts are labeled as ours.