- Level
- confirmed
- Worst impact
- Major outage
- Components
- Lokalise API, Lokalise App
- Down Index
- 51of 100
What to do
Check whether you use Lokalise Lokalise API, Lokalise App. Check your stack
Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.Confirm your retries, queues and scheduled jobs caught up after the 5h 57m it lasted.
Anything that failed between 2023-04-19 04:20 UTC and 2023-04-19 10:17 UTC may need a replay.Subscribe to Lokalise's feed to hear about the next one first. RSS feed
https://down.guide/feed?provider=lokalise works in Slack (/feed subscribe), Discord feed bots and any RSS reader.
Updates from Lokalise 8
- investigating
The Lokalise platform (https://app.lokalise.com/) is experiencing performance issues related to loading times, navigation speed within projects, and search functionality. API is also affected. Our team is investigating the root cause of these issues and working diligently to resolve them immediately.
provider
- investigating
We are continuing to investigate this issue.
provider
- investigating
We are continuing to investigate this issue.
provider
- investigating
We are continuing to investigate this issue.
provider
- identified
The issue has been identified, and a fix is being implemented. The app is accessible now, however we disabled search, filtering, and statistics functionality to work on the root cause.
provider
- monitoring
A fix has been implemented and the search, filtering, and statistics functionality are restored. We are monitoring the results.
provider
- resolved
This incident has been resolved.
provider
- postmortem
To better handle the growing amount of data and users in Lokalise, we constantly work on scaling resources for the application. After a routine operational change that was previously executed multiple times and tested on staging environment successfully, Elasticsearch cluster that powers many Lokalise features has become suddenly overloaded. Once more users have started coming online the service began struggling with the load leading to increased latency and general slowness of Lokalise application. We have turned off filters, search, and statistics to make application performant again while in limited mode, and continued to work on resolving the issue. The source of the issue has been established quickly, however full performance restoration took more than an hour before we could re-enable all functionality. It took this long because the Elasticsearch index that had to be relocated was very large. The root cause of the incident was an incorrect estimation of the resources required for scaling the backend service. This was unexpected as it turned out that metrics we have in place did not reveal the full extent of the actual service’s load. We apologize for the inconvenience and frustration caused by the downtime experienced by our customers. Our team takes this incident seriously and is committed to taking all necessary measures to prevent similar incidents from occurring in the future. We appreciate your patience and understanding and will continue to work diligently to improve our system’s performance and reliability for you.
provider
Reports count in aggregate; nothing about you is shown.
Corrections
None so far.
Something wrong on this page? Report an error or send us a note. Every fix is listed here.
Questions about DWN-397224cb7d
Is Lokalise still down?
No, this incident was resolved 2023-04-19 10:17 UTC. For Lokalise's live state, see https://down.guide/lokalise.
How long did it last?
5h 57m, from 2023-04-19 04:20 UTC to 2023-04-19 10:17 UTC, by the times Lokalise posted.
What was affected?
Lokalise named Lokalise API, Lokalise App.
What caused it?
Lokalise hasn't published a root cause. If it does, it shows here.
Where does this come from?
Lokalise's official status page, https://status.lokalise.com, quoted verbatim. down.guide's own probes, reports and social counts are labeled as ours.