Migration causing temporary table lock

What to do

• Check whether you use Flagsmith Admin API, Admin Dashboard in us. Check your stack

  Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

• Confirm your retries, queues and scheduled jobs caught up after the 0m it lasted.

  Anything that failed between 2022-02-03 13:25 UTC and 2022-02-03 13:25 UTC may need a replay.

• Subscribe to Flagsmith's feed to hear about the next one first. RSS feed

  https://down.guide/feed?provider=flagsmith works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Flagsmith 2

1. resolvedFeb 3, 2022, 1:25 PM UTC

   We've resolved an issue with a database migration that caused a temporary full table lock when modifying an index. Total outage was around 70 seconds. We're going to investigate the root cause and provide an update when it's ready.

   provider

2. postmortemFeb 3, 2022, 1:58 PM UTC

   ## Root Cause At 13:15 UTC on 03 Feb 2022, we began deploying a routine release of the Flagsmith application to our production SaaS environment. This release included a database migration which added a new unique index to one of our tables which holds information about multivariate values for features. When the migration was run in our other environments we noticed no ill effects from the addition of the index, however, in production where we have substantially more data this index took longer to add than anticipated and required a full table lock during that period.  ## Downtime Our monitoring shows that the application was unresponsive for a period of just under 2 minutes while the migration was running.  ## Long Term Remediation To improve on this in the future, we are planning to upgrade our version of Django to allow us to easily add indexes concurrently. We will also be monitoring more carefully for future index additions and checking whether they will require a table lock. Finally, we will be looking at making our staging environment more representative of production in terms of data so that we can catch issues such as this in the future.

   provider

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.