Lever Outage

What to do

• Check whether your product depends on Lever. Check your stack

  Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

• Confirm your retries, queues and scheduled jobs caught up after the 0m it lasted.

  Anything that failed between 2026-08-12 18:13 UTC and 2026-08-12 17:31 UTC may need a replay.

• Subscribe to Lever's feed to hear about the next one first. RSS feed

  https://down.guide/feed?provider=lever works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from Lever 2

1. resolvedAug 12, 2026, 5:31 PM UTC

   At about 10:31am PST on 8/12/2026 Lever Support initiated a backend data query that caused Lever hire services to encounter an out of memory error. The result of this caused site degradation and failed page loads for hire.lever.co between 10:31am - 10:46am PST. Impacted users would notice a "500 error" during the affected page loads. Service was restored to all users as of 10:46am PST. At this time we are working on continued enhancements to mitigate the risk of Lever Support initiated queries triggering outages in the future. An full RCA will be posted to this issue within 7 to 10 business days.

   provider

2. postmortemAug 14, 2026, 6:26 PM UTC

   **Incident Summary** On August 12, 2026, between approximately 10:31 AM and 10:45 AM PDT, the Lever platform experienced partial unavailability. During this 14-minute window, some customers were unable to access the platform, and others experienced elevated error rates for API requests.  **Detection** The issue was identified through internal monitoring. A Datadog alert for high database page faults triggered at 10:37 AM PDT, notifying the engineering team. This allowed responders to isolate the specific database nodes experiencing high load and identify the responsible operations.  **Root Cause** The incident was triggered by a safety mechanism within an internal administrative tool designed to prevent expensive database queries. When an operator searched for unindexed data, this safety check requested a "plan" from the database to see if the query was safe to run. However, because the data was unindexed, the database attempted to evaluate every possible retrieval method, resulting in the same large account being scanned 11 times for a single request. These diagnostic checks bypassed standard time limits, consuming the database read capacity required for normal customer traffic.  **Resolution** Engineering teams identified and terminated the long-running diagnostic operations, which immediately restored database performance and platform stability. Following the cleanup, a database-level safeguard was applied to force these specific queries into a single, efficient path, preventing the database from attempting multiple expensive scans in the future.  **Preventative Measures** * **Redesigning Safety Checks:** We are replacing the current diagnostic method with a new system that verifies query safety by inspecting database definitions directly, eliminating the need to run expensive tests on live data. * **Advanced Alerting**: We are implementing new monitoring for database read churn and long-running operations to detect and resolve similar issues before they affect customers. * **Enhanced Logging:** We are improving internal audit logs to ensure faster identification and attribution of administrative database activity. * Database Optimization: We are adding new indexes to properly support these search patterns, ensuring they are inherently efficient.

   provider

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.