Delayed data processing, Degradation in Fullstory Web application

What to do

• Check whether you use FullStory Data Capture - Web, Data Capture - Native Mobile, API, Fullstory Web Application and 1 more. Check your stack

  Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.

• Confirm your retries, queues and scheduled jobs caught up after the 1h 53m it lasted.

  Anything that failed between 2025-10-22 19:42 UTC and 2025-10-22 21:35 UTC may need a replay.

• Subscribe to FullStory's feed to hear about the next one first. RSS feed

  https://down.guide/feed?provider=fullstory works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from FullStory 3

1. identifiedOct 22, 2025, 7:42 PM UTC

   Customers may have experienced delayed data processing and indexing, or degradation to the Fullstory Web application between 2:46 PM ET to 2:55 PM ET. Data ingested within this timeframe may be delayed or affected. We have identified the issue and are actively working on full resolution.

   provider

2. resolvedOct 22, 2025, 9:35 PM UTC

   This issue is resolved. The Fullstory platform, including login, data processing and indexing, webhooks, APIs, and destinations, are fully operational. If there are any follow up questions, please reach out to support@fullstory.com.

   provider

3. postmortemOct 23, 2025, 10:13 PM UTC

   2025/10/22 Fullstory Networking Disruption On October 22, 2025, the Fullstory platform experienced a 30-minute networking control plane partial outage, beginning at 2:44 PM EDT. This incident led to widespread degradation, impacting network requests for many applications and resulting in customer logouts, partial capture failure, and delayed indexing and warehousing sync. The root cause was identified as a misconfiguration of resource pinning within a component of Fullstory's central networking infrastructure. This misconfiguration prevented the component from restarting successfully after an update, leading to a cycle of crashing and restarting as the remaining service replicas became overloaded. # Customer Impact During the incident window, some API requests, Fullstory Analytics app usage, and data captures would have resulted in HTTP errors. Some users logged into Fullstory Analytics were logged out of the application. Affected sessions will have missing data causing gaps/disruptions when loading playback, webhooks not firing, and events not being exported via the Anywhere: Warehouse feature. Data capture may have been affected for all orgs during the incident window. Please reach out to [support@fullstory.com](mailto:support@fullstory.com) if you have any questions or concerns. # Root Cause Due to an update in configuration, our networking control plane required a progressive restart. Normally this restart happens very quickly without effect on any systems. Due to an unforeseen misconfiguration of resource pinning, the control plane spiked in CPU and caused a domino effect on itself, unable to recover without intervention. # Resolution After receiving alerts on the failed networking components, our Core Infrastructure team took action to quickly implement changes to stop the cascading failures. These interventions were strategically designed to prevent a cascading series of failures across our interconnected systems, thereby mitigating further impact and providing a rapid restoration of services. # Process Changes and Prevention **Actions Taken:** * Stabilized the core networking components * Added resiliency to these components to better handle full load during replica loss * Fixed misconfiguration of resource pinning to allow networking components to fully utilize all available resources ‌ We deeply regret this incident and invite any Fullstory customer who was materially affected to contact [support@fullstory.com](mailto:support@fullstory.com). We stand by ready to fully address all of your concerns.

   provider

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.