Skip to main content
ServiceNow Cloud Observability

Span Ingestion & Trace Assembly Degraded

Partial outage Lasted 1h 11m · started · ServiceNow Cloud Observability's page

Level
confirmed
Worst impact
Partial outage
Components
US - Trace Assembly, US - Trace Statistics
Regions
us
Down Index
32of 100

What to do

  • Check whether you use ServiceNow Cloud Observability US - Trace Assembly, US - Trace Statistics in us. Check your stack

    Paste a package.json, lockfile, Terraform or a list of domains at down.guide/stack to see every provider you depend on.
  • Confirm your retries, queues and scheduled jobs caught up after the 1h 11m it lasted.

    Anything that failed between 2023-02-16 16:39 UTC and 2023-02-16 17:50 UTC may need a replay.
  • Subscribe to ServiceNow Cloud Observability's feed to hear about the next one first. RSS feed

    https://down.guide/feed?provider=lightstep works in Slack (/feed subscribe), Discord feed bots and any RSS reader.

Updates from ServiceNow Cloud Observability 8

  1. investigating

    We are currently investigating an issue with span ingest into: ingest.lightstep.com.

    provider

  2. investigating

    We have identified an issue ingesting spans for customers sending traffic directly to ingest.lightstep.com. Customers that are running microsatellites are not affected.

    provider

  3. identified

    A suspected issue has been identified and a fix is being rolled out.

    provider

  4. identified

    We have identified an issue ingesting spans for customers sending traffic directly to ingest.lightstep.com. Customers that are running microsatellites are not affected.

    provider

  5. identified

    Span ingestion appears to be functional. Trace assembly is still impacted and actively being worked on.

    provider

  6. identified

    We are continuing to work on a fix for this issue.

    provider

  7. monitoring

    We have deployed the fixes and are actively monitoring the affected systems.

    provider

  8. resolved

    All affected systems appear to be returned to a working state.

    provider

Reports count in aggregate; nothing about you is shown.

Corrections

None so far.

Something wrong on this page? Report an error or send us a note. Every fix is listed here.

Questions about DWN-e0e0c28df9

Is ServiceNow Cloud Observability still down?

No, this incident was resolved 2023-02-16 17:50 UTC. For ServiceNow Cloud Observability's live state, see https://down.guide/lightstep.

How long did it last?

1h 11m, from 2023-02-16 16:39 UTC to 2023-02-16 17:50 UTC, by the times ServiceNow Cloud Observability posted.

What was affected?

ServiceNow Cloud Observability named US - Trace Assembly, US - Trace Statistics in us.

What caused it?

ServiceNow Cloud Observability hasn't published a root cause. If it does, it shows here.

Where does this come from?

ServiceNow Cloud Observability's official status page, https://status.lightstep.com, quoted verbatim. down.guide's own probes, reports and social counts are labeled as ours.