# CI Linux Cloud VM Networking issues - intermittent

- Provider: [Harness](https://down.guide/providers/harness)
- Confidence: Confirmed
- Impact: Degraded
- lasted 3h 16m
- started 2026-09-29 18:45 UTC
- Continuous Integration Enterprise(CIE) - Linux Cloud Builds, Continuous Integration Enterprise(CIE) - Linux Cloud Builds
- us, us-west1
- id: DWN-6aaefe013e
- Official page: https://stspg.io/3ldn66vdtbzp

## Updates from Harness
### 2026-09-29 18:45 UTC: investigating
We are currently investigating this issue.

### 2026-09-29 21:49 UTC: monitoring
A fix has been implemented and we are monitoring the results.

### 2026-09-29 22:01 UTC: resolved
This incident has been resolved.

### 2026-10-02 22:40 UTC: postmortem
Summary
Between approximately 11:37 AM and 3:01 PM PDT \\(18:37–22:01 UTC\\) on September 29, 2026, customers using Harness CI self-hosted Linux cloud runners in prod1/prod2 \\(us-west1\\) experienced intermittent networking failures during pipeline execution — primarily Git/network timeouts when fetching source code. 
Root Cause
This was primarily due to saturation of our NAT which was futher exacerbated by a temporary cloud provider compute capacity, preventing new VM provisioning there.
Impact
\* Customers running self-hosted Linux CI builds in us-west1 saw intermittent build/pipeline failures, not a full outage.
\* Observed customer-facing errors included Git fetch timeouts, e.g. Failed to connect to github.com port 443: Connection timed out 
Remediation
\* Immediate: Rebalanced ARM build-VM load from the saturated us-west1 path toward central1 \\(partially limited by central1 capacity\\); restarted the affected Cloud NAT gateways, which stabilized pipeline networking.
Action Items
1. Secure additional NAT gateway/IP capacity \\(In progress\\)
2. Improve NAT saturation detection and alerting ahead of customer impact.
3. Evaluate cross-region capacity contingencies so a single region's compute shortage doesn't block failover mitigation.

From the builder: does your app depend on Harness? [Sazabi](https://www.sazabi.com/?utm_source=down-guide&utm_medium=markdown&utm_content=harness) detects problems in your own logs without hand-written monitors, and you can ask it in plain language which errors mention Harness.

---
down.guide reads providers' official status pages every minute and adds its own evidence, always labeled. Free, no account. Agents: MCP at https://down.guide/mcp, REST at https://down.guide/api/v1, feeds at https://down.guide/feed. Built by [Sazabi](https://www.sazabi.com/?utm_source=down-guide&utm_medium=markdown&utm_content=DWN-6aaefe013e), AI-native observability and security: automatic issue detection, plain-language log search and live dashboards.