# Intermittent failures in Policy step evaluations on Prod 7

- Provider: [Harness](https://down.guide/providers/harness)
- Confidence: Confirmed
- Impact: Degraded
- lasted 0m
- started 2026-10-16 10:41 UTC
- id: DWN-8ddfffce69
- Official page: https://stspg.io/pfwpb7pvmlf7

## Updates from Harness
### 2026-10-06 20:00 UTC: resolved
On October 7, 2026, from 08:00 UTC to 11:00 UTC (1:30 PM to 4:30 PM IST), some pipeline executions on the Prod 7 cluster saw intermittent failures in Policy (OPA) step evaluations.

### 2026-11-10 22:17 UTC: postmortem
Summary
Policy evaluation for a pipeline step intermittently failed for one customer with an "unsupported principal type for internal token" error once an enhanced tenant-isolation safeguard was turned on.
Root cause
Our internal services authenticate to each other using signed internal tokens. The service that evaluates policy only accepts tokens whose principal is marked as one of a small set of recognized types; a token with a missing or unrecognized principal type is rejected.
The affected pipeline step was missing a safeguard that other callers rely on to guarantee their internal token always carries a recognized principal type. As a result, under certain conditions it could present a token with an unsupported principal type to the policy service, causing the request to be rejected.
‌
Mitigation
We mitigated the immediate impact by disabling the safeguard.
Remediation
We are applying the fix at the shared client layer used by every caller of the policy evaluation service, including the affected pipeline step, OPA helper utilities, secrets, and connectors, so the fix protects all current and future callers consistently.
\* Tokens with a recognized principal type continue to be forwarded as-is.
\* Tokens with a missing or unsupported principal type will instead be re-signed as the calling service.
Next steps 
We are rolling out the fix gradually, starting in shadow mode, with additional logging and observability added across all call paths so we can confirm the fix covers every use case, including lower-traffic ones, before it is enabled broadly.

From the builder: does your app depend on Harness? [Sazabi](https://www.sazabi.com/?utm_source=down-guide&utm_medium=markdown&utm_content=harness) detects problems in your own logs without hand-written monitors, and you can ask it in plain language which errors mention Harness.

---
down.guide reads providers' official status pages every minute and adds its own evidence, always labeled. Free, no account. Agents: MCP at https://down.guide/mcp, REST at https://down.guide/api/v1, feeds at https://down.guide/feed. Built by [Sazabi](https://www.sazabi.com/?utm_source=down-guide&utm_medium=markdown&utm_content=DWN-8ddfffce69), AI-native observability and security: automatic issue detection, plain-language log search and live dashboards.