SurePath AI - Intermittent Failures with New Session Authentication and AWS Connectors – Incident details

All systems operational

Intermittent Failures with New Session Authentication and AWS Connectors

Resolved
Degraded performance
Started 5 months agoLasted about 5 hours

Affected

Platform Authentication

Degraded performance from 3:32 PM to 5:51 PM, Operational from 5:51 PM to 8:35 PM

Traffic Ingestion & Monitoring

Operational from 3:32 PM to 4:34 PM, Degraded performance from 4:34 PM to 5:51 PM, Operational from 5:51 PM to 8:35 PM

Policy Enforcement

Operational from 3:32 PM to 4:34 PM, Degraded performance from 4:34 PM to 5:51 PM, Operational from 5:51 PM to 8:35 PM

Context Connectors

Operational from 3:32 PM to 4:34 PM, Degraded performance from 4:34 PM to 5:51 PM, Operational from 5:51 PM to 8:35 PM

Cloud Connectors

Operational from 3:32 PM to 4:34 PM, Degraded performance from 4:34 PM to 5:51 PM, Operational from 5:51 PM to 8:35 PM

Telemetry Delivery

Operational from 3:32 PM to 4:34 PM, Degraded performance from 4:34 PM to 5:51 PM, Operational from 5:51 PM to 8:35 PM

Updates
  • Resolved
    Resolved

    This incident has been resolved.

  • Monitoring
    Monitoring

    AWS has provided an update that their hosting outage has been resolved, and we no longer see degraded performance. We are monitoring for ongoing issues, and continue to implement mitigations in-case the AWS infrastructure outage returns or continues.

  • Update
    Update

    Due to the ongoing AWS outage in us-east-1, we are seeing increased error rates for traffic monitoring and customer model usage. Customers who use AWS connectors that leverage us-east-1 may notice failed inference requests, issues with context data, and delayed telemetry delivery. We continue to monitor the AWS incident and will provide updates as they become available.

  • Identified
    Identified

    Due to issues occurring within AWS infrastructure, we have observed some intermittent failures authenticating new user sessions. AWS has identified the problem and they have applied mitigations while working on their permanent fix. We will continue to monitor and are preparing a workaround in case the increased failure rate persists. As these failures are intermittent, users are asked to retry failed authentications until this issue is fully resolved.