Railway - Potential upstream GitHub issue with "Failed to clone" for small subset of builds. We're looking into it – Incident details

Potential upstream GitHub issue with "Failed to clone" for small subset of builds. We're looking into it

ResolvedPartial outage4 hours 7 minutes

Affected

Builds (Legacy)

Partial outage from 5:43 PM to 8:03 PM, Degraded performance from 8:03 PM to 9:27 PM, Operational from 9:27 PM to 9:50 PM

Build Machines (GCP)

Partial outage from 5:43 PM to 8:03 PM, Degraded performance from 8:03 PM to 9:27 PM, Operational from 9:27 PM to 9:50 PM

  • ResolvedUTC
    We've deployed a permanent fix and consider the issue resolved.
  • MonitoringUTC
    We have rolled out a temporary fix and are working on a permanent one.
  • UpdateUTC
    We continue to investigate the incident. It occurs on a very small % of builds and retries resolve
  • InvestigatingUTC
    Potential upstream GitHub issue with "Failed to clone" for small subset of builds. We're looking into it