We're investigating reports of build failures on a subset of builder hosts

Resolved
Partial outage
Started 8 months ago Lasted 13 minutes

Affected

Builds & Deployments
Builders
Updates
  • Resolved
    Resolved

    The affected build node has been cycled. A critical daemon on the node crashed and did not correctly update the nodes healthcheck. Any work landing on that node was thus instantly failing. We're taking measures to fix the healthcheck to ensure only healthy build nodes take on work.

  • Identified
    Identified

    We've identified the affected builder instance and are cycling it out.

  • Investigating
    Investigating

    We're investigating reports of build failures on a subset of builder hosts