Railway - Notice history

All systems operational

Dashboard - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 99.86%May · 99.99%

Previous page

Next page

Mar 2025
Apr 2025
May 2025
100% - uptime

Builders - Operational

100% - uptime
Mar 2025 · 99.83%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Image Registry - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Deploy - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Third-Party: npm Registry - Operational

Third-Party: Github Integrations - Operational

Third Party: Docker Hub - Operational

Third-Party: GitHub Container Registry - Operational

100% - uptime

EU West (europe-west4 / Amsterdam, Netherlands) - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

US East (us-east4 / Virginia, USA) - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

US West (us-west1 / Oregon, USA) - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Southeast Asia (asia-southeast1 / Singapore) - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

US West Metal (us-west2 / California, USA) - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Southeast Asia Metal (asia-southeast2 / SIngapore) - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025
100% - uptime

US West (us-west1 / Oregon, USA) - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 99.93%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

US East (us-east4 / Virginia, USA) - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 99.93%May · 99.83%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

EU West (europe-west4 / Amsterdam, Netherlands) - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 99.93%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Southeast Asia (asia-southeast1 / Singapore) - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 99.93%May · 99.95%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

TCP Proxy - Operational

100% - uptime
Mar 2025 · 99.98%Apr · 99.93%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025
100% - uptime

DNS - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 99.93%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Traffic - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 99.93%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025
100% - uptime

Logs - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Metrics - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025
100% - uptime

Payments - Operational

100% - uptime
Mar 2025 · 100.0%Apr · 100.0%May · 100.0%

Previous page

Next page

Mar 2025
Apr 2025
May 2025

Notice history

May 2025

Small % of Project 404's
  • Resolved
    Resolved
    This incident has been resolved.
  • Identified
    Identified

    We rolled out a change where we dropped a column unsafely. While the new deploy was rolling out (~5 minutes), old deploys, the ORM we used automatically queried the old field. As a result, some small % of users who hard refreshed (and had to load project state) may have seen a 404 briefly.

    We're rolling out a lint rule to catch this in the future. Apologies

Single host on Railway Metal (us-west2) is offline
  • Resolved
    Resolved

    This incident has been resolved.

  • Monitoring
    Monitoring

    All deployments on the offline host have successfully recovered.

  • Identified
    Identified

    We have identified an issue with the offline host, and confirmed that full recovery is in progress.

  • Investigating
    Investigating

    We are currently investigating this incident. A small subset of impacted users on us-west2 region may notice their deployments are unresponsive during this period. You can re-deploy your service to mitigate this.

Dashboard timeouts
  • Resolved
    Resolved
    This incident has been resolved.
  • Identified
    Identified

    A small subset of users may have noticed a dashboard timeout for up to 2 minutes.

Dashboard Timeouts
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Identified
    Identified

    Some users may notice dashboard timeouts. We are rolling out a fix for this

Elevated Latency all Edge proxies
  • Resolved
    Resolved
    This incident has been resolved.
  • Identified
    Identified

    We have identified the issue, implemented a fix, and are already seeing recovery.

Apr 2025

Elevated latency US East edge proxies
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    We are seeing full recovery, and we are continuing to monitor.

  • Investigating
    Investigating
    We are currently investigating this incident.
Dashboard is experiencing degraded performance. You may notice occassional latency spikes or 503s
  • Resolved
    Resolved

    This incident has been resolved. A full incident report is available at https://blog.railway.com/p/incident-report-april-30-2025.

  • Update
    Update

    We have implemented a fix and are monitoring the results.

  • Monitoring
    Monitoring

    We have implemented a fix and are monitoring the results.

  • Identified
    Identified

    We are continuing to work on a fix for this Issue.

  • Investigating
    Investigating

    The dashboard is experiencing degraded performance. You may notice occasional latency spikes or 503s.

  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    We have implemented a fix and are monitoring the results.

  • Update
    Update

    We are continuing to work on a fix for this Issue.

  • Update
    Update

    The Dashboard is currently inaccessible. We are continuing to work on a fix for the issue.

  • Update
    Update

    We are continuing to work on a fix for the issue.

  • Update
    Update

    We are continuing to work on a fix for the issue.

  • Update
    Update

    We are continuing to work on a fix for the issue.

  • Update
    Update
    Dashboard may be inaccessible for some users
  • Identified
    Identified

    We have identified the issue and are working on a fix

  • Investigating
    Investigating
    Dashboard is experiencing degraded performance. You may notice occassional latency spikes or 503s
US-East delayed placements
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    We have fixed a bug preventing new deployments from landing on the newly provisioned hosts.

  • Investigating
    Investigating

    We are continuing to investigate a subset of deploys that are delayed in the creating container step.

  • Monitoring
    Monitoring

    We have finished adding capacity and are monitoring the process of deployments that were stuck in the queue.

  • Identified
    Identified

    We are scaling up capacity in US-East

A single host on Railway GCP US-West has rebooted
  • Resolved
    Resolved

    All applications are operational. A host was rebooted at 17:48 UTC. Most applications recovered by 18:06 UTC and ALL applications recovered by 18:13 UTC.

  • Monitoring
    Monitoring

    Most applications are operational. We're monitoring for ALL applications to become operational before resolving.

  • Identified
    Identified

    Affected deploys are coming back online. Impacted users may see their services re-deployed by Railway.

  • Investigating
    Investigating

    We are failing over deploys from the host to a different host. Impacted users may see their services re-deployed by Railway.

A single host on Railway GCP US-West has rebooted
  • Resolved
    Resolved

    All applications are operational. A host was rebooted at 13:53 UTC. Most applications recovered by 14:03 UTC and ALL applications recovered by 14:10 UTC.

  • Identified
    Identified

    Affected deploys are coming back online. Impacted users may see their services re-deployed by Railway.

  • Investigating
    Investigating

    We are failing over deploys from the host to a different host. Impacted users may see their services re-deployed by Railway.

Mar 2025

Some TCP proxy endpoints may be experiencing connectivity issues
  • Resolved
    Resolved

    This incident has been resolved and we're seeing full recovery and have successfully rolled out the revert.

  • Identified
    Identified

    We're reverting a change that has interrupted TCP proxy connectivity to a small subset of workloads still on Runtime V1. Workloads on Runtime V2 are unaffected.

  • Investigating
    Investigating
    Some TCP proxy endpoints may be experiencing connectivity issues
Potential upstream issue causing delayed login via Magic emails. Login via GitHub may be used as a workaround.
  • Resolved
    Resolved
    Login via Magic emails are being delivered timely
  • Investigating
    Investigating
    Potential upstream issue causing delayed login via Magic emails. Login via GitHub may be used as a workaround.
A singular google cloud host is failing over
  • Resolved
    Resolved
    Failover completed
  • Investigating
    Investigating
    A singular google cloud host is failing over
A single host in US West was rebooted due to a kernel failure
  • Resolved
    Resolved
    All workloads have now recovered
  • Monitoring
    Monitoring
    We are monitoring as workloads come back online
  • Investigating
    Investigating
    A single host in US West was rebooted due to a kernel failure
Potential upstream GitHub issue with "Failed to clone" for small subset of builds. We're looking into it
  • Resolved
    Resolved
    We've deployed a permanent fix and consider the issue resolved.
  • Monitoring
    Monitoring
    We have rolled out a temporary fix and are working on a permanent one.
  • Update
    Update
    We continue to investigate the incident. It occurs on a very small % of builds and retries resolve
  • Investigating
    Investigating
    Potential upstream GitHub issue with "Failed to clone" for small subset of builds. We're looking into it

Mar 2025 to May 2025

Next