Railway - Notice history

All systems operational

Dashboard (railway.com) - Operational

100% - uptime
Apr 2025 · 99.86%May · 99.99%Jun · 100.0%

Previous page

Next page

Apr 2025
May 2025
Jun 2025
100% - uptime

DNS - Operational

100% - uptime
Apr 2025 · 99.93%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2025
May 2025
Jun 2025

Traffic - Operational

100% - uptime
Apr 2025 · 99.93%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2025
May 2025
Jun 2025

Notice history

Jun 2025

Some users are failing to login. We're looking into it
  • Postmortem
    Postmortem

    Incident Report: https://blog.railway.com/p/incident-report-june-6-2025

  • Resolved
    Resolved
    We've resolved the issue and put mitigation in place. We will issue a post mortem today.
  • Monitoring
    Monitoring
    Logins should be recovered. We are monitoring.
  • Update
    Update
    We are still working on this issue.
  • Identified
    Identified

    It appears GitHub has rate limited initial logins again. We have re-escalated.

  • Resolved
    Resolved
    Looking good after monitoring. Resolved
  • Monitoring
    Monitoring
    Logins should be recovered. We are monitoring.
  • Identified
    Identified
    GitHub is rate limiting some secondary sign ins. We're rolling out mitigation strategies and escalating upstream
  • Investigating
    Investigating
    Some users are failing to login. We're looking into it
Deploys are taking longer than normal to go out. We are investigating
  • Resolved
    Resolved
    All deployments are moving through normally
  • Monitoring
    Monitoring
    Hobby provisions have been re-enabled
  • Identified
    Identified
    We are rolling out a fix. Deploys will start moving through normally
  • Update
    Update
    We are still investigating. Trial deploys have been temporarily paused
  • Investigating
    Investigating
    Deploys are taking longer than normal to go out. We are investigating
404 flashing observed for some users. We're rolling out a fix
  • Resolved
    Resolved
    Resolved
  • Monitoring
    Monitoring
    Fix rolled out. Issue should be resolved. Monitoring
  • Investigating
    Investigating
    404 flashing observed for some users. We're rolling out a fix

May 2025

we're temporarily pausing deployments while we investgate delays in network propagation
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    We have shipped a remedy for network discovery. We have re-enabled deployments. We implemented a fix and are currently monitoring the result.

  • Investigating
    Investigating
    we're temporarily pausing deployments while we investgate delays in network propagation
Elevated latency on Edge Network
  • Resolved
    Resolved

    During a routine update to BGP configuration for Railway's Anycast network 66.33.22.0/23, a small proportion (approximately 8%) of upstream ISPs failed to propagate a routing update - this led to a small amount of traffic to be dropped for a period less than 5 mins.

    Once routes propagated, some ISPs were routed in a suboptimal manner leading to elevated latency for some users. We have reverted the routing configuration and are liaising with our vendors to root cause the issue. We will also be creating maintenance windows for future routing config changes and staging their rollout to minimize risk.

  • Monitoring
    Monitoring

  • Identified
    Identified

    We are continuing to work on a fix for this incident.

  • Investigating
    Investigating

    As we're rolling out a routing reconfiguration of our metal edge, a small subset of users may notice latency issues while the new routing configuration is being propagated to all ISPs.

GCP edge network connectivity issues in the EU region
  • Resolved
    Resolved

    This incident has been resolved.

  • Monitoring
    Monitoring

    We are seeing recovery and continuing to monitor the situation.

  • Identified
    Identified

    Attack targeting EU edge identified. Mitigation efforts underway.

  • Investigating
    Investigating
    We are currently investigating this incident.
[Retroactive] Magic Login Outage
  • Resolved
    Resolved

    (This is a retroactive incident) magic login links failed to go out since this morning

Small % of Project 404's
  • Resolved
    Resolved
    This incident has been resolved.
  • Identified
    Identified

    We rolled out a change where we dropped a column unsafely. While the new deploy was rolling out (~5 minutes), old deploys, the ORM we used automatically queried the old field. As a result, some small % of users who hard refreshed (and had to load project state) may have seen a 404 briefly.

    We're rolling out a lint rule to catch this in the future. Apologies

Apr 2025

Elevated latency US East edge proxies
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    We are seeing full recovery, and we are continuing to monitor.

  • Investigating
    Investigating
    We are currently investigating this incident.
Dashboard is experiencing degraded performance. You may notice occassional latency spikes or 503s
  • Resolved
    Resolved

    This incident has been resolved. A full incident report is available at https://blog.railway.com/p/incident-report-april-30-2025.

  • Update
    Update

    We have implemented a fix and are monitoring the results.

  • Monitoring
    Monitoring

    We have implemented a fix and are monitoring the results.

  • Identified
    Identified

    We are continuing to work on a fix for this Issue.

  • Investigating
    Investigating

    The dashboard is experiencing degraded performance. You may notice occasional latency spikes or 503s.

  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    We have implemented a fix and are monitoring the results.

  • Update
    Update

    We are continuing to work on a fix for this Issue.

  • Update
    Update

    The Dashboard is currently inaccessible. We are continuing to work on a fix for the issue.

  • Update
    Update

    We are continuing to work on a fix for the issue.

  • Update
    Update

    We are continuing to work on a fix for the issue.

  • Update
    Update

    We are continuing to work on a fix for the issue.

  • Update
    Update
    Dashboard may be inaccessible for some users
  • Identified
    Identified

    We have identified the issue and are working on a fix

  • Investigating
    Investigating
    Dashboard is experiencing degraded performance. You may notice occassional latency spikes or 503s
US-East delayed placements
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    We have fixed a bug preventing new deployments from landing on the newly provisioned hosts.

  • Investigating
    Investigating

    We are continuing to investigate a subset of deploys that are delayed in the creating container step.

  • Monitoring
    Monitoring

    We have finished adding capacity and are monitoring the process of deployments that were stuck in the queue.

  • Identified
    Identified

    We are scaling up capacity in US-East

A single host on Railway GCP US-West has rebooted
  • Resolved
    Resolved

    All applications are operational. A host was rebooted at 17:48 UTC. Most applications recovered by 18:06 UTC and ALL applications recovered by 18:13 UTC.

  • Monitoring
    Monitoring

    Most applications are operational. We're monitoring for ALL applications to become operational before resolving.

  • Identified
    Identified

    Affected deploys are coming back online. Impacted users may see their services re-deployed by Railway.

  • Investigating
    Investigating

    We are failing over deploys from the host to a different host. Impacted users may see their services re-deployed by Railway.

A single host on Railway GCP US-West has rebooted
  • Resolved
    Resolved

    All applications are operational. A host was rebooted at 13:53 UTC. Most applications recovered by 14:03 UTC and ALL applications recovered by 14:10 UTC.

  • Identified
    Identified

    Affected deploys are coming back online. Impacted users may see their services re-deployed by Railway.

  • Investigating
    Investigating

    We are failing over deploys from the host to a different host. Impacted users may see their services re-deployed by Railway.

Apr 2025 to Jun 2025

Next