An update to our routing control plane incorrectly propagated a bad route for regional egress proxies, this led to the drop of static IP egress traffic from 21:28 UTC to approximately 22:00 UTC. Our static IP monitoring was configured globally in our largest region. We're adding regional static IP monitoring to catch this in staged rollouts (aka, much quicker).
Monitoring · UTC
We've reverted the routing config and static IPs traffic is flowing normally.
Identified · UTC
we've identified a configuration change in our network routing config that looks to have caused the problem. We're rolling out a fix globally.
Investigating · UTC
We're investigating routing failures for deployments using static IPs for egress.