Incident Report: May 19, 2026 – GCP Account Suspension

And this is Railway, a big enough name to top the HN main page and presumably find someone from Google to intervene at some point. I would have zero recourse if it was some little product that I built.

> Don't expect Google quality from the name.

It sounds exactly like what I have experienced in terms of Google quality over the decades.

GCP was never known for their support and deprecation of services was always a huge risk. Its very sad because its actually a quality product. They should easily be the number 2 provider. Azure is extremely unreliable and their documentation is subpar. GCP being in 3rd place is more of their doing.

All google products work like this. Should never be used for anything critical.

> Don't expect Google quality

Google has an extremely poor reputation. Why are you thinking differently to that?

Hasn't theGoog acted this way of quick to suspend accounts well before Gemini? I like to bash on LLMs as much as the next guy, but this seems very much like the memory of a gold fish. Or, you are just too young to remember pre-LLMs???

What/who is TK? What is OCI?

This feels like google applying the same anti-spam mindset everywhere: detect risk, ban first, ask questions later.

> It seems like they use Gemini 3.1 Pro to run their production decisions.

They said they are already using Gemini 3.5 Pro internally.

Meta is no different. I know a company that had their OAuth app on Meta rendered completely unusable just because one of their employees (a dev) had their personal Facebook account banned by Meta for no reason. They tried to escalate it multiple times but got nowhere, lol. Meta is even worse because accounts need to be 'personal'; if you have a Business Manager, the users added to it are all tied to their personal Meta/Facebook accounts. This is ludicrous.

More businesses need to hear this message. Google has proven time and time again they cannot be trusted as a service provider, exactly because of this problem.

They trust them enough to still give them money, just goes to show how entrenched big tech is and why they need to be broken up into dozens of pieces.

There is a history going back many years of Google suspending or terminating accounts with no explanation, often having to backtrack after users published their frustration and the incident went viral.

Google has always acted as if they have no obligations whatsoever to their paying customers.

Somebody else's computer

They have not explained WHY their account was suspended. That's the most important part, imo. Cloud Providers don't suspend entire accounts for no reason.

Never could. Google might block your entire company because one of your workers did something nasty on their personal account, and their ban hammer is mighty and blocks all related accounts to the Nth degree

Hasn't every cloud provider had issues? Is the enshitification of servces really isolated to Google, or are we all doomed.

Railway has an overwhelming incentive to pin the blame on Google. This report doesn't answer why Google suspended Railway's account.

I'd wait for more details before adjudicating.

I'm not sure that's the lesson to learn from this outage. Hell Google resolved the problem in 7 minutes which is as good as you could hope for.

The resulting action should be you have proper disaster recovery, failover, etc.

Not sure I would trust these folks if this is the conclusion they are coming to from this experience. Any cloud provider can/will do this to you.

Railway don't have a great reputation for building scalable systems (effects of vibe coding?). It's worth waiting for Google's response before jumping to conclusions. They can move to Azure/AWS/own datacenter, but there's a good chance this will repeat in a few months.

This exact thing happened to me when I ran https://www.fatherly.com/ circa 2017. Google just shut down our account without notice. We were spending like $10k/month. It also locked us out of our premium support account, so we couldn't even get anyone there to notice that they'd locked us out.

After about 8 hours, a random Google support tech said it was because we were mining bitcoin, which was laughably untrue. We had CPU usage graphs and logs for the whole time and there was no spike. At around 12 hours, they turned it back on, said it was "misconfiguration of our abuse detection" and gave us like $100 in credit.

Absurd. Say what you will about AWS, they would never do that to a customer without a rep reaching out to you first. I have not trusted GCP since.

Shouldn't Google answer this if they are unhappy with this incident report? Are we even sure that Railway knows?

I don't think you're typically told why for these things, and it's mostly automated from what I can tell. The automated systems make mistakes but more importantly they're completely opaque. Nobody, not even Google, knows how they work exactly.

Who is the "You" in "you haven't addressed the root cause"? If you are asking Railway to spend effort doing this rather than simply moving away from GCP, I'm not sure why they would unless they want to sue GCP to recover damages to brand and long term customer retention.

The moment GCP shut off without any forewarning, its done deal, no need to ask any further questions.

That‘s the point where Google tells you they won’t tell you the exact reason because of security reasons

it sounds to me like templated text from the UVic ESS office somewhere:P

Building on someone else's platform is always gonna be a risky move, and building a platform on top of someone else's platform is even riskier.

My company used to use a hosting provider that was basically AWS plus some extra guarantees. We just finished migrating onto regular AWS because they now offer what we need directly.

In the case of them giving AI admin credentials to delete their production database, and it deleted their production database: that's on them. They were the only ones who put the admin account credentials into their AI.

Then they took no personal responsibility. That definitely damaged their reputation. Here, they are taking at least some responsibility. Props to them on improving.

Also, GCP does indeed have serious reliability issues, and Google does indeed have serious customer support issues.

EDIT: It has been brought to my attention below that the first 2 paragraphs are misattributed, and were not Railway, but rather a customer of theirs. Sorry, Railway!

What were your reasons for going with railway in the first place? I'm not super familiar with them, but did you choose them for unique offerings, or essentially just VMs? If unique offerings, how rough was the migration out?

Azure suspended your account as well?

DigitalOcean. Seriously. They have been around a long long time and built a lot of the core infrastructure you rely on every day (e.g. Ceph).

If you are unable to use IaaS directly. You need to accept that your service might be down.

Even if you use AWS and the like, if you aren't building your app with redundancy across multiple AZs, then you'll have some downtime occasionally.

And even if you do build redundancy with multiple AZ, some services might fail anyway as AWS is not entirely isolated. So you might have downtimes.

So just accept downtimes and use the best tool for you (unless they are really bad, like GitHub level bad). If you cannot accept any downtime, you'll have to spend millions of dollars and months of work to have the confidence to expect no downtime. Something like Netflix's chaos monkey and infrastructure would be enough.

I think the message here is that you can't trust any single cloud provider. You at least need two with full operational capability.

Ramnode had always worked well for my projects.

An intermediary can provide value but there’s also a risk so I’d consider why you don’t want to use AWS, GCP, etc. directly. All of the major cloud providers have services which are only slightly harder than what Railway does but allow you to grow into more advanced things as your needs expand without adding a third-party who controls your features, security, and availability.

As an example, I note that GCP responded within 7 minutes according to their timeline. If you’d been using Cloud Run, that would have reduced downtime by over 7 hours — and there’s a good chance that you never would have gone down in the first place if the unknown trigger event was related to other customer activity or something odd Railway did.

There’s also a complexity factor: note how much complex infrastructure they mentioned having to fix that you wouldn’t need for your own account. That code does useful things, I’m sure, but it’s also a lot of moving parts which a hosting provider needs and you don’t – this outage took everyone down, whereas individual AWS or bare metal users would’ve otherwise been unaffected. There isn’t a global optimum which is the same for everyone but I think developers are prone to wildly over-estimating how much time they save by removing a couple of deployment steps relative to the direct costs and the less obvious costs of working within someone else’s environment.

Fly, Render, and even Heroku still are all better choices then working with Railway I think

Why does nobody consider that you can buy a baremetal box or even a VPS and that will get you very far without paying a metered fee

Hatchbox + Digital Ocean is an unbeatable combo and provides Railway-like automation with self-owned infra.

Depending on exactly what you're building, all of these things sounds like one VPS. A bit of maintenance/security burden managing the machine if you're not used to it but as the others have said: Next.js can be selfhosted, unless you need the serverless/edge stuff; then I would go to Cloudflare Workers.

Maybe a VPS? Simple to manage and way cheaper.

But really any service (or even on-site hosting) can have downtime, if that's not acceptable then I suppose building/using a tool that can be distributed between multiple hosts located in different geographical areas is the best option.

Haven't used railway but my understanding is they are something similar to Heroku. Fly.io has been pretty great for tiny projects in that niche.

For Vercel if your nextjs site can be compiled statically you could probably throw it up on almost anything. We've self hosted before which is pretty straightforward but you lose a lot of the image optimization stuff unless you go deep into setting up open next.

Fly.io (AFAIK) still has a relatively good track record?

Hetzner (or any VM provider) + Dokku works best.

I love how everyone in Silicon Valley acts like Microsoft doesn’t exist.

Azure!

It’s the enterprise cloud with enterprise support. They won’t randomly pull the plug on your account, unlike companies that have a wildly different cultural background:

Google - ad tech (you’re the product)

Amazon - shop front (you’re a comptetitor)

Oracle - lawyers (you’re a future lawsuit for license extortion)

Etc…

[dead]

Shameless self plug but check out: https://specific.dev (especially if you use coding agents)

No code lock-in through SDKs and built on top of AWS with great DX for both developer and coding agents

> If the timestamps are accurate, what was causing the errors 10 minutes before the account was suspended?

Assuming the timestamps are accurate, Google probably started terminating resources while the account was not "suspended" and only completed that after all resources were disabled.

The 22:20 timestamp from the body of the post is wrong. The timeline section (where the 22:10 timestamp came from) is consistent with itself, and also contains:

> May 19, 22:19 UTC - Root cause identified: Google Cloud Platform has suspended Railway's production account.

They couldn't have identified the root cause before it happened.

That 10 minutes is likely very normal. Possibly...

* A Google employee messes up a setting (like one of the previous incidents) triggers something that looks like a suspension is warranted and it takes 10 minutes to flow through the process to suspend.

* A Railway customer does something corrupt, or seemingly corrupt, Google's system starts limiting access and take 10 minutes to decide it should be a suspension.

These are even more likely if there is a person in the loop to approve, who obvious did not dig deep enough to see that they should not have done so.

The category is “Platform as a Service”

Alternative to Fly or Heroku

Here is my source code Run it on the cloud for me I do not care how

In this case it looks like they also bundle together a bunch of the other services you would need to get code onto the platform, monitor it once it’s there and so on

It could be automated action based on abuse reports. TONS of spam comes from Railway associated networks.

The problem is scale. Google uses automation and doesn't have the people to review the actions of that automation. I never worked at Google but this is the most obvious explanation from watching these things happen for years and years.

Please, someone that worked at Google, please comment.

I’m 100% willing to believe it based on Google’s track record.

Looks like it was endorsed by dang: https://news.ycombinator.com/item?id=48210941

Honestly, i have been wanting to suggest to my leaders that we should go to on-prem for primary, and use cloud only as extra for peak traffic and/or failover, etc...but, the culture where i'm at is so bought into cloud as if it solves all problems...and then, in the next breath they all ask me to drastically reduce cloud costs and ensure 100% uptime at all times 24/7/365 (1005 uptime without complexity and without any added costs!).

It really is amazing that there is not some level at which "human review" becomes mandatory. Customers of that size already have dedicated account rep contacts.

I can't believe Kurian has not put his foot down about this. Adverse action against accounts over $X ARR absolutely must have review by revenue-carrying people before the action is taken.

I think this is just the default endgame of large corporates which suck up large quantities of customers. They are a race to the bottom and you end up with service by footgun. My own company is responsible for doing this in our sector. Literally every technology decision favours automation over verification because it's cheaper to say sorry than do it right.

I’m no Microsoft fan, but they are pretty good at long term support of enterprises. Way, way better than Google from what I can tell.

Amazon played AWS from day 1 as if they were the runner-up (and in a sense they were), and while it does look like it's day 2 there, they are not letting the momentum down

Microsoft might have technical warts but commercially they are strong and Azure is a lot of times bundled with other services and you know you can get someone on the phone if needed

Google has... ?

Honestly they really are starting to look that way. Total opinionated Walled Garden that's against an open and thriving ecosystem. Unlike Microsoft the technology is not yet garbage but I hope this isn't where they're going to end up

What intermediate is involved?

TK TK TK TK. Welcome to the jungle.

> These cloud providers need to step back and observe how terrible they've made these products.

They don't, because the allure of effortless scaling is hard to resist: everyone thinks of themselves as the next tech unicorn. And if you actually become an unicorn, you're already too dependent on AWS / Azure / GCP to easily move somewhere else. At best, your strategy is to become "multi-cloud".

It's really surprising how much cheaper colo becomes if you have an even vaguely predictable workload. And you don't have to be a major customer, either -- the data centers will happily sell you single U's or a couple U's, even on a monthly basis if you ask, making it perfectly viable for startups or advanced personal projects.

> ...These cloud providers need to step back and observe how terrible they've made these products...

I doubt that will happen because none of them want to stop the money-making machine they have! And, if your thought after my comment is that all us techies are making a fuss, so the cloud providers and businesses using them will hear our cries and trigger a backlash...? I doubt that to...because some senior business leaders that i see are bent on listening more to management consultants as opposed to abalance of folks including their own internal experts...but, alas, maybe i'm just having too cynical a day today. :-)

The thing that's nice about physical datacenters with people is that they often have to physically walk over to disconnect you - it's not as easy as some automated system doing an AI.

But not so wrong that they could get their account back in 10 minutes.

Yup...