> Viva.com's outgoing verification emails lack a Message-ID header, a requirement that has been part of the Internet Message Format specification (RFC 5322) since 2008
> ...
> `Message-ID` is one of the most basic required headers in email.
+----------------+--------+------------+----------------------------+
| Field | Min | Max number | Notes |
| | number | | |
+----------------+--------+------------+----------------------------+
| | | | |
|/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
... bla bla bla ...
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
| message-id | 0* | 1 | SHOULD be present - see |
| | | | 3.6.4 |
|/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
... more bla bla ...
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/|
| optional-field | 0 | unlimited | |
+----------------+--------+------------+----------------------------+
and in section 3.6.4:
... every message SHOULD have a "Message-ID:" field.
That says SHOULD, not MUST, so how is it a requirement?
I know you're looking for "pedant points" but the specification generally take a backseat to implementation. If Message-ID is expected out here where the rubber meets the road, then you are the squeaky wheel in this scenario for not including it.
SHOULD is a requirement. It means that you have to do it unless you know some specific reason that the requirement doesn't apply in your case. "I don't want to" is not a valid excuse, "I don't see a reason to" isn't either.
IIRC this particular rule is a SHOULD because MUAs often send messages without a Message-ID to their submission server, and the submission server adds one if necessary. https://www.rfc-editor.org/rfc/rfc6409.html#section-8.3 The SHOULD lets those messages be valid. Low-entropy devices that can't generate a good random ID are rare these days, but old devices remain in service, so the workaround is IMO justified.
> "I don't want to" is not a valid excuse
"The client SHOULD but didn't" isn't a valid excuse for the server to reject a client either.
You can do it anyway, you might even have good reasons for it, but you sure don't get to point at the RFC and call the client broken.
As someone who does systems engineering, the only valid requirements include the word "shall".
The official definition of SHOULD per RFC2119:
3. SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.
Not sure how the people at Google interpreted this about the message-id
You can argue that you not obligated to use message-id but if you don't use it you should blame only yourself that your messages are not accepted. In requiring message-id I would side with google (though in general I think they anti-spam is too aggressive and lacks ways to report false positives). Full RFC compliance (as in not only MUST but also SHOULD unless you have a very good reason) is the easiest part of making sure your emails will be delivered.
> if you don't use it you should blame only yourself that your messages are not accepted
I think it's a gray area
- If the receiver declines your message because "Message-id" is required - then I blame the receiver; because that's not true
- If the receiver declines your message because "most systems do include it, and it's lack of presence is highly correlated with spam email", then it's on the sender
Admittedly, the end result is the same.
I think it's the latter. But, in either case, you're right in that you get the same result.
Now, let's assume that if it is the latter (it's spam related), and Google were to accept the message, but then internally bin the message, it would be worse. At least in this case, they are bouncing the message. Because of this, the sender is at least aware that the message wasn't delivered.
Also, the author was able to get their mail delivered to a personal gmail.com address. The issue was with a Google Workspace custom email domain. This further makes me think of this as a security/spam related issue. Google is clearly capable of processing the message without a Message-id, they are just refusing for business customers.
My takeaway is that I think that Google is doing the least-wrong thing. And by being explicit in how they are handling it, it at least made the debugging for the author possible.
Also note: in a quick reading of RFC5321 (SMTP), rejecting messages for "policy reasons" is an acceptable outcome. I'm not sure if it applies completely here. The author should probably also be taking into account RFC5321 (SMTP) instead of just 5322 (message format).
> Also, the author was able to get their mail delivered to a personal gmail.com address. The issue was with a Google Workspace custom email domain. This further makes me think of this as a security/spam related issue. Google is clearly capable of processing the message without a Message-id, they are just refusing for business customers.
That's the annoying part to me.
An email is an email. By applying different rules for rejection on different mailboxes, gmail creates a system where it's harder for would-be implementers to test compliance.
If tomorrow gmail creates a new type of mailbox, will there be a third set of rules to have your message delivered?
There are dozens of spam and security settings that admins can change in the Google Workspace console, presumably because different businesses have different requirements. So in practice, there's not just two sets of rules in gmail -- there are probably thousands or millions (however many combinations of settings are actually in use).
In my experience, email is an unreliable way to communicate any time-bounded critical information. When I want to be sure an email was transmitted on either side, the only reliable way to ensure this is to use a distinct channel to validate reception and confirm content.
That is, when some hotline tell me that they just sent and email with the information, I ensure they hold the line until I got the actual email and checked it delivers the relevant information to fulfill the intended process. And when I want to make sure an email was received, I call the person and ask to check, waiting until confirmation.
It’s not that much SMTP/IMAP per se as the whole ecosystem. People can legitimately get fatigue of "is it in my junk directory", "it might be relayed but only after the overloaded spam/junk analyzer accept it", or whatever can go wrong in the MUA, MSA, MTA, MX, MDA chain. And of course people can simply lie, and pretend the email was sent/received when they couldn’t bother less with the actual delivery status.
There are of course many cases where emails is fantastic.
Does it even matter when in reality it's more likely that this is intentional anti-competitive behavior by Google?
They once made all emails from my very reputable small German email provider (a company that has existed and provided email services long before Google existed) go into a black whole - not bounce them back or anything like that, mind you, their servers accepted them and made them disappear forever. I was in contact with the technicians then to get the problem fixed and they told me it's very difficult for them to even reach anyone at Google. It took them several days to get the problem fixed.
Of course, no one will ever be able to prove an intention behind these kind of "technical glitches." Nothing of significance ever happened when Google had large optics fiber connections with NSA installed illegally and claimed to have no knowledge of it, so certainly nothing will happen when small issues with interoperability occur and drive more people to Gmail.
On the other hand, by erroneously treating a SHOULD as a MUST, I would say that Google is the one who's not RFC-compliant
Google is rejecting it to ensure incoming messages aren't spam. SHOULD means "you should do this unless you have a really, really good reason not to." Do they have a good reason not to? It doesn't seem so, meaning Viva is in the wrong here.
No, SHOULD is defined in the RFC, not by colloquial usage. Google is on the wrong, regardless of their "safety" intent.
After all, linguistics is full with examples of words that are spelled the same, but have different meaning in different cultures. I'm glad the RFC spelled it out it for everyone.
The RFC says a SHOULD is to be treated like a MUST, but well-justified exceptions are allowed.
The RFC says a SHOULD is to be treated like a MUST, but well-justified exceptions are allowed.
[deleted]
What is the point of SHOULD then?
(No seriously, I’m asking; are there examples of where it’s actually different from a MUST)?
Also this reminds me of something I read somewhere a long time ago: when specifying requirements don’t bother with SHOULD. Either you want it or you don’t. Because if it’s not a requirement, some people won’t implement it.
I guess the one time it’s good is if you want an optional feature or are moving towards requiring it. In this case Google has decided it’s not an optional feature.
SHOULD generally means: some people might require it. implement it for best results
backward compatibility makes it hard to add MUST. using SHOULD is a good alternative
Typically, MUST means that if you don't do that then something will break at the protocol level.
SHOULD means that if you don't that, bad things are likely to happen, but it will not immediately break at the protocol level and during discussion in the IETF some people thought there could be valid reasons to violate the SHOULD.
Typically, IETF standards track RFCs consider the immediate effects of the protocol but often do not consider operational reality very well.
Sometimes operational reality is that a MUST gets violated because the standard is just wrong. Sometimes a SHOULD becomes required, etc.
Certainly for email, there is a lot you have to do to get your email accepted that is not spelled out in the RFCs.
MUST means omission is unacceptable. SHOULD means MUST unless you have a good, well-reasoned excuse.
“When jump getting over a wall, you SHOULD use three points of contact.”
For most cases you should use three points of contact. However, there may be other situations for example if someone is giving you a leg up, or you can pole vault, where another solution is preferred.
You assume that internet standards are prescriptivist; that the document describes how it is to be implemented. In practice it's often descriptivist, with the standards documents playing catch-up with how things are actually going in practice.
Anyway, in general you can expect that doing unusual but technically valid things with email headers will very often get your messages rejected or filtered as spam.
Standards are definitely prescriptive. But just like a medical prescription, it doesn’t ensure that actors in the wild will conform to what’s prescribed. People will not follow prescriptions for whatever reason, willingly or otherwise. It doesn’t mean the document wasn’t prescriptive.
For producers, ignoring a SHOULD is riskier because it shifts the burden to every consumer.
For consumers, ignoring a SHOULD mostly affects their own robustness.
But here Google seems to understand it as a MUST... maybe the scale of spam is enough to justify it. Users are stuck between two parties that expect the other to behave.
> maybe the scale of spam is enough to justify it.
This is 100 percent the case, and why these things are this way.
If you wanted to make email two point oh, I dont think it would look a lot like what we have today.
> This is 100 percent the case, and why these things are this way.
But gmail accepts emails without message-id on personal mailboxes apparently.
I think a mail 2.0 would be notify and pull based.... you notify a recipient's mail server that there's a message from <address> for them, then that server connects to the MX of record for the domain of <address> and retrieves <message-id> message.
Would this make mass emails and spam harder, absolutely. Would it be a huge burden for actual communications with people, not so much. From there actual white/black listing processes would work all that much better.
jmap is the communication between a mail client and shared directory/mail services on a server. It does not include server to server communications (that I am aware of) for sending mail to other users/servers.
Google interpreted it that way because it drives more people to use gmail.
The reason that European tech sucks is that people in Europe are open to such arguments. If an engineer in the US started talking about SHOULD vs MUST, some PM would just give them that "what the fuck did I just listen to" face, spend the next few minutes gently trying to convince them that the customer experience matters more than the spec, and if they fail, escalate and get the decision they want.
For example, why does Google handle this differently for consumer and enterprise accounts? Well it's Google so the answer could always just be "they are disorganized" but there's a good chance that in both cases, it was the pragmatic choice given the slightly different priorities of these types of customers.
Not my PM (in the US). My PM would try to avoid anything that is not absolutely necessary and therefore ask developers not to develop anything that isn't a MUST. I know that we like making fun of Europe for their alleged lack of innovation but this isn't a Europe thing.
Your PM most definitely would not tell you to skip a feature that is needed for your emails to be delivered to Gmail accounts. What a preposterous thing to lie about.
My take, as a postmaster for hosting company, who don't have any sympathy to gmail (that should be visible from my comments history):
Message-ID is absolutely MUST in production e-mails. You can send your test stuff without it, but real messages always have it. Not having Message-ID's causes lot of fun things. All somewhat competent software is capable to add Message-ID's, so lack of it is good indication of poorly made custom (usually spamming) solution.
Rspamd and spamassassin have missing MID check in their default rules, I am sure that most antispam software is same.
Why? If I'm writing a mail receiver, and I'm told there is some unique ID generated by the sender in a loosely specified way, the first thing I'm doing is ignoring that value forever. One lesson surely most everyone learns in CS is that unique identifiers are maybe unique to the system generating them, but to rely on foreign generated IDs being unique globally is a terrible idea that will break within the minute.
Exactly. Message-ID is not required.
An unrelated frustration of mine is that Message-ID really should not be overridden but SES for instance throws away your Message-ID and replaces it with another one :(
I would read this as a requirement for email to be 'legit' and not classified as spam.
Sure, you can send email with whatever headers you want, use weird combos, IP addresses, reply-to, and it might be still a technically valid email, but not something that should land in people's inboxes.
Also, a payment processor not testing their email on the most popular email provider in the world is quite ridiculous.
The reason it’s recommended is that it’s useful for detecting when an email you receive is already in your mailbox, so that you don’t accumulate duplicates. Otherwise one would have to compare the complete email, which probably no MUA does. Another reason is that replies can include a reference to the original message, so that it can be properly threaded by MUAs.
So these are mostly quality-of-life reasons, it’s not a reason to reject an email.
The only messages I receive without one are spam/phishing. I check because they're not recognised by notmuch, so I don't see them otherwise.
You SHOULD follow the wording of the RFC, you MUST follow Google's interpretation of the RFC.
That is the difference.
GMail SHOULD handle your messages, not MUST.
You're totally right. I've updated the blog to reflect this. Thank you!
Also email as a protocol (SMTP) predates RFC5322 by 25 years or so.
Avoid SHALL, SHOULD and all other crap, use Elon MUST.
SHALL has been interpreted/clarified by US courts as not being a fancy MUST or REQUIRED that many people were taught it to mean, but SHOULD still has it's purposes, e.g., to provide contractual flexibility in development, i.e., a MUST/REQUIRED requirement was more challenging or complicated and took up more time/resources than anticipated, so SHOULDs can be trimmed due to contingencies.
Another example may be a lightweight implementation of a spec in a limited and/or narrow environment, which remains technically compliant with full implementations of a spec but interaction with such a limited/narrow environment comes with awareness about such limitations.
[deleted]
Worked on an ESP. We had a couple of server software we used on low-level for sending. None of them would accept the message without a Message-ID. But even if you have a super-custom, SMTP-injecting service built, how can you ignore all of these bounces from a provider thats likeliest to be the major one you are sending to? Unthinkable. I would not like to have business with such a payment provider.
This is the one that gets me - sometimes you're forced to work with systems that do annoying things that you have to accommodate. It's annoying, but it's more important to do the thing that prevents your users from having issues than it is to be theoretically right about whether something's required by a standard.
I've dealt with many worse cases than this, where the systems I was integrating with were doing things that weren't even close to reasonable, but they had the market power so I sucked it up and dealt with it for the sake of my users. Maybe Google's wrong here, but how do you not just implement the solution anyway?
I doubt Google Workspace is going to be the major provider for European businesses
My pet peeve are services that go out of their way to include a text/plain alternative message part but send something useless, such as the message without the key link. One time I seriously ran into a service just send a short one-sentence note along the lines of "this is a plain text email" as the plain text part. If you don't want to support plain text, maybe just don't send the alternative part?
I find the ones that try to be cute the most frustrating because these appear on the new message notifications so I can't just delete them straight from the notification.
We'd love to share this exciting announcement but you'll a different email app.
Although I guess the argument will be that email clients should use AI to summarise the HTML into a plain text summary.
the best is when some put the same payload in the text/plain part as in the text/html part. yes. the html source. as text/plain.
So I'm wondering a bit here - I've seen an implementation where emails to send only have html versions, but as part of the sending process the html is run through a Lynx browser process with the -dump command to get the plain text, which is included as the text/plain part of the email.
Is there actual value to this? e.g. Is the output of Lynx's text dump better for plain-text email clients than whatever they'd display for html emails?
Some (old?) spam filters may be triggered by html only emails.
With fintech that surprises me not the slightest bit. Financial institutions are filled to the brim with unbelievably incompetent people. A large part of it is probably willful ignorance, too. It's often truly staggering that a financial company I interact with in day to day live is even able to exist. That's until I remember that all the others are just as incompetent.
"Major European Payment Processor" really just translates to "Major European Incompetence Center".
With a broad statement like this, I would usually just suggest this is inflammatory and surely overstated.
However, I've also worked at a financial institution which used core systems by Harland Financial Systems. Their "encryption" for data in transit from teller workstations to the core system was just a two byte XOR, and they sent the key at the beginning of the connection!
Was so unbelievable to be able to crack this in under a half-hour after noticing patterns in a PCAP. Wouldn't have believed it if I hadn't seen it with my own eyes.
That fraud was good enough for our regulators and theirs, so I have no doubt the industry is filled with rotten incompetence through and through.
There is plenty amount of incompetence in FAAMG. Notepad ....
Do Europe financial institutions have the same level of corruption as the USA? Such as a credit card company authorizing credit card transactions with incorrect expiration date to maximum profit, Bank of America? Or opening new accounts without consumer consent, Wells Fargo?
It's a broad question, but in many ways, very clearly yes, re. corruption of financial institutions, and to a far greater extent in many, albeit different ways.
Incompetence and corruption only slightly overlap in most cases, i.e., being competent at corruption is a very real thing. The incompetently corrupt, usually end up punished... and there are few and far between...as we all very well know.
The kind of schemes you mentioned are generally not going to be how "corruption" will manifest itself in European financial institutions, because although it is also difficult to speak in general across Europe since the EU has not yet subsumed democratic self-determination all across the continent yet, so there is wide variation; the competent corruption is largely in the form of money laundering and tax evasion, not lower level quantitative schemes that would quickly come to light because Europeans are also a lot more cognizant of money and value than Americans, so people are paying attention a lot more closely and will raise hell over a single cent, where Americans are known to have hundreds of dollars draining out of their pockets every month just alone on recurring payments for things they don't even use anymore and don't bother dealing with it.
What we all don't really seem to internalize as a human species, is the absolutely demonic type of pernicious nature of "banking", i.e., a kind of LotR, ring, that consumes you especially if you are weak... and human, or at least European civilization seems to frequently go through periods of immense weakness where things are going springily and everyone is dancing to the music the "bankers" are playing as they are pandering our pockets, and when they realize they could get away with that and all the pockets are plundered, they move on to plundering our homes, then our accounts, then they want to take our first born... "Banking" is like humanity's cocaine, the seemingly innocuous, feel good drug that will consume your soul if you do not rage and fight against that demon taking over aggressively. It's no coincidence that cocaine is so widely used by the most parasitic elements of European societies, especially in "banking"/finance in general.
> Who's in the right
I don't think either are. The payment processor should be sending it, but, at least according to the RFC, it is incorrect to reject an email that doesn't have it. I suspect the reason it is SHOULD, and not MUST is for backwards compatibility with software that predates the RFC that adds the message-id header.
Maybe there is a correlation between missing that header and being spam, but then it should go to the spam folder, not be outright rejected.
----------------------------
The experience with support is also similar to experiences I've had with support at many companies. I provide enough details that an engineer could probably easily fix the problem, but the support representative just dismisses it, and it is doubtful an engineer even hears about it.
Exactly. This minutiae is all so weird. Email as a formal specification does not work, and the industry as a whole has accepted that for decades now. It's not possible to filter spam from valid traffic without applying a truckload of heuristics and leveraging an ever growing set of auxiliary signals (SPF, DKIM, yada yada).
To wit: basically everything in this world is a "SHOULD", at best. The rules are a conversation.
"Email is tough", software development is tough, IT is tough, walking and talking at the same time is tough, mailing a letter is tough.
When orgs frame problems like this, it erodes trust in the message they try to convey. Email isn't a tough problem, but its a problem nobody wants to really deal with. Email is simple - its a text based protocol, that started out open, but now you need to add security to ensure your email is delivered.
I have some level of sympathy with Google here, which isn’t something I often say.
I recently switched from Gmail to Fastmail and by and large I’m happy with it. But I’ve been surprised by the amount of spam and (particularly) phishing emails I get in a regular basis. Google might be too strict in its filtering but it does serve a legitimate purpose.
Interesting that you mention this, as I also switched to Fastmail recently and got more spam than before, but after marking it as spam for some time it now died down I think. This may also be a symptom of changing providers, where the previous provider knew the kind of spam I tended to get from past years, while Fastmail needed some time to get up to speed.
Fingers crossed that the experience will be the same for you.
Fastmail seems to go through periods where they're a little slower to adjust to new spam techniques, and they do rely on users filtering somewhat. About twice a year a few will slip through, but if I report them as spam they soon stop.
I've been a happy customer otherwise for years, for what its worth.
I've considered this switch. You're saying that previously gmail was dropping the emails, or they were landing in spam?
Presumably they were in spam. But I rarely ever checked my spam folder to know with certainty.
The most damning thing about this is they didn't test their email infra w/ Google Workspaces. Imagine what else they didn't test.
yeah, because the whole world uses Google workspaces, right /s
That and MS Office are pretty darn popular. Not the whole world, but a very decent percentage of your users.
Maybe the whole thing was intentional, right at the footer of viva "Cloud services by Microsoft Azure" ; #1 I've never heard of viva before #2 I've never seen an azure logo at the footer of a website.
The first thing that comes to my mind is: how come viva.com is unable to send emails to google workspaces and nobody at viva.com noticed before ? For how long has this going on ?
The second thing is, what email software are they using ? If it was any relatively used software I would not expect this problem to arise (maybe it is some commond software but misconfigured).
Third, while the header is not mandatory, I usually read SHOULD as a "if you don't implement it prepare for possible problems". SHOULD is not MAY.
Fourth, they should be thankful that Google bounced the messages with some appropriate error explaining how to solve it. I have plenty of issues in the past with both Google and Microsoft where they accept the message for then sending to /dev/null
If that's how they handle email, I wouldn't want to see what they do with payment data.
10 percent of the effort in building software compatibility with open source file specifications is dealing with knowing the specifications. 90 percent of the effort is dealing with errors in generated files by less worthy software programs.
The RSS spec is one way. RSS readers do a fine job of interpreting files done the right way. Publishers don’t always do a good job with publishing error free RSS files. So RSS readers devs have to anticipate all sorts of errors and conduct error handling to ensure RSS items are properly handled.
This is why companies want to keep their file format proprietary. Other devs can really do damage to the ecosystem and ruin the experience
My personal fork of ttrss, from 2005, is a dodgy patchwork of fixes for badly formatted RSS. I can't imagine trying to host a service that deals with RSS feeds from random sites at scale.
One that always irks me to no end, is every time I see someone ham fisting csv handling by hand instead of using an established, well-sourced library. They almost always fail at commas or newlines in quoted text... It's one of the more annoying things.
Currently working on replacing a couple decades old system, and my csv output is using a library that isn't quoting all the strings that don't require quotes... so I'm forced to do it (for compatibility) with the other system this csv is going to. (sigh).
Maybe that's something to report to the "European System of Financial Supervision" or some other EU government agency.
I've never heard of them. Looks to be a company from Greece. That would explain their reply. Not exactly known for their tech.
Huh I've lived in Europe for most of my life and I've never heard of viva except as a poor name choice for Microsoft's corporate Facebook (yammer)
Most companies here use stripe on their website.
I vaguely remember hitting this message id issue in Google Workspace, and being able to work around it in mail routing configuration.
Saidly I don't remember the specifics, it was something along the lines of not all, but only specific routing features requiring it. Workspace settings are a moving target anyway, so the behavior probably changed more than once since.
I'm not saying it's a good idea to send emails without message id, but i'd also double-check that workspace configuration.
> For viva.com's engineering team, in case this reaches you: add a Message-ID header to your outgoing transactional emails.
Don't know what they're using for sending emails, but that's something that should be handled by their email service provider, unless they're hosting their own email servers.
Interestingly the MX record of via.com points to Google, but their verification emails could come from anywhere of course. The IP address in the log is also a Google IP, although that could also be the receiving IP.
It used to be said that the reason the Internet evolved so well was because of the basic principle of "be strict when you send, but tolerant when you receive". Clearly Google has forgotten this.
The specific bug is annoying, but that there's no way to report such a thing is an exact hallmark of our current corposphere.
Google's Postmaster Tools site has a "Report deliverabilty issue" link at the bottom left navigation column.
I borderline wanted to say "shibboleet" to the customer support agent, but I had serious doubts it's a relevant cultural reference. (https://xkcd.com/806/)
> Why this matters
Hello AI (Claude?)
[deleted]
literally everything is tough when comes to emails
Postel’s Law would put the onus on Google to be forgiving in what it receives. Unsure how you could safely use a sender-created Message-Id for anything anyway.
Following Postel's law results in the normalisation and proliferation of defective implementations. The actual standard becomes irrelevant, and new implementations have to be coded against the defective ones.
My opinion is that Postel's law should be approached in the same way that Linus Torvalds did CVS when designing Git. If in doubt about an implementation decision, consider what Postel's law would recommend, and then do the exact opposite.
That “law” if from a different time, before protocols like SMTP became adversarial. It assumed everyone was acting in good faith.
Yep. And even a world of perfect good faith, "forgiving in what you receive" has both costs and scaling problems - from researching what "spec" you'll need to design to, to customer service when the added complexity and permissiveness cause interesting stuff to happen.
Might want to consider Adyen, which should support IRIS, the Greek instant payment system.
Thank you for the recommendation! That I couldn't sign up using a form and I had to "talk to their team" was a turn-off for my (extremely extroverted) self.
That usually means you can't afford it unless you have people working for you that do the 'talk to the team' thing.
Typically I'm a DIY type who loves tinkering and building...
HOWEVER, I have learned the hard way to never apply that spirit to email.
In Europe you see this stuff all the time with old school "IT" (what old industrial companies call tech) people balking at the prices of commercial API-based senders and email marketing ESPs.
"Money to send emails in the cloud? HAH! Back at Siemens in 90s we were running millions of emails out of our servers just fine!"
Nobody understands that deliverability has gotten immensely harder these days, and trying to DIY it if its not your core business is just plain stupid. I would never in a million years try to roll my own email, it's nightmarish legacy cruft and footguns all the way down, in everything from IP/Domain Rep to something as simple as the HTML in the email templates themselves.
Microsoft Outlook and Gmail have the last word on everything in email, and their defacto duopoly (over B2B and consumer email respectively) means you play by the rules they set in 2008 and are too lazy to change or you don't get delivered. The protocol of email exists separately from the world of the actual inbox providers, which are locked down to insane degree given the security/spam concerns with email.
Google is at least less arbitrary than Microsoft. Microsoft will decide an email is spam today, and tomorrow the exact same email is perfectly fine. I think Google relies more and more on sending IP and domain reputation rather than content.
Google regularly sends legitimate email to my spam folder.
Microsoft regularly sends legitimate emails from Microsoft to my spam folder.
Deliverability to Microsoft famously took a dive a bit over a year ago due to random arbitrary failures within their infrastructure causing DMARC/DKIM problems which they clearly were having problems diagnosing.
Even with a six-figure email spend and weeks of troubleshooting the best response we could get from our mail provider was that they were having problems getting traction with Microsoft on the issue.
Worth mentioning is that there are several email umbrellas under Microsoft... including the newer office/365, the slightly older outlook.com hosting, the old corp hosting and hotmail and sub-properties... each with different rules and services to determine spam in inconsistent ways between them.
One of my main emails is still on a "free" outlook.com hosted with a personal domain that I never shifted to paid 365. I've also got an MTA server (mailu) of my own that I've been testing with... my own email under outlook.com is literally the only one of the MS systems I can't seem to deliver to, the rest work fine. Same for google.com for that matter... kinda wild.
This issue didn't seem to discriminate. I was seeing deliverability failures to Office 365 clients as well as consumer-facing brands like hotmail and msn
The problem is always e-mail itself. It is terrible standardised and hard to get "right".
Do you want to enable receiving email for viva.com? sign up for VibeCodedSAAS for E49.99/month
Just did! My mac mini got pwned though and I wish I didnt give it SMTP accesss… sigh
I just hope my OpenClaw skills registry doesnt have malware anymore. I sure trust my supply chain of vibecoded software!
Gripe only related to email in general: what annoys me to no end is that if my boss forwards me an email and asks me to reply to it (to everybody in the original email) then I have to type in or copy+paste all the addresses from the Fwd attachment (using Fastmail, but this problem exists everywhere). Instead, there should be a button to make that easy.
There's actually nothing that prevents that, if you craft the right headers you can reply to a thread you were not included in, and have it show up as a reply in the thread of common clients (tested Gmail and Outlook).
We added this feature at my $dayjob and I was quite surprised there is no authentication. But thinking about it, this is how mailing lists work (you aren't explicitly specified in "To:") so it makes sense you can do this.
Sudden realization that one of my American banks must be having email problems with this too because I use a Google custom email and recently got an in-app notification from my bank saying "we're unable to email you" (and a letter) yet my email works perfectly fine... switching to consumer gmail worked.
Google NOT following the spec is not surprising. SHOULD does not mean MUST and they are completely in the wrong here.
> Their support team's response to my detailed bug report: "your account has a verified email, so there's no problem."
Sadly I doubt their system is xkcd806 compatible ether.
This isn't an engineering problem, it's an ITIL problem. To be fair 99% of these complaints will be dealt with by the flow chart. Sadly people on the front line are either not knowledgable enough or not empowered enough to bust out of that straightjacket.
I usually reply with snark dialed up about 50% asking if anyone had actually read the original message. And include detail about how it is not, in fact, "ok"
The other day, I literally had trouble signing into a website... then I tried filling the contact us form, about the bug... only to have that fail... call in, have the person on the other end schedule my appointment, then almost drop the call without actually logging my bug report/complaint about the whole issue that had me calling in the first place.
SHIBBOLEET. I was seriously thinking of this when I contacted them :)
Email deliverability is the reason I gave up on email entirely for my side project and built on Telegram instead. Setting up SPF, DKIM, DMARC, warming up a domain, monitoring reputation, dealing with bounces and complaints... all of that just to maybe land in someone's inbox.
With Telegram you send a message via the Bot API and it arrives. 100% deliverability. No spam filters. No authentication chain. The message just shows up with a notification on their phone.
Obviously Telegram has its own limitations (smaller user base in the US, less formal). But for anything where you need reliable message delivery to people who opted in, messaging platforms have a massive advantage over email in 2026.
Unless you are running a more complex setup SPF, DKIM and DMARC really aren't that complicated. They are annoying and additional checkboxes you have to go trough that are hard to fully automate because they require access to DNS, but they are more busywork than difficult.
Domain and IP reputation and all the other quirks of deliverability are much more of a headache. DMARC is setup, test and done. But deliverability in praxis is something you cannot just test and can break at any time. The second worst are email providers that do whitelisting for email and require you to go through their process to even be allowed to send emails to their customers. The worst are providers that randomly decide to drop your emails without informing you or giving you a proper way to appeal as a small sender. If you're not a large email provider they have no problem just dropping you and the fault is on you because your service is the only one that is not working.
And then there are so many more intricacies of the ancient email protocol. Compatibility with horrendously outdated and misconfigured mail infrastructure is particular frustrating to me. I'm running a modern, properly configured, state of the art email server, but get ghosted by large email providers, yet have to deal with the broken mess, much bigger providers than myself are, which of course have no trouble delivering their broken spoofable email just because they are large enough.
I found it maddening that I couldn't whitelist a sender (MS Outlook web, Gmail); well I could, but they still blocked the messages.
In my case, it was reportedly (for MS) an IP associated with mine (same hosting provider) had previously been used to send spam.
My domain is decades old, never sent any spam, and I whitelisted it .. but nope, my host wasn't perfect.
This was some time ago now, but it looks like they've still not adopted proper whitelisting.
Most of that can be mitigated, or at least centrally managed, using an ESP like Mailgun or Sendgrid.
It is a pain in the ass though, coming from someone that had to dig their domain out of "low" reputation with Google Postmaster.
Some of us selfhost email, like me for 30+ years, and have 100% deliverability.
Until the platform owner bans for whatever reason and if communication by way of the platform was your only means of communication with your customer base, that's the platform owner having the power to destroy your business. No different that businesses that rely on the neverending goodwill of the mobile app store owners. One misstep and your business is gone with no recourse whatsoever. Protocols > Platforms. Always.
> Viva.com, one of Europe's largest payment processors, sends verification emails without a Message-ID header — a basic requirement of RFC 5322 since 2008. Google Workspace rejects them outright. Their support team's response to my detailed bug report: "your account has a verified email, so there's no problem."
Their emails do arrive tho? It was your email that didn't arrive? I find it unbelievable that a payment provider ignored customer complaining about no emails being delivered since it would breach their SLAs with their customers and their customers' customers would have complained. Especially since at the top you say Google says you got the verified email.
Dude, you may be liable for damages on this. This is an extremely serious allegation to be making in my opinion. I would delete this asap.
Edit: I think Ycombinator needs to realise they're liable for spreading this too. Holy crap, it's bad. They're lying through their teeth saying an email bounced but ended up in their logs. That's not now emails bounce is it? They bounce because it wasn't found. How was he able to verify his email if he didn't get the code?
Interesting, your take away is that Google is the one with the bug here?
If Gmail rejects emails from your domain it is up to you to fix it. Google is not going to change, and enough of your users will be interacting with people on Gmail that you have to fix it. It doesn't help that Google has been pushing people away from running their own email and into Google's services by ever tightening what it accepts over the years. More than one person has given up on their email server because it was a constant battle with Google, Microsoft, and company to not have important emails disappear into the void.
Their @gmail.com servers accept the messages (as said in the post) so it's not a problem for 99% of Google users either.
If you choose to host your email with Google, it's up to you to fix your email delivery settings (or find a better provider) for your domain.
An attempt no doubt to extenguish a standard google doesn't control
My takeaway is there is no bug. My takeaway is that his test email bounced because he didn't have the reputation Viva does. Emails are handled on a reputation basis, this is why we use email service providers like Sendgrid, Mailgun, Postmark, etc.
It always amazes me how people can read a blog post like this one that has a clear description of the problem with a log excerpts demonstrating the problem, and then people will confidently make up a completely different scenario that was not mentioned at all and blame the problem on that.
A log that clearly was from them and not the service provider. It amazes me you think you're so smart but haven't realised he doesn't have access to the logs you think he is showing.
Comments like this are why he's just landed himself with a major liability and I bet he'll be getting sued over this.
Pretty certain that you're wrong.
TFA shows an excerpt from the email log for his google workspace account, showing the bounce of email sent from viva.com.
Then, TFA states that he switched "the account" (his viva.com account) from using his GWorkspace address to a personal @gmail.com address, and asked viva to send another verification email. That one arrived.
At no point does TFA describe the author themselves sending a test email.
I've added a screenshot at the end of the blog post just to clarify that.
> I decided to dig into Google Workspace's Email Log Search to see what was happening on the receiving end.
It amazes me that you can read an article and draw the exact wrong conclusions
I wish I had your confidence in life
Please read the blog post you are making such strong claims about.
What that is liable? That is a very small claim.
I think that's a misunderstanding of the tale. Viva sent a "click here to verify your email" to OP. That email never arrived because Google rejected it for missing a header. OP tried to tell viva, but they don't wanna hear it because OP worked around it.
> My takeaway is that his test email bounced
What test email? I see no mention of a test email in the blog post. The mail that bounced was the one with the verification link from Viva.
So you think he had access to Viva's email servers to see the response? No, he clearly tested it himself and used his credentials to send it.
The log line is from Google Workspace which exposes it to its customers for incoming mail
Thank you! I added a screenshot of the Google Workspace Admin log screen... just becuase.
Yeah. I think email receiving is a game of exceptions… the email receivers (In the business world it’s essentially just MSFT and GOOG of course) answer to the addressees because they are the customer, and those customers will start to shriek if their inbox doesn’t receive “Important Messages.” But GOOG or MS have no leverage over the senders in this case so they just add an exception: “if IP range is just right and message fault ___ is present, fix message” (or otherwise allow)
Of course, they do have leverage over “marketing email” senders since they can block it and no one will complain, so those senders always have impeccable compliance with every year’s new “anti-spam standard.”
Apple is another major player in the email receiving game for consumers. And they are awful, by far the worst of all the big providers. They do not send dmarc reports and they make it very difficult to tell why they accept some email and not others.
If you read two paragraphs further than the Tl;Dr:
> To unblock myself, I switched to a personal @gmail.com address for the account. Gmail's own receiving infrastructure is apparently more lenient with messages, or perhaps routes them differently. The verification email came through.
1. Email didn’t arrive in his inbox of his Google workspace
2. He checked workspace email logs (with admin you can do this on gsuite)
3. It showed the intentional non-accept
4. Comprehending the problem, he switched to personal Gmail
5. The email arrived
6. He informed the sender of the original problem which he worked around
7. Sender is tech-illiterate and did not realize what the problem is. This is common with first line customer support so that happens.
The question to ask is whether you are literate in English or you skimmed too fast. Because I did a 30 s read of the article and got that.
It does seem unlikely that there are no customers on google workspace who have tried to use viva. I don't do payment processing, and my email is via zoho, so I've no idea how large either of those groups are.
I wonder what google workspace support said.
I suspect that Google going out of their way to make this required had a very reasonable and thought-out process, while the sender's omission was on oversight, so I haven't contacted Google Workspace support.
What's truly iffy is that GMail doesn't have the same strict requirements, and there's no way (at least that I found) to turn it off for my Google Workspace domain.
Wikipedia says Viva.com is a multi-billion dollar startup
It seems unlikely you're the first company using viva.com and using google workspace.
Clearly the problem here is that viva.com emails aren't arriving on your google workspace, despite what their support process says.
viva.com emails do arrive on other email providers, so seems unlikely to be problem with your viva.com account
It seems unlikely workplace blocks all viva.com emails otherwise more than you would have complained.
Whether that's viva's problem or google's problem is a separate problem.
The bigger issue here is that Europe depends way too much on the USA in so many areas. This is not good - you can be constantly blackmailed when you have people such as Trump in charge. I don't think the EU can be fixed, but at the same time I also think the less Europeans depend on outside factors (in particular the USA) the better. Canada kind of showed how to do it. Granted, Canada is also dependent on the USA in numerous ways and most of this is hard to fix (most Canadians live in the south aka close to the USA and trade is primarily done via the USA; security has also been largely outsourced onto the USA and so forth). The sooner people in Canada and Europe get moving away towards more independence from the USA, the better. And more cooperation would not harm either.
As a European (who spent 15 years in the US), I coudln't agree more. And while I agree, at the end of the day, I just want the better product for me.
This bug will not be fixed before the Environmental Impact Study is concluded on it.
> Viva.com's outgoing verification emails lack a Message-ID header, a requirement that has been part of the Internet Message Format specification (RFC 5322) since 2008
> ...
> `Message-ID` is one of the most basic required headers in email.
Section 3.6. of the RFC in question (https://www.rfc-editor.org/rfc/rfc5322.html) says:
and in section 3.6.4: That says SHOULD, not MUST, so how is it a requirement?I know you're looking for "pedant points" but the specification generally take a backseat to implementation. If Message-ID is expected out here where the rubber meets the road, then you are the squeaky wheel in this scenario for not including it.
SHOULD is a requirement. It means that you have to do it unless you know some specific reason that the requirement doesn't apply in your case. "I don't want to" is not a valid excuse, "I don't see a reason to" isn't either.
IIRC this particular rule is a SHOULD because MUAs often send messages without a Message-ID to their submission server, and the submission server adds one if necessary. https://www.rfc-editor.org/rfc/rfc6409.html#section-8.3 The SHOULD lets those messages be valid. Low-entropy devices that can't generate a good random ID are rare these days, but old devices remain in service, so the workaround is IMO justified.
> "I don't want to" is not a valid excuse
"The client SHOULD but didn't" isn't a valid excuse for the server to reject a client either.
You can do it anyway, you might even have good reasons for it, but you sure don't get to point at the RFC and call the client broken.
As someone who does systems engineering, the only valid requirements include the word "shall".
The official definition of SHOULD per RFC2119:
Not sure how the people at Google interpreted this about the message-idYou can argue that you not obligated to use message-id but if you don't use it you should blame only yourself that your messages are not accepted. In requiring message-id I would side with google (though in general I think they anti-spam is too aggressive and lacks ways to report false positives). Full RFC compliance (as in not only MUST but also SHOULD unless you have a very good reason) is the easiest part of making sure your emails will be delivered.
> if you don't use it you should blame only yourself that your messages are not accepted
I think it's a gray area
- If the receiver declines your message because "Message-id" is required - then I blame the receiver; because that's not true
- If the receiver declines your message because "most systems do include it, and it's lack of presence is highly correlated with spam email", then it's on the sender
Admittedly, the end result is the same.
I think it's the latter. But, in either case, you're right in that you get the same result.
Now, let's assume that if it is the latter (it's spam related), and Google were to accept the message, but then internally bin the message, it would be worse. At least in this case, they are bouncing the message. Because of this, the sender is at least aware that the message wasn't delivered.
Also, the author was able to get their mail delivered to a personal gmail.com address. The issue was with a Google Workspace custom email domain. This further makes me think of this as a security/spam related issue. Google is clearly capable of processing the message without a Message-id, they are just refusing for business customers.
My takeaway is that I think that Google is doing the least-wrong thing. And by being explicit in how they are handling it, it at least made the debugging for the author possible.
Also note: in a quick reading of RFC5321 (SMTP), rejecting messages for "policy reasons" is an acceptable outcome. I'm not sure if it applies completely here. The author should probably also be taking into account RFC5321 (SMTP) instead of just 5322 (message format).
> Also, the author was able to get their mail delivered to a personal gmail.com address. The issue was with a Google Workspace custom email domain. This further makes me think of this as a security/spam related issue. Google is clearly capable of processing the message without a Message-id, they are just refusing for business customers.
That's the annoying part to me.
An email is an email. By applying different rules for rejection on different mailboxes, gmail creates a system where it's harder for would-be implementers to test compliance.
If tomorrow gmail creates a new type of mailbox, will there be a third set of rules to have your message delivered?
There are dozens of spam and security settings that admins can change in the Google Workspace console, presumably because different businesses have different requirements. So in practice, there's not just two sets of rules in gmail -- there are probably thousands or millions (however many combinations of settings are actually in use).
In my experience, email is an unreliable way to communicate any time-bounded critical information. When I want to be sure an email was transmitted on either side, the only reliable way to ensure this is to use a distinct channel to validate reception and confirm content.
That is, when some hotline tell me that they just sent and email with the information, I ensure they hold the line until I got the actual email and checked it delivers the relevant information to fulfill the intended process. And when I want to make sure an email was received, I call the person and ask to check, waiting until confirmation.
It’s not that much SMTP/IMAP per se as the whole ecosystem. People can legitimately get fatigue of "is it in my junk directory", "it might be relayed but only after the overloaded spam/junk analyzer accept it", or whatever can go wrong in the MUA, MSA, MTA, MX, MDA chain. And of course people can simply lie, and pretend the email was sent/received when they couldn’t bother less with the actual delivery status.
There are of course many cases where emails is fantastic.
Does it even matter when in reality it's more likely that this is intentional anti-competitive behavior by Google?
They once made all emails from my very reputable small German email provider (a company that has existed and provided email services long before Google existed) go into a black whole - not bounce them back or anything like that, mind you, their servers accepted them and made them disappear forever. I was in contact with the technicians then to get the problem fixed and they told me it's very difficult for them to even reach anyone at Google. It took them several days to get the problem fixed.
Of course, no one will ever be able to prove an intention behind these kind of "technical glitches." Nothing of significance ever happened when Google had large optics fiber connections with NSA installed illegally and claimed to have no knowledge of it, so certainly nothing will happen when small issues with interoperability occur and drive more people to Gmail.
On the other hand, by erroneously treating a SHOULD as a MUST, I would say that Google is the one who's not RFC-compliant
Google is rejecting it to ensure incoming messages aren't spam. SHOULD means "you should do this unless you have a really, really good reason not to." Do they have a good reason not to? It doesn't seem so, meaning Viva is in the wrong here.
No, SHOULD is defined in the RFC, not by colloquial usage. Google is on the wrong, regardless of their "safety" intent.
After all, linguistics is full with examples of words that are spelled the same, but have different meaning in different cultures. I'm glad the RFC spelled it out it for everyone.
The RFC says a SHOULD is to be treated like a MUST, but well-justified exceptions are allowed.
The RFC says a SHOULD is to be treated like a MUST, but well-justified exceptions are allowed.
What is the point of SHOULD then?
(No seriously, I’m asking; are there examples of where it’s actually different from a MUST)?
Also this reminds me of something I read somewhere a long time ago: when specifying requirements don’t bother with SHOULD. Either you want it or you don’t. Because if it’s not a requirement, some people won’t implement it.
I guess the one time it’s good is if you want an optional feature or are moving towards requiring it. In this case Google has decided it’s not an optional feature.
SHOULD generally means: some people might require it. implement it for best results
backward compatibility makes it hard to add MUST. using SHOULD is a good alternative
Typically, MUST means that if you don't do that then something will break at the protocol level.
SHOULD means that if you don't that, bad things are likely to happen, but it will not immediately break at the protocol level and during discussion in the IETF some people thought there could be valid reasons to violate the SHOULD.
Typically, IETF standards track RFCs consider the immediate effects of the protocol but often do not consider operational reality very well.
Sometimes operational reality is that a MUST gets violated because the standard is just wrong. Sometimes a SHOULD becomes required, etc.
Certainly for email, there is a lot you have to do to get your email accepted that is not spelled out in the RFCs.
MUST means omission is unacceptable. SHOULD means MUST unless you have a good, well-reasoned excuse.
“When jump getting over a wall, you SHOULD use three points of contact.”
For most cases you should use three points of contact. However, there may be other situations for example if someone is giving you a leg up, or you can pole vault, where another solution is preferred.
You assume that internet standards are prescriptivist; that the document describes how it is to be implemented. In practice it's often descriptivist, with the standards documents playing catch-up with how things are actually going in practice.
Anyway, in general you can expect that doing unusual but technically valid things with email headers will very often get your messages rejected or filtered as spam.
Standards are definitely prescriptive. But just like a medical prescription, it doesn’t ensure that actors in the wild will conform to what’s prescribed. People will not follow prescriptions for whatever reason, willingly or otherwise. It doesn’t mean the document wasn’t prescriptive.
For producers, ignoring a SHOULD is riskier because it shifts the burden to every consumer.
For consumers, ignoring a SHOULD mostly affects their own robustness.
But here Google seems to understand it as a MUST... maybe the scale of spam is enough to justify it. Users are stuck between two parties that expect the other to behave.
> maybe the scale of spam is enough to justify it.
This is 100 percent the case, and why these things are this way.
If you wanted to make email two point oh, I dont think it would look a lot like what we have today.
> This is 100 percent the case, and why these things are this way.
But gmail accepts emails without message-id on personal mailboxes apparently.
I think a mail 2.0 would be notify and pull based.... you notify a recipient's mail server that there's a message from <address> for them, then that server connects to the MX of record for the domain of <address> and retrieves <message-id> message.
Would this make mass emails and spam harder, absolutely. Would it be a huge burden for actual communications with people, not so much. From there actual white/black listing processes would work all that much better.
https://jmap.io
jmap is the communication between a mail client and shared directory/mail services on a server. It does not include server to server communications (that I am aware of) for sending mail to other users/servers.
Google interpreted it that way because it drives more people to use gmail.
The reason that European tech sucks is that people in Europe are open to such arguments. If an engineer in the US started talking about SHOULD vs MUST, some PM would just give them that "what the fuck did I just listen to" face, spend the next few minutes gently trying to convince them that the customer experience matters more than the spec, and if they fail, escalate and get the decision they want.
For example, why does Google handle this differently for consumer and enterprise accounts? Well it's Google so the answer could always just be "they are disorganized" but there's a good chance that in both cases, it was the pragmatic choice given the slightly different priorities of these types of customers.
Not my PM (in the US). My PM would try to avoid anything that is not absolutely necessary and therefore ask developers not to develop anything that isn't a MUST. I know that we like making fun of Europe for their alleged lack of innovation but this isn't a Europe thing.
Your PM most definitely would not tell you to skip a feature that is needed for your emails to be delivered to Gmail accounts. What a preposterous thing to lie about.
My take, as a postmaster for hosting company, who don't have any sympathy to gmail (that should be visible from my comments history): Message-ID is absolutely MUST in production e-mails. You can send your test stuff without it, but real messages always have it. Not having Message-ID's causes lot of fun things. All somewhat competent software is capable to add Message-ID's, so lack of it is good indication of poorly made custom (usually spamming) solution.
Rspamd and spamassassin have missing MID check in their default rules, I am sure that most antispam software is same.
Why? If I'm writing a mail receiver, and I'm told there is some unique ID generated by the sender in a loosely specified way, the first thing I'm doing is ignoring that value forever. One lesson surely most everyone learns in CS is that unique identifiers are maybe unique to the system generating them, but to rely on foreign generated IDs being unique globally is a terrible idea that will break within the minute.
Exactly. Message-ID is not required.
An unrelated frustration of mine is that Message-ID really should not be overridden but SES for instance throws away your Message-ID and replaces it with another one :(
I would read this as a requirement for email to be 'legit' and not classified as spam.
Sure, you can send email with whatever headers you want, use weird combos, IP addresses, reply-to, and it might be still a technically valid email, but not something that should land in people's inboxes.
Also, a payment processor not testing their email on the most popular email provider in the world is quite ridiculous.
The reason it’s recommended is that it’s useful for detecting when an email you receive is already in your mailbox, so that you don’t accumulate duplicates. Otherwise one would have to compare the complete email, which probably no MUA does. Another reason is that replies can include a reference to the original message, so that it can be properly threaded by MUAs.
So these are mostly quality-of-life reasons, it’s not a reason to reject an email.
The only messages I receive without one are spam/phishing. I check because they're not recognised by notmuch, so I don't see them otherwise.
You SHOULD follow the wording of the RFC, you MUST follow Google's interpretation of the RFC.
That is the difference.
GMail SHOULD handle your messages, not MUST.
You're totally right. I've updated the blog to reflect this. Thank you!
Also email as a protocol (SMTP) predates RFC5322 by 25 years or so.
Avoid SHALL, SHOULD and all other crap, use Elon MUST.
SHALL has been interpreted/clarified by US courts as not being a fancy MUST or REQUIRED that many people were taught it to mean, but SHOULD still has it's purposes, e.g., to provide contractual flexibility in development, i.e., a MUST/REQUIRED requirement was more challenging or complicated and took up more time/resources than anticipated, so SHOULDs can be trimmed due to contingencies.
Another example may be a lightweight implementation of a spec in a limited and/or narrow environment, which remains technically compliant with full implementations of a spec but interaction with such a limited/narrow environment comes with awareness about such limitations.
Worked on an ESP. We had a couple of server software we used on low-level for sending. None of them would accept the message without a Message-ID. But even if you have a super-custom, SMTP-injecting service built, how can you ignore all of these bounces from a provider thats likeliest to be the major one you are sending to? Unthinkable. I would not like to have business with such a payment provider.
This is the one that gets me - sometimes you're forced to work with systems that do annoying things that you have to accommodate. It's annoying, but it's more important to do the thing that prevents your users from having issues than it is to be theoretically right about whether something's required by a standard.
I've dealt with many worse cases than this, where the systems I was integrating with were doing things that weren't even close to reasonable, but they had the market power so I sucked it up and dealt with it for the sake of my users. Maybe Google's wrong here, but how do you not just implement the solution anyway?
I doubt Google Workspace is going to be the major provider for European businesses
My pet peeve are services that go out of their way to include a text/plain alternative message part but send something useless, such as the message without the key link. One time I seriously ran into a service just send a short one-sentence note along the lines of "this is a plain text email" as the plain text part. If you don't want to support plain text, maybe just don't send the alternative part?
I find the ones that try to be cute the most frustrating because these appear on the new message notifications so I can't just delete them straight from the notification.
We'd love to share this exciting announcement but you'll a different email app.
Although I guess the argument will be that email clients should use AI to summarise the HTML into a plain text summary.
the best is when some put the same payload in the text/plain part as in the text/html part. yes. the html source. as text/plain.
So I'm wondering a bit here - I've seen an implementation where emails to send only have html versions, but as part of the sending process the html is run through a Lynx browser process with the -dump command to get the plain text, which is included as the text/plain part of the email.
Is there actual value to this? e.g. Is the output of Lynx's text dump better for plain-text email clients than whatever they'd display for html emails?
Some (old?) spam filters may be triggered by html only emails.
With fintech that surprises me not the slightest bit. Financial institutions are filled to the brim with unbelievably incompetent people. A large part of it is probably willful ignorance, too. It's often truly staggering that a financial company I interact with in day to day live is even able to exist. That's until I remember that all the others are just as incompetent.
"Major European Payment Processor" really just translates to "Major European Incompetence Center".
With a broad statement like this, I would usually just suggest this is inflammatory and surely overstated.
However, I've also worked at a financial institution which used core systems by Harland Financial Systems. Their "encryption" for data in transit from teller workstations to the core system was just a two byte XOR, and they sent the key at the beginning of the connection!
Was so unbelievable to be able to crack this in under a half-hour after noticing patterns in a PCAP. Wouldn't have believed it if I hadn't seen it with my own eyes.
That fraud was good enough for our regulators and theirs, so I have no doubt the industry is filled with rotten incompetence through and through.
There is plenty amount of incompetence in FAAMG. Notepad ....
Do Europe financial institutions have the same level of corruption as the USA? Such as a credit card company authorizing credit card transactions with incorrect expiration date to maximum profit, Bank of America? Or opening new accounts without consumer consent, Wells Fargo?
It's a broad question, but in many ways, very clearly yes, re. corruption of financial institutions, and to a far greater extent in many, albeit different ways.
Incompetence and corruption only slightly overlap in most cases, i.e., being competent at corruption is a very real thing. The incompetently corrupt, usually end up punished... and there are few and far between...as we all very well know.
The kind of schemes you mentioned are generally not going to be how "corruption" will manifest itself in European financial institutions, because although it is also difficult to speak in general across Europe since the EU has not yet subsumed democratic self-determination all across the continent yet, so there is wide variation; the competent corruption is largely in the form of money laundering and tax evasion, not lower level quantitative schemes that would quickly come to light because Europeans are also a lot more cognizant of money and value than Americans, so people are paying attention a lot more closely and will raise hell over a single cent, where Americans are known to have hundreds of dollars draining out of their pockets every month just alone on recurring payments for things they don't even use anymore and don't bother dealing with it.
What we all don't really seem to internalize as a human species, is the absolutely demonic type of pernicious nature of "banking", i.e., a kind of LotR, ring, that consumes you especially if you are weak... and human, or at least European civilization seems to frequently go through periods of immense weakness where things are going springily and everyone is dancing to the music the "bankers" are playing as they are pandering our pockets, and when they realize they could get away with that and all the pockets are plundered, they move on to plundering our homes, then our accounts, then they want to take our first born... "Banking" is like humanity's cocaine, the seemingly innocuous, feel good drug that will consume your soul if you do not rage and fight against that demon taking over aggressively. It's no coincidence that cocaine is so widely used by the most parasitic elements of European societies, especially in "banking"/finance in general.
> Who's in the right
I don't think either are. The payment processor should be sending it, but, at least according to the RFC, it is incorrect to reject an email that doesn't have it. I suspect the reason it is SHOULD, and not MUST is for backwards compatibility with software that predates the RFC that adds the message-id header.
Maybe there is a correlation between missing that header and being spam, but then it should go to the spam folder, not be outright rejected.
----------------------------
The experience with support is also similar to experiences I've had with support at many companies. I provide enough details that an engineer could probably easily fix the problem, but the support representative just dismisses it, and it is doubtful an engineer even hears about it.
Exactly. This minutiae is all so weird. Email as a formal specification does not work, and the industry as a whole has accepted that for decades now. It's not possible to filter spam from valid traffic without applying a truckload of heuristics and leveraging an ever growing set of auxiliary signals (SPF, DKIM, yada yada).
To wit: basically everything in this world is a "SHOULD", at best. The rules are a conversation.
"Email is tough", software development is tough, IT is tough, walking and talking at the same time is tough, mailing a letter is tough.
When orgs frame problems like this, it erodes trust in the message they try to convey. Email isn't a tough problem, but its a problem nobody wants to really deal with. Email is simple - its a text based protocol, that started out open, but now you need to add security to ensure your email is delivered.
I have some level of sympathy with Google here, which isn’t something I often say.
I recently switched from Gmail to Fastmail and by and large I’m happy with it. But I’ve been surprised by the amount of spam and (particularly) phishing emails I get in a regular basis. Google might be too strict in its filtering but it does serve a legitimate purpose.
Interesting that you mention this, as I also switched to Fastmail recently and got more spam than before, but after marking it as spam for some time it now died down I think. This may also be a symptom of changing providers, where the previous provider knew the kind of spam I tended to get from past years, while Fastmail needed some time to get up to speed.
Fingers crossed that the experience will be the same for you.
Fastmail seems to go through periods where they're a little slower to adjust to new spam techniques, and they do rely on users filtering somewhat. About twice a year a few will slip through, but if I report them as spam they soon stop.
I've been a happy customer otherwise for years, for what its worth.
I've considered this switch. You're saying that previously gmail was dropping the emails, or they were landing in spam?
Presumably they were in spam. But I rarely ever checked my spam folder to know with certainty.
The most damning thing about this is they didn't test their email infra w/ Google Workspaces. Imagine what else they didn't test.
yeah, because the whole world uses Google workspaces, right /s
That and MS Office are pretty darn popular. Not the whole world, but a very decent percentage of your users.
Maybe the whole thing was intentional, right at the footer of viva "Cloud services by Microsoft Azure" ; #1 I've never heard of viva before #2 I've never seen an azure logo at the footer of a website.
The first thing that comes to my mind is: how come viva.com is unable to send emails to google workspaces and nobody at viva.com noticed before ? For how long has this going on ?
The second thing is, what email software are they using ? If it was any relatively used software I would not expect this problem to arise (maybe it is some commond software but misconfigured).
Third, while the header is not mandatory, I usually read SHOULD as a "if you don't implement it prepare for possible problems". SHOULD is not MAY.
Fourth, they should be thankful that Google bounced the messages with some appropriate error explaining how to solve it. I have plenty of issues in the past with both Google and Microsoft where they accept the message for then sending to /dev/null
If that's how they handle email, I wouldn't want to see what they do with payment data.
10 percent of the effort in building software compatibility with open source file specifications is dealing with knowing the specifications. 90 percent of the effort is dealing with errors in generated files by less worthy software programs.
The RSS spec is one way. RSS readers do a fine job of interpreting files done the right way. Publishers don’t always do a good job with publishing error free RSS files. So RSS readers devs have to anticipate all sorts of errors and conduct error handling to ensure RSS items are properly handled.
This is why companies want to keep their file format proprietary. Other devs can really do damage to the ecosystem and ruin the experience
My personal fork of ttrss, from 2005, is a dodgy patchwork of fixes for badly formatted RSS. I can't imagine trying to host a service that deals with RSS feeds from random sites at scale.
One that always irks me to no end, is every time I see someone ham fisting csv handling by hand instead of using an established, well-sourced library. They almost always fail at commas or newlines in quoted text... It's one of the more annoying things.
Currently working on replacing a couple decades old system, and my csv output is using a library that isn't quoting all the strings that don't require quotes... so I'm forced to do it (for compatibility) with the other system this csv is going to. (sigh).
Maybe that's something to report to the "European System of Financial Supervision" or some other EU government agency.
They even have a Whistleblowing link at the bottom of their website: https://www.bankingsupervision.europa.eu/about/esfs/html/ind...
I've never heard of them. Looks to be a company from Greece. That would explain their reply. Not exactly known for their tech.
Huh I've lived in Europe for most of my life and I've never heard of viva except as a poor name choice for Microsoft's corporate Facebook (yammer)
Most companies here use stripe on their website.
I vaguely remember hitting this message id issue in Google Workspace, and being able to work around it in mail routing configuration.
Saidly I don't remember the specifics, it was something along the lines of not all, but only specific routing features requiring it. Workspace settings are a moving target anyway, so the behavior probably changed more than once since.
I'm not saying it's a good idea to send emails without message id, but i'd also double-check that workspace configuration.
> For viva.com's engineering team, in case this reaches you: add a Message-ID header to your outgoing transactional emails.
Don't know what they're using for sending emails, but that's something that should be handled by their email service provider, unless they're hosting their own email servers.
Interestingly the MX record of via.com points to Google, but their verification emails could come from anywhere of course. The IP address in the log is also a Google IP, although that could also be the receiving IP.
It used to be said that the reason the Internet evolved so well was because of the basic principle of "be strict when you send, but tolerant when you receive". Clearly Google has forgotten this.
The specific bug is annoying, but that there's no way to report such a thing is an exact hallmark of our current corposphere.
Google's Postmaster Tools site has a "Report deliverabilty issue" link at the bottom left navigation column.
https://postmaster.google.com/v2/sender_compliance
I borderline wanted to say "shibboleet" to the customer support agent, but I had serious doubts it's a relevant cultural reference. (https://xkcd.com/806/)
> Why this matters
Hello AI (Claude?)
literally everything is tough when comes to emails
Postel’s Law would put the onus on Google to be forgiving in what it receives. Unsure how you could safely use a sender-created Message-Id for anything anyway.
Following Postel's law results in the normalisation and proliferation of defective implementations. The actual standard becomes irrelevant, and new implementations have to be coded against the defective ones.
My opinion is that Postel's law should be approached in the same way that Linus Torvalds did CVS when designing Git. If in doubt about an implementation decision, consider what Postel's law would recommend, and then do the exact opposite.
That “law” if from a different time, before protocols like SMTP became adversarial. It assumed everyone was acting in good faith.
Yep. And even a world of perfect good faith, "forgiving in what you receive" has both costs and scaling problems - from researching what "spec" you'll need to design to, to customer service when the added complexity and permissiveness cause interesting stuff to happen.
Might want to consider Adyen, which should support IRIS, the Greek instant payment system.
Thank you for the recommendation! That I couldn't sign up using a form and I had to "talk to their team" was a turn-off for my (extremely extroverted) self.
That usually means you can't afford it unless you have people working for you that do the 'talk to the team' thing.
Typically I'm a DIY type who loves tinkering and building...
HOWEVER, I have learned the hard way to never apply that spirit to email.
In Europe you see this stuff all the time with old school "IT" (what old industrial companies call tech) people balking at the prices of commercial API-based senders and email marketing ESPs.
"Money to send emails in the cloud? HAH! Back at Siemens in 90s we were running millions of emails out of our servers just fine!"
Nobody understands that deliverability has gotten immensely harder these days, and trying to DIY it if its not your core business is just plain stupid. I would never in a million years try to roll my own email, it's nightmarish legacy cruft and footguns all the way down, in everything from IP/Domain Rep to something as simple as the HTML in the email templates themselves.
Microsoft Outlook and Gmail have the last word on everything in email, and their defacto duopoly (over B2B and consumer email respectively) means you play by the rules they set in 2008 and are too lazy to change or you don't get delivered. The protocol of email exists separately from the world of the actual inbox providers, which are locked down to insane degree given the security/spam concerns with email.
Google is at least less arbitrary than Microsoft. Microsoft will decide an email is spam today, and tomorrow the exact same email is perfectly fine. I think Google relies more and more on sending IP and domain reputation rather than content.
Google regularly sends legitimate email to my spam folder.
Microsoft regularly sends legitimate emails from Microsoft to my spam folder.
Deliverability to Microsoft famously took a dive a bit over a year ago due to random arbitrary failures within their infrastructure causing DMARC/DKIM problems which they clearly were having problems diagnosing.
Even with a six-figure email spend and weeks of troubleshooting the best response we could get from our mail provider was that they were having problems getting traction with Microsoft on the issue.
Worth mentioning is that there are several email umbrellas under Microsoft... including the newer office/365, the slightly older outlook.com hosting, the old corp hosting and hotmail and sub-properties... each with different rules and services to determine spam in inconsistent ways between them.
One of my main emails is still on a "free" outlook.com hosted with a personal domain that I never shifted to paid 365. I've also got an MTA server (mailu) of my own that I've been testing with... my own email under outlook.com is literally the only one of the MS systems I can't seem to deliver to, the rest work fine. Same for google.com for that matter... kinda wild.
This issue didn't seem to discriminate. I was seeing deliverability failures to Office 365 clients as well as consumer-facing brands like hotmail and msn
The problem is always e-mail itself. It is terrible standardised and hard to get "right".
Do you want to enable receiving email for viva.com? sign up for VibeCodedSAAS for E49.99/month
Just did! My mac mini got pwned though and I wish I didnt give it SMTP accesss… sigh
I just hope my OpenClaw skills registry doesnt have malware anymore. I sure trust my supply chain of vibecoded software!
Gripe only related to email in general: what annoys me to no end is that if my boss forwards me an email and asks me to reply to it (to everybody in the original email) then I have to type in or copy+paste all the addresses from the Fwd attachment (using Fastmail, but this problem exists everywhere). Instead, there should be a button to make that easy.
There's actually nothing that prevents that, if you craft the right headers you can reply to a thread you were not included in, and have it show up as a reply in the thread of common clients (tested Gmail and Outlook).
We added this feature at my $dayjob and I was quite surprised there is no authentication. But thinking about it, this is how mailing lists work (you aren't explicitly specified in "To:") so it makes sense you can do this.
Sudden realization that one of my American banks must be having email problems with this too because I use a Google custom email and recently got an in-app notification from my bank saying "we're unable to email you" (and a letter) yet my email works perfectly fine... switching to consumer gmail worked.
Google NOT following the spec is not surprising. SHOULD does not mean MUST and they are completely in the wrong here.
> Their support team's response to my detailed bug report: "your account has a verified email, so there's no problem."
Sadly I doubt their system is xkcd806 compatible ether.
This isn't an engineering problem, it's an ITIL problem. To be fair 99% of these complaints will be dealt with by the flow chart. Sadly people on the front line are either not knowledgable enough or not empowered enough to bust out of that straightjacket.
I usually reply with snark dialed up about 50% asking if anyone had actually read the original message. And include detail about how it is not, in fact, "ok"
The other day, I literally had trouble signing into a website... then I tried filling the contact us form, about the bug... only to have that fail... call in, have the person on the other end schedule my appointment, then almost drop the call without actually logging my bug report/complaint about the whole issue that had me calling in the first place.
SHIBBOLEET. I was seriously thinking of this when I contacted them :)
Email deliverability is the reason I gave up on email entirely for my side project and built on Telegram instead. Setting up SPF, DKIM, DMARC, warming up a domain, monitoring reputation, dealing with bounces and complaints... all of that just to maybe land in someone's inbox.
With Telegram you send a message via the Bot API and it arrives. 100% deliverability. No spam filters. No authentication chain. The message just shows up with a notification on their phone.
Obviously Telegram has its own limitations (smaller user base in the US, less formal). But for anything where you need reliable message delivery to people who opted in, messaging platforms have a massive advantage over email in 2026.
Unless you are running a more complex setup SPF, DKIM and DMARC really aren't that complicated. They are annoying and additional checkboxes you have to go trough that are hard to fully automate because they require access to DNS, but they are more busywork than difficult.
Domain and IP reputation and all the other quirks of deliverability are much more of a headache. DMARC is setup, test and done. But deliverability in praxis is something you cannot just test and can break at any time. The second worst are email providers that do whitelisting for email and require you to go through their process to even be allowed to send emails to their customers. The worst are providers that randomly decide to drop your emails without informing you or giving you a proper way to appeal as a small sender. If you're not a large email provider they have no problem just dropping you and the fault is on you because your service is the only one that is not working.
And then there are so many more intricacies of the ancient email protocol. Compatibility with horrendously outdated and misconfigured mail infrastructure is particular frustrating to me. I'm running a modern, properly configured, state of the art email server, but get ghosted by large email providers, yet have to deal with the broken mess, much bigger providers than myself are, which of course have no trouble delivering their broken spoofable email just because they are large enough.
I found it maddening that I couldn't whitelist a sender (MS Outlook web, Gmail); well I could, but they still blocked the messages.
In my case, it was reportedly (for MS) an IP associated with mine (same hosting provider) had previously been used to send spam.
My domain is decades old, never sent any spam, and I whitelisted it .. but nope, my host wasn't perfect.
This was some time ago now, but it looks like they've still not adopted proper whitelisting.
Most of that can be mitigated, or at least centrally managed, using an ESP like Mailgun or Sendgrid.
It is a pain in the ass though, coming from someone that had to dig their domain out of "low" reputation with Google Postmaster.
Some of us selfhost email, like me for 30+ years, and have 100% deliverability.
Until the platform owner bans for whatever reason and if communication by way of the platform was your only means of communication with your customer base, that's the platform owner having the power to destroy your business. No different that businesses that rely on the neverending goodwill of the mobile app store owners. One misstep and your business is gone with no recourse whatsoever. Protocols > Platforms. Always.
> Viva.com, one of Europe's largest payment processors, sends verification emails without a Message-ID header — a basic requirement of RFC 5322 since 2008. Google Workspace rejects them outright. Their support team's response to my detailed bug report: "your account has a verified email, so there's no problem."
Their emails do arrive tho? It was your email that didn't arrive? I find it unbelievable that a payment provider ignored customer complaining about no emails being delivered since it would breach their SLAs with their customers and their customers' customers would have complained. Especially since at the top you say Google says you got the verified email.
Dude, you may be liable for damages on this. This is an extremely serious allegation to be making in my opinion. I would delete this asap.
Edit: I think Ycombinator needs to realise they're liable for spreading this too. Holy crap, it's bad. They're lying through their teeth saying an email bounced but ended up in their logs. That's not now emails bounce is it? They bounce because it wasn't found. How was he able to verify his email if he didn't get the code?
Interesting, your take away is that Google is the one with the bug here?
If Gmail rejects emails from your domain it is up to you to fix it. Google is not going to change, and enough of your users will be interacting with people on Gmail that you have to fix it. It doesn't help that Google has been pushing people away from running their own email and into Google's services by ever tightening what it accepts over the years. More than one person has given up on their email server because it was a constant battle with Google, Microsoft, and company to not have important emails disappear into the void.
Their @gmail.com servers accept the messages (as said in the post) so it's not a problem for 99% of Google users either.
If you choose to host your email with Google, it's up to you to fix your email delivery settings (or find a better provider) for your domain.
An attempt no doubt to extenguish a standard google doesn't control
My takeaway is there is no bug. My takeaway is that his test email bounced because he didn't have the reputation Viva does. Emails are handled on a reputation basis, this is why we use email service providers like Sendgrid, Mailgun, Postmark, etc.
It always amazes me how people can read a blog post like this one that has a clear description of the problem with a log excerpts demonstrating the problem, and then people will confidently make up a completely different scenario that was not mentioned at all and blame the problem on that.
A log that clearly was from them and not the service provider. It amazes me you think you're so smart but haven't realised he doesn't have access to the logs you think he is showing.
Comments like this are why he's just landed himself with a major liability and I bet he'll be getting sued over this.
Pretty certain that you're wrong.
TFA shows an excerpt from the email log for his google workspace account, showing the bounce of email sent from viva.com.
Then, TFA states that he switched "the account" (his viva.com account) from using his GWorkspace address to a personal @gmail.com address, and asked viva to send another verification email. That one arrived.
At no point does TFA describe the author themselves sending a test email.
I've added a screenshot at the end of the blog post just to clarify that.
Would you mind scrolling to the end of the post? Or, if you're in a hurry: https://atha.io/_next/image?url=%2Fstatic%2Fblog%2F2026%2Fvi...
https://support.google.com/a/answer/2618874?hl=en
> I decided to dig into Google Workspace's Email Log Search to see what was happening on the receiving end.
It amazes me that you can read an article and draw the exact wrong conclusions
I wish I had your confidence in life
Please read the blog post you are making such strong claims about.
What that is liable? That is a very small claim.
I think that's a misunderstanding of the tale. Viva sent a "click here to verify your email" to OP. That email never arrived because Google rejected it for missing a header. OP tried to tell viva, but they don't wanna hear it because OP worked around it.
> My takeaway is that his test email bounced
What test email? I see no mention of a test email in the blog post. The mail that bounced was the one with the verification link from Viva.
So you think he had access to Viva's email servers to see the response? No, he clearly tested it himself and used his credentials to send it.
The log line is from Google Workspace which exposes it to its customers for incoming mail
Thank you! I added a screenshot of the Google Workspace Admin log screen... just becuase.
What test email?
https://atha.io/_next/image?url=%2Fstatic%2Fblog%2F2026%2Fvi...
Yeah. I think email receiving is a game of exceptions… the email receivers (In the business world it’s essentially just MSFT and GOOG of course) answer to the addressees because they are the customer, and those customers will start to shriek if their inbox doesn’t receive “Important Messages.” But GOOG or MS have no leverage over the senders in this case so they just add an exception: “if IP range is just right and message fault ___ is present, fix message” (or otherwise allow)
Of course, they do have leverage over “marketing email” senders since they can block it and no one will complain, so those senders always have impeccable compliance with every year’s new “anti-spam standard.”
Apple is another major player in the email receiving game for consumers. And they are awful, by far the worst of all the big providers. They do not send dmarc reports and they make it very difficult to tell why they accept some email and not others.
If you read two paragraphs further than the Tl;Dr:
> To unblock myself, I switched to a personal @gmail.com address for the account. Gmail's own receiving infrastructure is apparently more lenient with messages, or perhaps routes them differently. The verification email came through.
1. Email didn’t arrive in his inbox of his Google workspace
2. He checked workspace email logs (with admin you can do this on gsuite)
3. It showed the intentional non-accept
4. Comprehending the problem, he switched to personal Gmail
5. The email arrived
6. He informed the sender of the original problem which he worked around
7. Sender is tech-illiterate and did not realize what the problem is. This is common with first line customer support so that happens.
The question to ask is whether you are literate in English or you skimmed too fast. Because I did a 30 s read of the article and got that.
It does seem unlikely that there are no customers on google workspace who have tried to use viva. I don't do payment processing, and my email is via zoho, so I've no idea how large either of those groups are.
I wonder what google workspace support said.
I suspect that Google going out of their way to make this required had a very reasonable and thought-out process, while the sender's omission was on oversight, so I haven't contacted Google Workspace support.
What's truly iffy is that GMail doesn't have the same strict requirements, and there's no way (at least that I found) to turn it off for my Google Workspace domain.
Wikipedia says Viva.com is a multi-billion dollar startup
It seems unlikely you're the first company using viva.com and using google workspace.
Clearly the problem here is that viva.com emails aren't arriving on your google workspace, despite what their support process says.
viva.com emails do arrive on other email providers, so seems unlikely to be problem with your viva.com account
It seems unlikely workplace blocks all viva.com emails otherwise more than you would have complained.
Whether that's viva's problem or google's problem is a separate problem.
The bigger issue here is that Europe depends way too much on the USA in so many areas. This is not good - you can be constantly blackmailed when you have people such as Trump in charge. I don't think the EU can be fixed, but at the same time I also think the less Europeans depend on outside factors (in particular the USA) the better. Canada kind of showed how to do it. Granted, Canada is also dependent on the USA in numerous ways and most of this is hard to fix (most Canadians live in the south aka close to the USA and trade is primarily done via the USA; security has also been largely outsourced onto the USA and so forth). The sooner people in Canada and Europe get moving away towards more independence from the USA, the better. And more cooperation would not harm either.
As a European (who spent 15 years in the US), I coudln't agree more. And while I agree, at the end of the day, I just want the better product for me.
This bug will not be fixed before the Environmental Impact Study is concluded on it.