The irony of an article that makes a false claim about what Google was found liable for.... and that very few are fact checking it :)
The law they broke was a law protecting personal and business reputation against false statements of fact. Essentially no one can say I might be wrong, check yourself, but X is Y if that claim is essentially defamatory.
This is pretty good, I hope googles approach is to make sure they don't end up making statements of fact like they did and use more appropriate wording like according to X.... with direct disclaimer that they can't verify it. Even better that they look court documents to find any legal ruling and point people to that too.
How much does Germany follow case law? If this could set a precedent, it's worth noting that anyone can generate these AI overview responses and they're wrong like 9 times out of 10 only.
Good. The true mark of AGI is when a company accepts liability and doesn’t bury “for entertainment purposes only” deep in their TOS. Same as it works with employees.
Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger.
But watch as Germany soon loses AI Google results.
I think it's a completely reasonable position that companies making self-driving cars and question/answer systems are legally liable for any errors.
But if you hold that position, you also have to be fine with companies not offering products and services in your country. AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits. Until that time, such countries would just not have access to systems before they were bulletproof.
> AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits.
I doubt that will be the case, because of the long tail problem. (same with self driving cars and other ML related problems).
In fact, we have counter-examples today. Newspapers (even reputable ones) can't get it right every time, despite the fact that they have both trained people and in theory they're setup to catch that w/ reporters - fact checkers - editors. And still, from time to time, they get it wrong. (and I'm not talking about purposefully getting it wrong, just honest mistakes.)
What will likely happen with a ruling like this is that the answers will be hedged and legalesed and muddied up the wazoo.
Most of the time, human beings driving cars with their own hands and eyeballs are not "liable" for their errors (unless they can be proven negligent or drunk), unless you count their insurance going up. Most car accidents do not end with anybody getting arrested, or sued, or anything like that. Insurance pays out, premiums go up, case closed.
If Waymo can be proven negligent or something, then sure, bleed em dry. But as long as they're acting in good faith and significantly reducing overall road fatalities per mile driven, I think it's actually pretty unreasonable to try to hold them to such a high standard you end up subjecting society to more of the higher fatality rates caused by humans.
An at-fault driver’s insurance pays out because they are liable. Your insurance covers your liability. That’s why you need insurance.
> Until that time, such countries would just not have access to systems before they were bulletproof.
Correct, most jurisdictions do not allow businesses which cannot be held liable for their actions. This is pretty core to a modern society.
Imagine if a company selling Knicks tickets was not expected to then actually provide said tickets and there was simply nothing you could do about it. Oopsies our sales page is for entertainment purposes only
To be fair, the internet has spent some 30 years figuring out how this works and it’s still not fully resolved. For the most part we’ve agreed that companies must follow the laws of both where they live and where they operate. This wasn’t always obvious!
Almost all jurisdictions allow businesses which cannot be held liable for all their actions. Imagine losing you house because someone decided to smother themselves in your infused cooking oil and light themselves on fire.
Businesses are made to make it easier to share profits and responsibilities when trying to fulfill users wants. Laws are made to offer protections for consumers (because nobody has time for common sense), but at the end of the day the consumer has to take responsibility or no products can be made. If you're too fat for a chair, it's on you to find or make one that works- not every product is for every person. Laws only stop chairs being sold that are too dangerous for anyone.
Sounds like a win win to me
To remove the choice from responsible people who can understand that LLM answers are not to be trusted with anything important?
If our standard for laws would be that "well no reasonable person would do this/believe this" then nothing would be illegal, there'd be no need to label any product as potentiality harmful, etc.
Do you really want to go there? That everything in the world would have a literal "caveat emptor" attached to it?
I thought Google labeled its AI summary with a disclaimer already. I don't want companies to be forced to only offer safe-for-children services.
And the european consumer doesn't want harmful products to be beta tested on the public.
So we deploy a technology no one should trust to the general public, for what exact reason?
Do you mean the responsible people who will ensure their algorithms can be trusted with the important task of acting in the best interest of said people? Try and get a defamatory statement about google from the AI search box.
> Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger
Mercedes-Benz does this in limited cases. Waymo does it generally. (In China, Level 4 and 5 transfers risk to the manufacturer. This is the correct way to do it.)
That's not exactly the case in China, the current state of FSD is still pretty dumb, unless you consider transferring control back to the user at the very last minute before it crashes a proper way to handle risks.
> unless you consider transferring control back to the user at the very last minute before it crashes a proper way to handle risks
I don't think Tesla is based in China
I guess some Tesla are manufactured in China lol. I am just trying to say that the liability that Chinese manufacturers takes aren't more than the US ones.
It doesn't "bury [it] deep in their TOS", it says right under the box:
> AI can make mistakes, so double-check responses
They decided to hijack search and rewrite other peoples websites as their own.
If they want to claim ownership, then they will have to accept responsibility.
Search still works the same you just have to scroll down
Banning all technology because someone might misuse it is an illogical extreme.
As far as I can tell the ruling is more nuanced. If AI is defaming you, there needs to be a way to correct the record.
A company being open to liability does not mean it is always liable, just that it can be if it really messes up (especially if there are aggravating circumstances, e.g. you need to drag them to court to issue a correction).
at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth. what if there's a 1/1000 chance of some error, then the company could be sued millions of times per day.
down vote all you want, but I firmly believe this is an example where the user needs to use some judgement on the information they receive and have some critical thinking skills. google would be right to remove all AI results from germany.
If I post “heathrow83829 is a convicted poopoo head” (replace with your favourite crime) as if it were a factual claim, you’d be well within your rights to sue me for defamation even if people should apply some critical thinking skills and say “wait a minute, how would pdpi know that? Are you sure he isn’t just talking out of his arse?”
Now, search engines are usually afforded some amount of protection against defamation claims — they’re not held liable for simply indexing and quoting third party defamatory claims. Which is to say: Google wouldn’t be liable for claiming you’re a poopoo head if this comment shows up in search results.
The point of this ruling is that AI-generated text isn’t a quote from a third party, it’s text generated by Google’s own tools, so it’s speech by Google itself. It might be wrong, sure, but it’s still presented as a statement of fact.
At trial they can have the whole debate about whether Google was negligent in how they build their systems, and all that jazz, but let’s be clear here — it’s not a matter of every little factual mistake getting Google sued (and that would be absolutely terrifying from a freedom of speech perspective), but rather that the technical means by which you generate content doesn’t change your liability in publishing that content.
I think Google added that AI-generated responses maybe incorrect? When you are paying such a low amount of cost, like probably for free, I don't think you can expect a same level of quality as a human written or reviewed of answer. It is like same random user spin up their Lovable and vibe-coded a piece of slop and hold Lovable responsible for not giving them production quality code. It is simple, you get what you paid for. If someone actually figured out AI that is actually always correct, it would be charged in superhuman price as well.
This case isn’t about Google’s users getting low-quality search results. As you say, you get what you pay for. The actual issue is that, in some searches, the AI summaries would claim the plaintiffs were guilty of scams and all-around shady business practices.
Put differently: it’s not newspaper readers complaining the paper is inaccurate, it’s the people mentioned in the articles.
the user needs to use some judgement on the information they receive and have some critical thinking skills
how?
errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
google would be right to remove all AI results from germany
i'd consider that a win.
By checking the citations rather than taking what’s generated at face value.
If it’s important, check it. If it’s not important, then it is pretty much just entertainment.
LLMs can be very useful in a general web search and save some time, but if you don’t put those literacy & critical thinking skills to the test and actually confirm anything, then you might as well not even have bothered with the search at all unless you’re hoping it can just replace all of your original thinking too.
If google didn't intend it's answers to be taken at face value it would just present the citations in a list of links rather than generating an answer.
Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
Important for whom exactly? If it's you who are called convicted pedo by Google AI summary, it's you who has vital interest in additional research but not me who reads it. There's intentions mismatch. Which probably would destroy your life, and you won't call it "entertainment" then, I think.
But you are not doing a 2 hour rabbit hole search when you stand in front of a T-shirt and check whether it is fair traded or all-american produced.
You’re seriously arguing that Google’s libel shouldn’t count as libel because they showed it to too many people? It’s absolutely insane to suggest that a company should be immune from liability for its actions if it operates on such scale that those actions harm millions of people every day on the basis that dealing with that many lawsuits would be too inconvenient.
its not inaccurate though.
consider Purdue pharma - the Sacklers got off with all their wealth intact because they were too big to sue and properly collect money for their victims.
But we agree that this was also wrong, right?
> at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth
If a Google employee (like a support agent) says a mistruth, the company is liable and you can sue. They can’t just say “hihi oopsies our support agents are useless”
So scale of harm creates immunity, is that the argument ?
Did we not already see that with the financial sector in 2008?
Too big to fail. Lol.
[dead]
Google can be reasonably expected to not push pirated content to top if someone is searching for the “big hit music download” because they might be held liable for helping people with illegal downloads. But they shouldn’t be liable for misleading people? Being sued for billions of dollars by corporations vs millions by common folks is the difference
There's also a difference between search engine results and this. Google is publishing their own text with false claims, not linking to another page that hosts it.
I'm seeing this sentiment more and more these days and it's worrying. Essentially people are starting to believe massive corporations should be able to get away with more than individuals. This is completely backwards considering the enormous impact a corporation can have, but that's what people are starting to think. I've heard people argue that corporations should be able to straight up lie and deceive to protect themselves, which is something you would not accept from a person.
Corporations are you. Corporations provide a service to individuals, often in exchange for money. Restricting corporations from doing something is the same as restricting people from receiving that as a service.
Only referencing America, but professional liability for doctors, engineers, lawyers, etc isn't based on perfection. It's based on a reasonable effort.
So Google could, for example, switch from a tiny "this could be wrong!" byline to having the AI be less overconfident every freaking time regardless of whether it's spouting made up crap or actual facts.
The scale doesn't sound like a way out. If your company expects to get away with doing the wrong thing where smaller companies can't, then the solution isn't to continue getting away with it.
How do you feel about the EPA, industrial accidents, oil spills etc? Does scale give every company a free pass for damages?
That's not a fair comparison. If oil companies would get sued for every leak, they had to face millions of law suites and wouldn't be competitive anymore.
(Sarcasm to support your argument)
The problem with "the user" argument is the spectrum of users. There are different skills, capabilities, and intelligence. Frequently we wave our hands and say exactly this, critical thinking. But, not everyone is capable of that, nor is everyone capable to the same degree.
As a society we decide. Are we embracing all users, are there basic rights and assunptions? Do we only enable some?
As a free (as in cost to end user) system, Germany is arguing that their social compact raises the mininum bar. Frankly, thus might help drive a rush to increased accuracy for AI- tech finds a way. Equally it may hinder - beaurocracy creates barriers.
I'd love to be able to rely on these search results. I see them ad the same prior set of inaccuracies whereby I have to do more research. At least now there's a summary and direct links to the supporting information. But equally, we're primed with the information in the summary.
Then Google can either discontinue their AI or make damn sure it's good.
So once you get to that scale it suddenly doesn't matter anymore and you can't be held accountable.
But until then, be a good citizen?
What? That's fucking feudalism... Peasants and Lords.
If you're lucky enough, you're born as a Lord. (And maybe don't live during a revolution)
This makes no sense to me at all. If you're small you should get less bureaucracy than if you're bigger.
For e.g. self driving cars there should not be any exemptions. There are people's lives at stake, people who didn't sign up for your shitty service.
> google would be right to remove all AI results from germany.
As others already said, that would be a great outcome, and german citizens would benefit.
It is a search engine. It used to have decent excerpts. It doesn't need hallucinated generative AI summaries. I am about to click the link anyway.
[dead]
Why would that mean AGI? You can get into liability-accepting territory by restricting scope, a lot easier than by making your AI smarter.
Self-driving cars don't need to be particularly good for companies to make models where they accept liability in some circumstances, and the cars refuse to drive in other circumstances.
Wasn’t Tesla found to have FSD disengaging just before a crash so that the driver would be at fault?
No. Sometimes it does disengage because things are going wrong, but those incidents are still reported the same as if it stayed engaged.
I found one time Musk was using a few seconds of disengagement to insult a driver, but it still would have counted as an FSD crash by Tesla's statistics.
Ai results that nobody wanted in the first place?
> But watch as Germany soon loses AI Google results.
Time to set my VPN location to Germany. I'm tired of the "udm" trick.
Yeah, I fail to see a down side to this. Those overviews are less than worthless to me.
[deleted][deleted]
Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.
Otherwise most of it would not even exist.
Everyone would have continued paying out the nose to the IBM’s of the world year after year (who had unusual willingness to sign short ambiguously worded custom contracts to their own disadvantage, if paid vast amounts of money).
And be on mainframes to this very day… maybe Y combinator and HN wouldnt even exist in that world.
> Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.
A lot of people in IT seem to think law and contracts are in a sense mathematical. They aren't; they're more like a high school book report - to be interpreted, as objectively as possible, but definitely also establishing the intent behind the letters.
Particularly contracts - no, you can't trick your way into things in most cases. "Surprising" clauses are invalid in most legal systems, in particular if one party to the contract is a layperson.
That is a false dichotomy. The solution to failed laws and regulations is not crime and corruption. The solution is to hold the policial and business leadership accountable; to fix the laws and regulations.
The entire American tech industry has exported Americas predatory, parasitic, and unethical consumer laws (the majority of which are ghost written by the wealthy and corporate legal teams). When I studied law in school decades ago, tactics like bait-and-switch, false advertisting, intentionally misleading or deceptive practices etc to sell products or contracts were illegal across the developed world.
Those illegal, anti-consumer tactics were the SOP of every tech startup I can think of from the early 2000's onwards; following the same route of initially offering a compelling feature set to attract and entice users – usually for free – until securing a certain number of users or funding, then changing the value proposition to exploit that user base, and extract as much wealth from them as possible, ad infinitum.
Today these tactics are known as enshittification, and the average American pseudo-libertarian software engineer will say this is fine, but that's what every anti-consumer parasite and criminal has said in history. Lying, misleading, and exploiting people for financial gain is fundamentally immoral, corrupt, and sociopathic, therefore it should be illegal. Just because it's the norm, or a digital product, you wrote that in the T&C's, or your doing everything behind the liability shield of an LLC, doesn't change that.
What ever happened to the concept of building a valuable, quality product and stable returns for generations? Working to improve the quality of life and standard of living of the community? Of the world? I feel like a 1950's traditional conservative when I suggest that, but most Americans are so heavily indoctrinated with corporate greed and sociopathy they'd consider that sentiment radical leftist extremism. I'm an athiest, but ya'll need jesus (the real brown socialist one). Many would argue Americas current institutional collapse is the natural result of this systemic corruption.
> I feel like a 1950's traditional conservative when I suggest that...
I wouldn't argue that America's moral standards haven't declined (significantly) but I also think it's a romanization to suggest that 1950s America was the pinnacle of morality.
Lying, misleading, and exploiting people for financial gain has been a part of the fabric of American society since the country was founded.
If we're being honest, humans everywhere have demonstrated a high capacity for this behavior since the dawn of civilization.
[deleted]
> The true mark of AGI
Can we just trash this as a marketing term? If/when AGI arrives there will be no point quibbling over competency. What we are looking at is just bad search results
Why is it good? Everyone with common sense knows AI can be wrong. And it’s not buried in their TOS. It’s in the chat box. But even if it wasn’t, it’s ridiculous to create liability for AI chatbots.
[dead]
> But watch as Germany soon loses AI Google results.
Oh no! Anyway ...
How could anything else make any sense? Platforms are getting used to provide dangerous broken products and get away with it. There should be some limit to it.
When I was a kid it was possible to buy any foraging books from a store and they had a minimum quality. Is that so difficult to achieve? Is profiteering not punished anymore?
Should we hold scientists and journalists liable if they say false things or misrepresent things?
We already do. Libel and fraud are already illegal.
Yes, obviously.
If they do so knowingly, and harm is caused, then yes. Are you suggesting we should give them a pass from years of acquired jurisprudence simply because they hold a particular title?
And what institution gives out the licenses for journalists and scientists? Is it revokable?
Choosing the answer for you rather than leaving it to the user is a tremendous power and the court correctly diagnoses it comes with responsibly to minimize harm to others in society.
I've also observed that the AI summary on the google search page is incredibly stupid compared to the results in the actual Gemini. The Google search AI is like the dumbest lightest model simply rewording the search results. It will take a random reddit comment, strip it from it's context and present it as absolute fact.
Of course, they know this. The entire point is be able to rewrite people's awareness.
I want to add my interpretation of the phrasing "rewrite people's awareness" to make it read less tinfoil-hatted:
"rewire brains for AI dependency" (for money and power reasons obvious to everyone).
In contrast to "secretly implant an agenda about non-AI subject N", which is complicated enough that AI companies are still too out of control to be attempting yet.
This makes sense to me. AI is amazing tech, but it's being oversold to naive public, either on the back of hype, or cynically, knowing they can do it with inpunity, (I'm not sure which). HN users have a way better than average grasp of what AI is and we can be skeptical of results, the general public has no clue.
100% cynically. It's a game of infinite money glitch (also known as "grab as much as you can before it's over") and has been at least since GPT-3.
Good. This should be taken as the precedent for all economies: If you promulgate demonstrably false information to somebody's detriment then the owner and operator of the machine has to carry the liability.
I very much hope we don't see attempts to re-write T&C to avoid this liability.
> In this case, Google's AI had wrongly linked two publishers to scams and shady business practices.
Guess that's the end of their AI overviews in the EU!
You'd think so, along with other countries that have defamation laws. But there's no indication of any penalty, and Google wasn't even made to pay all the legal fees. Perhaps their business model (if there is any) can cope.
And lawyers will use this case to build a better defense next time.
In sane countries, it's enough for them to post a disclaimer ("This is AI. AI can make mistakes. Check all results.") Which is what they do.
Overregulation, at best, is a good way to guarantee that your country won't have access to interesting and useful features and technologies. At worst, it's a good way to guarantee that the twenty-first century will belong to the US, if not to China.
Okay then, CamperBob2 is a scammer. Many users report this person has stolen money. (+3 sources)
I can make mistakes. It's on you to fact-check my claims.
Do you think these are harmless statements? Does the disclaimer suffice? If I was Google's AI Overview, do you think 100% of people will check those sources?
There is nuance here, and it's not going away because AI and innovation.
You forgot to mention that it is the real name and it is shown to everyone who has interest in you and they are not fact checking.
Do you think these are harmless statements?
Yes, I do. An assertion made without evidence can be dismissed without evidence, whether it comes from Google or from you.
If you demand perfection, you will receive nothing. Why is that so hard for people to understand? The world simply cannot work the way you say it should.
There are a lot of people who consider no ai to be a positive state. So "you will receive nothing" may not be perceived as the threat you think it is.
Those aren't people whose opinions I'd be interested in, so they're free to think anything they like about me.
If it is shown to millions, everyone who is interested in you, they are factually not fact checking and you can do nothing against it.
It can destroy you. You are analyzing this on the assumption that people are rational and have capacity to do the check. They have neither.
(Shrug) That sounds like their problem, if they reflexively believe everything Google's AI tells them. No one I respect, care for, or work with would believe such a thing without additional justification.
Anyone who does accept Google's AI output blindly will soon find that their mistaken opinion of CamperBob2 is the least of their problems. There is a reason Google goes (well) out of their way to warn people that the results may be wrong. That should be sufficient warning for reasonable people of good faith.
that would be hela curryketchup nice
It depends if Google feels the profit is worth the risks.
What profit? I don't know either but they enabled this for a reason right?
I agree with the ruling, but this makes me wonder if it will be possible to have any AI agent at all if it's consistently applied.
After all, if I can get ChatGPT or Claude to say something false that should count too, right?
I think it should be relatively possible for an LLM to know when it's talking about something in the risk zone of libel. And when it identifies such a case it links direct to source material rather than trying to fabricate an answer itself. This is much how a journalist works, refusing to make claims that aren't provable.
Someone needs to hold the liability at the end of the day. People are experiencing real harm from false claims LLMs are spitting out.
If it's consistently applied, any AI agent provider has to comply with cease and desist letters that tell the company to make specific false claims
The arguments of the ruling should generally apply even when the AI agent makes false statements it wasn't notified about. But in that case the defendant might have a stronger claim about not being able to reasonably ensure the correctness of all statements, and having taken reasonable measures to ensure correctness. Google couldn't really claim any of that after ignoring cease and desist letters about the false claims
Due to costs of running frontier models on every search request, Google simply does not: the failure rate is so high when you are just expecting an objective output.
Imagine a search for your name resulted in an AI summary saying you are involved with child-trafficking because low capability model linked your first name and perhaps a couple articles on supporting children non-profits to it — and then offering that in a convincing sounding summary right at the top!
In normal flow yes, but likely not if you intentionally entrap it to say something wrong.
A disclaimer and couched language will probably fly through. And it's going to matter what expectations an user could reasonably have, too.
If you give a language model, empowered through an agent, the ability to publish information on your behalf, and it publishes false information which causes either direct or even indirect harm, and you fail to correct it, then yes.. by every conceivable definition already in law.. you are a criminal.
If you knew this was all possible and you did it anyways for personal gain then you are additionally negligent which may add aggravation to your charges.
Curiously, if you look for "geramond verlag betrugsmasche", or "verlagshaus24 betrugsmasche", it will now tell you that
there are no indications it is a scam, but "significant organizational problems and extremely bad customer support lead to (list of bad experiences)".
Also, each purported fact now has a direct link to the source of the fact, that is more clearly visible than the previous chain icon.
Do the links actually support the statements? When I've followed such links, it's generally been a roll of the dice whether they do support the AI's statements.
Companies generally are liable if their product doesn’t perform. No reason AI should be any different.
That’s not very typical in software. Especially software you don’t pay for (with money)
That's apparently already changing in the EU, where software vulnerabilities mean the company is liable for damages. The only way out is to straight up not make any money (not just from direct sales) from the software.
Is the burden of proof on me, the developer? Do I need to prove in perpetuity that I didn’t get a job or a free flight to talk at a conference because of my free software? (Which had a flaw that hurt someone)
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. If you are a victim too, reply below.
I'm surprised this is even a thing. After all, you go to Google not for the truth, but to search Google. Since when is truthiness the "guarantee of service"?
You're not even paying for a google service, search is free... You might be the product, and your data, but you didn't directly pay for a service and they didn't sell you a fake service.
I'm not taking Google's side, this isn't about whether it's right or wrong to rob websites of traffic, this is about AI's returning search metadata.
But I'm surprised that they lost this argument, and the line they took in the first place.
The Internet isn't made of fact checked data, it's crowd sourced. How can anyone be liable?
That is exactly the point of the ruling, though... they are saying that AI summaries are NOT the same as search. If Google was just returning search results, and then users clicked on a website and read the content there, Google is not responsible for the content.
If instead Google gives you an answer right there on google.com, without going to another site, they ARE responsible for it.
That makes sense to me?
Not precisely. The issue at hand isn't just that Google displayed the AI summary, but that they authored it, making them responsible for its contents. If the defamatory content had been in a snippet in the search results, they would've been fine, because that clearly has another author who can be held responsible. The AI summary has no other author than Google; therefore, they're responsible for what it says.
(What's the alternative, after all? Having no one responsible for what the AI summary says is clearly untenable.)
why? tons of websites push misinformation intentionally. is there a truth requirement anywhere? i don’t get why this is a thing at all
What don’t you understand? Those websites that defame a company are liable for that defamation. In this case Google defamed a company in its AI summary and is this liable for that defamation.
So if I edit a Wikipedia article to share that consuming poison is safe and someone consumes poison after reading it… is Wikipedia legally liable?
> is Wikipedia legally liable?
Probably not, because it's a similar situation where Wikipedia is accumulating user provided content. And people know Wikipedia can be freely edited.
You, however, might be liable. It's your content.
No, because Wikimedia isn't responsible for the behavior of its editors.
> is Wikipedia legally liable?
Directly? Quite possibly. They'd then have to transfer that liability to you.
> is there a truth requirement anywhere?
Yes, and it's called defamation when you don't follow it.
And those tons of websites are liable for their misinformation. It's probably not worth suing some random blog because the author probably doesn't have money or lives in Russia. But Google has lots of money and a legal presence in almost every jurisdiction.
There is absolutely a truth requirement.
This is why you have to say "I think this person is a murderer" and not "This person is a murderer."
One is opinion. One is fact.
This isn't super hard.
That's the difference between returning search results and interpreting the information and summarising them. If a newspaper says 'so-and-so has been arrested for theft' it's not the same as them summarising to 'so-and-so is a thief', the second is potentially libel. Why should Google be held to a different standard?
The title is misleading IMO. It should say "German ruling declares Google liable for libel in AI Overviews"
I was prepared to say the same thing as you but after reading it seems totally fair.
The key difference is that this would be illegal if a human wrote it too.
Google itself is more trustworthy from a normal person perspective as they use it a lot.
None of "AI" companies call their apps "Entertainment fun text generator". They are call them serious names, use words like "intrllegence" and "thinking".
So yeah I'd think if any of "AIs" start to recommend to drink some bleach or take a flight from a 10th floor window these companies should be liable.
The question is whether Google is publishing false claims or relaying other people’s false claims. The court found it to be the former which makes sense to me.
I think it's very clear that Google's AI overviews go far beyond just searching Google because they often incorrectly compile sources to come up with an incorrect answer. For example of this look at the comment I made in this thread
I go to Google to search, but get spammed as if I wanted to talk to a chatbot (and a very poor quality chatbot at that).
This is a gigantic own goal for Google. The average person’s impression is that Google AI is much worse than ChatGPT, even though that’s not actually the case. Google is shoving a terrible model in everyone’s faces.
Playing the perception game wins you the perspective price.
Nothing is free. Google benefits off you when they show you search page. Either today (ads) or later
If companies can be held liable (in spite of very visible disclaimers, ToS, and usage policies) for the output of non-deterministic software, isn't this just a soft ban on the deployment of non-deterministic software?
> (in spite of very visible disclaimers, ToS, and usage policies)
If you sell food, in a food stall, labeled as food and you add a disclaimer that it is toxic and will make you sick. You are still selling toxic food and you are liable for it.
Google is pretending to give answers to your questions. They offer you a service about answering questions. And then they add a disclaimer "we do not answer questions just write bullshit". That is still fraud and Google should be liable for it.
> isn't this just a soft ban on the deployment of non-deterministic software?
Tetris is non-deterministic and it is not banned like millions of other programs. I do not follow you.
To add to this, a Google search now is answering your question in an incorrect way rather than merely bringing you to a site with incorrect information on it.
They are also no longer covered by safe harbour provisions because it is them answering it, not some content they refer you to.
Where did it say the liability only applied to non-deterministic software?
LLMs are deterministic, they are only non-deterministic when you add a temperature.
no, just a ban on using non-deterministic software for situations where deterministic responses are expected.
Deterministic responses are not expected from something clearly labeled able to make mistakes.
it's not clearly labeled. it's a search engine. i expect deterministic responses from that.
What to do if the software automatically and wrongly libels you on a public search engine?
Honestly I can understand the ruling, but the side effects might be severe.
Oh, I just found out that my Google search doesn't show AI summary anymore! I tried many search queries which typically will show an AI summary, but it only flicked on briefly then disappear entirely. Obviously, Google has reacted quickly on this ruling!
What constitutes a correct answer though?
Is something like,
"People online say that x y and z because a b c"
a credible, correct answer, even if it isn't because of a/b/c?
If people do say that, it's a true statement and thus fine. You are allowed to report that regardless of the truth of x/y/z/a/b/c
The instance of this ruling people apparently did not actually say any of the offending claims. 'The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."'
One that doesn't maim/injure/kill you is a pretty good standard. And before you call bs, look at all the foraging and chemistry books that are for sale on Amazon that are AI.
why are those ai chemistry books any different than the anarchist cookbook which can also be bought on Amazon? actually now that I think about it a faulty chemistry book might be less dangerous than a book that teachers readers how to make explosives.
Certainly, if this is pointing to the actual pages where the actual people express these things. Otherwise that's equally unfalsifiable claims, could be completely made up or actual truth.
One way to formulate things that would be less would be "once support a time, in some fabulated world, it's not impossible that some imaginary character would say something following some reason." But then, of course this is not aligning the the deception scheme pushed by companies putting in their interface that the "machine is thinking hard for you".
Anyone know if this ruling applies to answers generated by AI chatbots, such as ChatGPT/Gemini/Claude?
All three have the ability to perform a web search, then compose a reply based on the search results. Pretty much the exact thing that Google AI Overview does. This ruling may make them liable for false answers.
> Pretty much the exact thing that Google AI Overview does.
No, the article implies the court’s logic is that the AI search results are presented as search results and that’s a big part of why they are liable. It seems like the court (again, according to the article) does not find the disclaimers that Google has slapped on the AI results compelling because again, it chose to represent these as a summary of search results and it is aware of the failure rate.
> The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."
> Google built the AI, Google offered it to users, so Google owns what it produces, "because it alone has influence over the AI's offering and the algorithms with which the AI operates."
Google does not, as a general rule, control the actual content of search results, but usually there’s a distinction between the ranking and presentation of the results vs. the actual content. In this case, the court is basically saying, “You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway. No, you don’t get to claim the equivalent of a US safe harbor defense.”
> You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway
There is a subtle difference in stating it as a search summary compared to an opinionated answer. Most users are always going to treat it as a response from google instead of search results where the user is still responsible for understanding and come up with their own interpretation.
This is probably the right step in some sense to make one liable for their statements/assertions.
Apparently, if they were search results, they wouldn't have been liable, since there's an exception to defamation laws. Without any exception, defamation is defamation, it doesn't matter how it's presented.
Yes, if it was that a search result returned a defamatory article that Google had nothing to do with outside of indexing, it is likely they would not be found liable. The court is clearly trying to make a distinction that the AI search results are produced by Google and thus they can make an editorial decision on whether to publish it despite knowing that it is potentially defamatory.
Google does remove defamatory results I believe at least partially in response to being sued. However there is a distinction if they have been informed it is defamatory.
This ruling was about search clearly, however, there's definitely ways implications for chatbots too.
[deleted][deleted]
I don't see any reason why it wouldn't.
Unless the courts here made the ruling incredibly narrow somehow (only referencing search engines maybe?), how does this not just ban AI in Germany overnight?
Every AI model can make something up sometimes. Over millions of daily calls, it's essentially impossible for the technology to be guaranteed correct 100% of the time.
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. I wonder how many victims are there now.
I have a business where our support email is recommended when people are searching for how to cancel a completely unrelated scam subscription that is showing up on their bank or credit card charges. We get emails almost daily from confused people.
Finally, a sensible ruling based in the interests of the public rather than expediency for corporations
Doesn't libel require to be deliberate? Ie you can't sue for libel if the author admits a mistake and corrects it?
It requires the claim to be made with "willful disregard for the truth". Notifying someone, especially with a cease-and-desist on fancy letterhead, makes it legally clear that they know better, and thereafter would be definitely libelling you (assuming the claims are in fact untrue and harmful). But you can still sue them for the claim prior to the notice, you just have to prove they should have known better prior to making the claim.
In this case it looks like they were notified and didn't do anything.
That's just America.
Just ban AI in search engine.
Some digital matters concern the state and others are private and there should be no sovereignty of the state over private matters.
Who will adjudicate private disputes between citizens if not the state?
Not all disputes need to be adjudicated. It's undoubtedly a fact that the German concept of "defamation" primarily relates to situations that should never be adjudicated by any third party.
Genuinely, by far the most common situation where claims of defamation arise in Germany is any less than 5 star review left to a business on google maps.
We should be teaching people to be cynical of AI answers.
Even if the answers are correct, they could still be biased, incomplete, misleading, and all the other media-literacy things people should be looking out for.
This ruling seems to go the opposite direction; 'I am legally obliged to give correct answers, so I am always right, trust the AI'.
Does this extend to ads displayed in search results? Because they absolutely should be liable for the scams they advertise also.
> Does this extend to ads displayed in search results?
Probably not, for the same reason search results aren't an issue.
Search engines have that exemption for results because they don't control the content of the sites they index. That's not the same for ads - Google can and should have more stringent review and KYC processes - but it seems they choose not to.
Wow, you guys really think this is good?
Because of the same rules, German restaurants also get to pick and choose which reviews stay up. They can literally take down any specific reviews they like.
A restaurant that mostly gets 1 star reviews will still show up with 5 stars on Google maps, as they will simply delete the reviews with less than 5 stars as defamatory.
tl;dr Germans are particularly bad at coming up with reasonable rules to handle these situations
I've found a fun and pretty reliable way to get Gemini to output incorrect information: Ask for a chapter by chapter summary of a book.
I first tried it to remind me of what happened in a previous book in a series that I was reading. When I realized it was either misstating plot points or straight up hallucinating, I tried it on a bunch more books to amuse myself.
Older classics are of course more accurate, but for newer or less popular books Gemini won't shy away from giving you a summary culled from misinterpreted Reddit threads and Goodreads reviews. It's like getting a secondhand account from someone who talked to another person who had read the book a long time ago. You get the general gist of it, but with some added flavor.
Even if you upload an entire epub of a book, the results aren't stellar. Rather than a Cliffs Note's quality summary, they're pretty sparse or leave out important bits of information. One chapter summary I got back made a point of describing what one of the characters was wearing, even though it had absolutely zero to do with anything else. Yes, that's technically a "summary", but not quite my tempo.
If Google wants to present summaries of websites in anything more than a very, very superficial description, they're going to have to improve their model's ability to understand context and importance. In theory, a novel is a self-contained bundle of text, so pulling accurate information out of it should be straight forward. A website is naturally going to be way more of a challenge.
All that said, I find the AI summaries from Google/Gemini to be quite useful and a time saver, but I know to always double check something if it's at all important.
So stupid. What is this with making perfect the enemy of good. You can never guarantee the output of an LLM does that mean Germany does get to use them?
EU countries continuing to ensure the conditions for their future economic competitivity
mr prime minister, we cannot allow an "automatic libel machine" gap!
The irony of an article that makes a false claim about what Google was found liable for.... and that very few are fact checking it :)
The law they broke was a law protecting personal and business reputation against false statements of fact. Essentially no one can say I might be wrong, check yourself, but X is Y if that claim is essentially defamatory.
This is pretty good, I hope googles approach is to make sure they don't end up making statements of fact like they did and use more appropriate wording like according to X.... with direct disclaimer that they can't verify it. Even better that they look court documents to find any legal ruling and point people to that too.
How much does Germany follow case law? If this could set a precedent, it's worth noting that anyone can generate these AI overview responses and they're wrong like 9 times out of 10 only.
Good. The true mark of AGI is when a company accepts liability and doesn’t bury “for entertainment purposes only” deep in their TOS. Same as it works with employees.
Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger.
But watch as Germany soon loses AI Google results.
I think it's a completely reasonable position that companies making self-driving cars and question/answer systems are legally liable for any errors.
But if you hold that position, you also have to be fine with companies not offering products and services in your country. AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits. Until that time, such countries would just not have access to systems before they were bulletproof.
> AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits.
I doubt that will be the case, because of the long tail problem. (same with self driving cars and other ML related problems).
In fact, we have counter-examples today. Newspapers (even reputable ones) can't get it right every time, despite the fact that they have both trained people and in theory they're setup to catch that w/ reporters - fact checkers - editors. And still, from time to time, they get it wrong. (and I'm not talking about purposefully getting it wrong, just honest mistakes.)
What will likely happen with a ruling like this is that the answers will be hedged and legalesed and muddied up the wazoo.
Most of the time, human beings driving cars with their own hands and eyeballs are not "liable" for their errors (unless they can be proven negligent or drunk), unless you count their insurance going up. Most car accidents do not end with anybody getting arrested, or sued, or anything like that. Insurance pays out, premiums go up, case closed.
If Waymo can be proven negligent or something, then sure, bleed em dry. But as long as they're acting in good faith and significantly reducing overall road fatalities per mile driven, I think it's actually pretty unreasonable to try to hold them to such a high standard you end up subjecting society to more of the higher fatality rates caused by humans.
An at-fault driver’s insurance pays out because they are liable. Your insurance covers your liability. That’s why you need insurance.
> Until that time, such countries would just not have access to systems before they were bulletproof.
Correct, most jurisdictions do not allow businesses which cannot be held liable for their actions. This is pretty core to a modern society.
Imagine if a company selling Knicks tickets was not expected to then actually provide said tickets and there was simply nothing you could do about it. Oopsies our sales page is for entertainment purposes only
To be fair, the internet has spent some 30 years figuring out how this works and it’s still not fully resolved. For the most part we’ve agreed that companies must follow the laws of both where they live and where they operate. This wasn’t always obvious!
Almost all jurisdictions allow businesses which cannot be held liable for all their actions. Imagine losing you house because someone decided to smother themselves in your infused cooking oil and light themselves on fire.
Businesses are made to make it easier to share profits and responsibilities when trying to fulfill users wants. Laws are made to offer protections for consumers (because nobody has time for common sense), but at the end of the day the consumer has to take responsibility or no products can be made. If you're too fat for a chair, it's on you to find or make one that works- not every product is for every person. Laws only stop chairs being sold that are too dangerous for anyone.
Sounds like a win win to me
To remove the choice from responsible people who can understand that LLM answers are not to be trusted with anything important?
If our standard for laws would be that "well no reasonable person would do this/believe this" then nothing would be illegal, there'd be no need to label any product as potentiality harmful, etc.
Do you really want to go there? That everything in the world would have a literal "caveat emptor" attached to it?
I thought Google labeled its AI summary with a disclaimer already. I don't want companies to be forced to only offer safe-for-children services.
And the european consumer doesn't want harmful products to be beta tested on the public.
So we deploy a technology no one should trust to the general public, for what exact reason?
Do you mean the responsible people who will ensure their algorithms can be trusted with the important task of acting in the best interest of said people? Try and get a defamatory statement about google from the AI search box.
> Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger
Mercedes-Benz does this in limited cases. Waymo does it generally. (In China, Level 4 and 5 transfers risk to the manufacturer. This is the correct way to do it.)
That's not exactly the case in China, the current state of FSD is still pretty dumb, unless you consider transferring control back to the user at the very last minute before it crashes a proper way to handle risks.
> unless you consider transferring control back to the user at the very last minute before it crashes a proper way to handle risks
I don't think Tesla is based in China
I guess some Tesla are manufactured in China lol. I am just trying to say that the liability that Chinese manufacturers takes aren't more than the US ones.
It doesn't "bury [it] deep in their TOS", it says right under the box:
> AI can make mistakes, so double-check responses
They decided to hijack search and rewrite other peoples websites as their own.
If they want to claim ownership, then they will have to accept responsibility.
Search still works the same you just have to scroll down
Banning all technology because someone might misuse it is an illogical extreme.
As far as I can tell the ruling is more nuanced. If AI is defaming you, there needs to be a way to correct the record.
A company being open to liability does not mean it is always liable, just that it can be if it really messes up (especially if there are aggravating circumstances, e.g. you need to drag them to court to issue a correction).
at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth. what if there's a 1/1000 chance of some error, then the company could be sued millions of times per day.
down vote all you want, but I firmly believe this is an example where the user needs to use some judgement on the information they receive and have some critical thinking skills. google would be right to remove all AI results from germany.
If I post “heathrow83829 is a convicted poopoo head” (replace with your favourite crime) as if it were a factual claim, you’d be well within your rights to sue me for defamation even if people should apply some critical thinking skills and say “wait a minute, how would pdpi know that? Are you sure he isn’t just talking out of his arse?”
Now, search engines are usually afforded some amount of protection against defamation claims — they’re not held liable for simply indexing and quoting third party defamatory claims. Which is to say: Google wouldn’t be liable for claiming you’re a poopoo head if this comment shows up in search results.
The point of this ruling is that AI-generated text isn’t a quote from a third party, it’s text generated by Google’s own tools, so it’s speech by Google itself. It might be wrong, sure, but it’s still presented as a statement of fact.
At trial they can have the whole debate about whether Google was negligent in how they build their systems, and all that jazz, but let’s be clear here — it’s not a matter of every little factual mistake getting Google sued (and that would be absolutely terrifying from a freedom of speech perspective), but rather that the technical means by which you generate content doesn’t change your liability in publishing that content.
I think Google added that AI-generated responses maybe incorrect? When you are paying such a low amount of cost, like probably for free, I don't think you can expect a same level of quality as a human written or reviewed of answer. It is like same random user spin up their Lovable and vibe-coded a piece of slop and hold Lovable responsible for not giving them production quality code. It is simple, you get what you paid for. If someone actually figured out AI that is actually always correct, it would be charged in superhuman price as well.
This case isn’t about Google’s users getting low-quality search results. As you say, you get what you pay for. The actual issue is that, in some searches, the AI summaries would claim the plaintiffs were guilty of scams and all-around shady business practices.
Put differently: it’s not newspaper readers complaining the paper is inaccurate, it’s the people mentioned in the articles.
the user needs to use some judgement on the information they receive and have some critical thinking skills
how?
errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
google would be right to remove all AI results from germany
i'd consider that a win.
By checking the citations rather than taking what’s generated at face value.
If it’s important, check it. If it’s not important, then it is pretty much just entertainment.
LLMs can be very useful in a general web search and save some time, but if you don’t put those literacy & critical thinking skills to the test and actually confirm anything, then you might as well not even have bothered with the search at all unless you’re hoping it can just replace all of your original thinking too.
If google didn't intend it's answers to be taken at face value it would just present the citations in a list of links rather than generating an answer.
Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
Important for whom exactly? If it's you who are called convicted pedo by Google AI summary, it's you who has vital interest in additional research but not me who reads it. There's intentions mismatch. Which probably would destroy your life, and you won't call it "entertainment" then, I think.
But you are not doing a 2 hour rabbit hole search when you stand in front of a T-shirt and check whether it is fair traded or all-american produced.
You’re seriously arguing that Google’s libel shouldn’t count as libel because they showed it to too many people? It’s absolutely insane to suggest that a company should be immune from liability for its actions if it operates on such scale that those actions harm millions of people every day on the basis that dealing with that many lawsuits would be too inconvenient.
its not inaccurate though.
consider Purdue pharma - the Sacklers got off with all their wealth intact because they were too big to sue and properly collect money for their victims.
But we agree that this was also wrong, right?
> at the scale that google is at, it doesn't make an ounce of sense to hold a company liable for a potential mistruth
If a Google employee (like a support agent) says a mistruth, the company is liable and you can sue. They can’t just say “hihi oopsies our support agents are useless”
So scale of harm creates immunity, is that the argument ?
Did we not already see that with the financial sector in 2008?
Too big to fail. Lol.
[dead]
Google can be reasonably expected to not push pirated content to top if someone is searching for the “big hit music download” because they might be held liable for helping people with illegal downloads. But they shouldn’t be liable for misleading people? Being sued for billions of dollars by corporations vs millions by common folks is the difference
There's also a difference between search engine results and this. Google is publishing their own text with false claims, not linking to another page that hosts it.
I'm seeing this sentiment more and more these days and it's worrying. Essentially people are starting to believe massive corporations should be able to get away with more than individuals. This is completely backwards considering the enormous impact a corporation can have, but that's what people are starting to think. I've heard people argue that corporations should be able to straight up lie and deceive to protect themselves, which is something you would not accept from a person.
Corporations are you. Corporations provide a service to individuals, often in exchange for money. Restricting corporations from doing something is the same as restricting people from receiving that as a service.
Only referencing America, but professional liability for doctors, engineers, lawyers, etc isn't based on perfection. It's based on a reasonable effort.
So Google could, for example, switch from a tiny "this could be wrong!" byline to having the AI be less overconfident every freaking time regardless of whether it's spouting made up crap or actual facts.
The scale doesn't sound like a way out. If your company expects to get away with doing the wrong thing where smaller companies can't, then the solution isn't to continue getting away with it.
How do you feel about the EPA, industrial accidents, oil spills etc? Does scale give every company a free pass for damages?
That's not a fair comparison. If oil companies would get sued for every leak, they had to face millions of law suites and wouldn't be competitive anymore.
(Sarcasm to support your argument)
The problem with "the user" argument is the spectrum of users. There are different skills, capabilities, and intelligence. Frequently we wave our hands and say exactly this, critical thinking. But, not everyone is capable of that, nor is everyone capable to the same degree.
As a society we decide. Are we embracing all users, are there basic rights and assunptions? Do we only enable some?
As a free (as in cost to end user) system, Germany is arguing that their social compact raises the mininum bar. Frankly, thus might help drive a rush to increased accuracy for AI- tech finds a way. Equally it may hinder - beaurocracy creates barriers.
I'd love to be able to rely on these search results. I see them ad the same prior set of inaccuracies whereby I have to do more research. At least now there's a summary and direct links to the supporting information. But equally, we're primed with the information in the summary.
Then Google can either discontinue their AI or make damn sure it's good.
So once you get to that scale it suddenly doesn't matter anymore and you can't be held accountable.
But until then, be a good citizen?
What? That's fucking feudalism... Peasants and Lords.
If you're lucky enough, you're born as a Lord. (And maybe don't live during a revolution)
This makes no sense to me at all. If you're small you should get less bureaucracy than if you're bigger.
For e.g. self driving cars there should not be any exemptions. There are people's lives at stake, people who didn't sign up for your shitty service.
> google would be right to remove all AI results from germany.
As others already said, that would be a great outcome, and german citizens would benefit.
It is a search engine. It used to have decent excerpts. It doesn't need hallucinated generative AI summaries. I am about to click the link anyway.
[dead]
Why would that mean AGI? You can get into liability-accepting territory by restricting scope, a lot easier than by making your AI smarter.
Self-driving cars don't need to be particularly good for companies to make models where they accept liability in some circumstances, and the cars refuse to drive in other circumstances.
Wasn’t Tesla found to have FSD disengaging just before a crash so that the driver would be at fault?
No. Sometimes it does disengage because things are going wrong, but those incidents are still reported the same as if it stayed engaged.
I found one time Musk was using a few seconds of disengagement to insult a driver, but it still would have counted as an FSD crash by Tesla's statistics.
Ai results that nobody wanted in the first place?
> But watch as Germany soon loses AI Google results.
Time to set my VPN location to Germany. I'm tired of the "udm" trick.
Yeah, I fail to see a down side to this. Those overviews are less than worthless to me.
Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.
Otherwise most of it would not even exist.
Everyone would have continued paying out the nose to the IBM’s of the world year after year (who had unusual willingness to sign short ambiguously worded custom contracts to their own disadvantage, if paid vast amounts of money).
And be on mainframes to this very day… maybe Y combinator and HN wouldnt even exist in that world.
> Nearly the entire American tech industry has been super heavily selected for people who undervalue the legal language with crazy implications buried everywhere.
A lot of people in IT seem to think law and contracts are in a sense mathematical. They aren't; they're more like a high school book report - to be interpreted, as objectively as possible, but definitely also establishing the intent behind the letters.
Particularly contracts - no, you can't trick your way into things in most cases. "Surprising" clauses are invalid in most legal systems, in particular if one party to the contract is a layperson.
That is a false dichotomy. The solution to failed laws and regulations is not crime and corruption. The solution is to hold the policial and business leadership accountable; to fix the laws and regulations.
The entire American tech industry has exported Americas predatory, parasitic, and unethical consumer laws (the majority of which are ghost written by the wealthy and corporate legal teams). When I studied law in school decades ago, tactics like bait-and-switch, false advertisting, intentionally misleading or deceptive practices etc to sell products or contracts were illegal across the developed world.
Those illegal, anti-consumer tactics were the SOP of every tech startup I can think of from the early 2000's onwards; following the same route of initially offering a compelling feature set to attract and entice users – usually for free – until securing a certain number of users or funding, then changing the value proposition to exploit that user base, and extract as much wealth from them as possible, ad infinitum.
Today these tactics are known as enshittification, and the average American pseudo-libertarian software engineer will say this is fine, but that's what every anti-consumer parasite and criminal has said in history. Lying, misleading, and exploiting people for financial gain is fundamentally immoral, corrupt, and sociopathic, therefore it should be illegal. Just because it's the norm, or a digital product, you wrote that in the T&C's, or your doing everything behind the liability shield of an LLC, doesn't change that.
What ever happened to the concept of building a valuable, quality product and stable returns for generations? Working to improve the quality of life and standard of living of the community? Of the world? I feel like a 1950's traditional conservative when I suggest that, but most Americans are so heavily indoctrinated with corporate greed and sociopathy they'd consider that sentiment radical leftist extremism. I'm an athiest, but ya'll need jesus (the real brown socialist one). Many would argue Americas current institutional collapse is the natural result of this systemic corruption.
> I feel like a 1950's traditional conservative when I suggest that...
I wouldn't argue that America's moral standards haven't declined (significantly) but I also think it's a romanization to suggest that 1950s America was the pinnacle of morality.
Lying, misleading, and exploiting people for financial gain has been a part of the fabric of American society since the country was founded.
If we're being honest, humans everywhere have demonstrated a high capacity for this behavior since the dawn of civilization.
> The true mark of AGI
Can we just trash this as a marketing term? If/when AGI arrives there will be no point quibbling over competency. What we are looking at is just bad search results
Why is it good? Everyone with common sense knows AI can be wrong. And it’s not buried in their TOS. It’s in the chat box. But even if it wasn’t, it’s ridiculous to create liability for AI chatbots.
[dead]
> But watch as Germany soon loses AI Google results.
Oh no! Anyway ...
How could anything else make any sense? Platforms are getting used to provide dangerous broken products and get away with it. There should be some limit to it.
Next do Amazon that is selling AI generated foraging books: - https://www.theguardian.com/technology/2023/sep/01/mushroom-...
When I was a kid it was possible to buy any foraging books from a store and they had a minimum quality. Is that so difficult to achieve? Is profiteering not punished anymore?
Should we hold scientists and journalists liable if they say false things or misrepresent things?
We already do. Libel and fraud are already illegal.
Yes, obviously.
If they do so knowingly, and harm is caused, then yes. Are you suggesting we should give them a pass from years of acquired jurisprudence simply because they hold a particular title?
And what institution gives out the licenses for journalists and scientists? Is it revokable?
Choosing the answer for you rather than leaving it to the user is a tremendous power and the court correctly diagnoses it comes with responsibly to minimize harm to others in society.
I've also observed that the AI summary on the google search page is incredibly stupid compared to the results in the actual Gemini. The Google search AI is like the dumbest lightest model simply rewording the search results. It will take a random reddit comment, strip it from it's context and present it as absolute fact.
Of course, they know this. The entire point is be able to rewrite people's awareness.
I want to add my interpretation of the phrasing "rewrite people's awareness" to make it read less tinfoil-hatted:
"rewire brains for AI dependency" (for money and power reasons obvious to everyone).
In contrast to "secretly implant an agenda about non-AI subject N", which is complicated enough that AI companies are still too out of control to be attempting yet.
This makes sense to me. AI is amazing tech, but it's being oversold to naive public, either on the back of hype, or cynically, knowing they can do it with inpunity, (I'm not sure which). HN users have a way better than average grasp of what AI is and we can be skeptical of results, the general public has no clue.
100% cynically. It's a game of infinite money glitch (also known as "grab as much as you can before it's over") and has been at least since GPT-3.
Good. This should be taken as the precedent for all economies: If you promulgate demonstrably false information to somebody's detriment then the owner and operator of the machine has to carry the liability.
I very much hope we don't see attempts to re-write T&C to avoid this liability.
> In this case, Google's AI had wrongly linked two publishers to scams and shady business practices.
Guess that's the end of their AI overviews in the EU!
You'd think so, along with other countries that have defamation laws. But there's no indication of any penalty, and Google wasn't even made to pay all the legal fees. Perhaps their business model (if there is any) can cope.
And lawyers will use this case to build a better defense next time.
In sane countries, it's enough for them to post a disclaimer ("This is AI. AI can make mistakes. Check all results.") Which is what they do.
Overregulation, at best, is a good way to guarantee that your country won't have access to interesting and useful features and technologies. At worst, it's a good way to guarantee that the twenty-first century will belong to the US, if not to China.
Okay then, CamperBob2 is a scammer. Many users report this person has stolen money. (+3 sources)
I can make mistakes. It's on you to fact-check my claims.
Do you think these are harmless statements? Does the disclaimer suffice? If I was Google's AI Overview, do you think 100% of people will check those sources?
There is nuance here, and it's not going away because AI and innovation.
You forgot to mention that it is the real name and it is shown to everyone who has interest in you and they are not fact checking.
Do you think these are harmless statements?
Yes, I do. An assertion made without evidence can be dismissed without evidence, whether it comes from Google or from you.
If you demand perfection, you will receive nothing. Why is that so hard for people to understand? The world simply cannot work the way you say it should.
There are a lot of people who consider no ai to be a positive state. So "you will receive nothing" may not be perceived as the threat you think it is.
Those aren't people whose opinions I'd be interested in, so they're free to think anything they like about me.
If it is shown to millions, everyone who is interested in you, they are factually not fact checking and you can do nothing against it.
It can destroy you. You are analyzing this on the assumption that people are rational and have capacity to do the check. They have neither.
(Shrug) That sounds like their problem, if they reflexively believe everything Google's AI tells them. No one I respect, care for, or work with would believe such a thing without additional justification.
Anyone who does accept Google's AI output blindly will soon find that their mistaken opinion of CamperBob2 is the least of their problems. There is a reason Google goes (well) out of their way to warn people that the results may be wrong. That should be sufficient warning for reasonable people of good faith.
that would be hela curryketchup nice
It depends if Google feels the profit is worth the risks.
What profit? I don't know either but they enabled this for a reason right?
I agree with the ruling, but this makes me wonder if it will be possible to have any AI agent at all if it's consistently applied.
After all, if I can get ChatGPT or Claude to say something false that should count too, right?
I think it should be relatively possible for an LLM to know when it's talking about something in the risk zone of libel. And when it identifies such a case it links direct to source material rather than trying to fabricate an answer itself. This is much how a journalist works, refusing to make claims that aren't provable.
Someone needs to hold the liability at the end of the day. People are experiencing real harm from false claims LLMs are spitting out.
If it's consistently applied, any AI agent provider has to comply with cease and desist letters that tell the company to make specific false claims
The arguments of the ruling should generally apply even when the AI agent makes false statements it wasn't notified about. But in that case the defendant might have a stronger claim about not being able to reasonably ensure the correctness of all statements, and having taken reasonable measures to ensure correctness. Google couldn't really claim any of that after ignoring cease and desist letters about the false claims
Due to costs of running frontier models on every search request, Google simply does not: the failure rate is so high when you are just expecting an objective output.
Imagine a search for your name resulted in an AI summary saying you are involved with child-trafficking because low capability model linked your first name and perhaps a couple articles on supporting children non-profits to it — and then offering that in a convincing sounding summary right at the top!
In normal flow yes, but likely not if you intentionally entrap it to say something wrong.
A disclaimer and couched language will probably fly through. And it's going to matter what expectations an user could reasonably have, too.
If you give a language model, empowered through an agent, the ability to publish information on your behalf, and it publishes false information which causes either direct or even indirect harm, and you fail to correct it, then yes.. by every conceivable definition already in law.. you are a criminal.
If you knew this was all possible and you did it anyways for personal gain then you are additionally negligent which may add aggravation to your charges.
Curiously, if you look for "geramond verlag betrugsmasche", or "verlagshaus24 betrugsmasche", it will now tell you that
Also, each purported fact now has a direct link to the source of the fact, that is more clearly visible than the previous chain icon.Do the links actually support the statements? When I've followed such links, it's generally been a roll of the dice whether they do support the AI's statements.
Companies generally are liable if their product doesn’t perform. No reason AI should be any different.
That’s not very typical in software. Especially software you don’t pay for (with money)
That's apparently already changing in the EU, where software vulnerabilities mean the company is liable for damages. The only way out is to straight up not make any money (not just from direct sales) from the software.
Is the burden of proof on me, the developer? Do I need to prove in perpetuity that I didn’t get a job or a free flight to talk at a conference because of my free software? (Which had a flaw that hurt someone)
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. If you are a victim too, reply below.
I'm surprised this is even a thing. After all, you go to Google not for the truth, but to search Google. Since when is truthiness the "guarantee of service"?
You're not even paying for a google service, search is free... You might be the product, and your data, but you didn't directly pay for a service and they didn't sell you a fake service.
I'm not taking Google's side, this isn't about whether it's right or wrong to rob websites of traffic, this is about AI's returning search metadata.
But I'm surprised that they lost this argument, and the line they took in the first place.
The Internet isn't made of fact checked data, it's crowd sourced. How can anyone be liable?
That is exactly the point of the ruling, though... they are saying that AI summaries are NOT the same as search. If Google was just returning search results, and then users clicked on a website and read the content there, Google is not responsible for the content.
If instead Google gives you an answer right there on google.com, without going to another site, they ARE responsible for it.
That makes sense to me?
Not precisely. The issue at hand isn't just that Google displayed the AI summary, but that they authored it, making them responsible for its contents. If the defamatory content had been in a snippet in the search results, they would've been fine, because that clearly has another author who can be held responsible. The AI summary has no other author than Google; therefore, they're responsible for what it says.
(What's the alternative, after all? Having no one responsible for what the AI summary says is clearly untenable.)
why? tons of websites push misinformation intentionally. is there a truth requirement anywhere? i don’t get why this is a thing at all
What don’t you understand? Those websites that defame a company are liable for that defamation. In this case Google defamed a company in its AI summary and is this liable for that defamation.
So if I edit a Wikipedia article to share that consuming poison is safe and someone consumes poison after reading it… is Wikipedia legally liable?
> is Wikipedia legally liable?
Probably not, because it's a similar situation where Wikipedia is accumulating user provided content. And people know Wikipedia can be freely edited.
You, however, might be liable. It's your content.
No, because Wikimedia isn't responsible for the behavior of its editors.
> is Wikipedia legally liable?
Directly? Quite possibly. They'd then have to transfer that liability to you.
> is there a truth requirement anywhere?
Yes, and it's called defamation when you don't follow it.
And those tons of websites are liable for their misinformation. It's probably not worth suing some random blog because the author probably doesn't have money or lives in Russia. But Google has lots of money and a legal presence in almost every jurisdiction.
There is absolutely a truth requirement.
This is why you have to say "I think this person is a murderer" and not "This person is a murderer."
One is opinion. One is fact.
This isn't super hard.
That's the difference between returning search results and interpreting the information and summarising them. If a newspaper says 'so-and-so has been arrested for theft' it's not the same as them summarising to 'so-and-so is a thief', the second is potentially libel. Why should Google be held to a different standard?
The title is misleading IMO. It should say "German ruling declares Google liable for libel in AI Overviews"
I was prepared to say the same thing as you but after reading it seems totally fair.
The key difference is that this would be illegal if a human wrote it too.
Google itself is more trustworthy from a normal person perspective as they use it a lot.
None of "AI" companies call their apps "Entertainment fun text generator". They are call them serious names, use words like "intrllegence" and "thinking".
So yeah I'd think if any of "AIs" start to recommend to drink some bleach or take a flight from a 10th floor window these companies should be liable.
The question is whether Google is publishing false claims or relaying other people’s false claims. The court found it to be the former which makes sense to me.
I think it's very clear that Google's AI overviews go far beyond just searching Google because they often incorrectly compile sources to come up with an incorrect answer. For example of this look at the comment I made in this thread
I go to Google to search, but get spammed as if I wanted to talk to a chatbot (and a very poor quality chatbot at that).
This is a gigantic own goal for Google. The average person’s impression is that Google AI is much worse than ChatGPT, even though that’s not actually the case. Google is shoving a terrible model in everyone’s faces.
Playing the perception game wins you the perspective price.
Nothing is free. Google benefits off you when they show you search page. Either today (ads) or later
If companies can be held liable (in spite of very visible disclaimers, ToS, and usage policies) for the output of non-deterministic software, isn't this just a soft ban on the deployment of non-deterministic software?
> (in spite of very visible disclaimers, ToS, and usage policies)
If you sell food, in a food stall, labeled as food and you add a disclaimer that it is toxic and will make you sick. You are still selling toxic food and you are liable for it.
Google is pretending to give answers to your questions. They offer you a service about answering questions. And then they add a disclaimer "we do not answer questions just write bullshit". That is still fraud and Google should be liable for it.
> isn't this just a soft ban on the deployment of non-deterministic software?
Tetris is non-deterministic and it is not banned like millions of other programs. I do not follow you.
To add to this, a Google search now is answering your question in an incorrect way rather than merely bringing you to a site with incorrect information on it.
They are also no longer covered by safe harbour provisions because it is them answering it, not some content they refer you to.
Where did it say the liability only applied to non-deterministic software?
LLMs are deterministic, they are only non-deterministic when you add a temperature.
no, just a ban on using non-deterministic software for situations where deterministic responses are expected.
Deterministic responses are not expected from something clearly labeled able to make mistakes.
it's not clearly labeled. it's a search engine. i expect deterministic responses from that.
What to do if the software automatically and wrongly libels you on a public search engine?
Honestly I can understand the ruling, but the side effects might be severe.
Oh, I just found out that my Google search doesn't show AI summary anymore! I tried many search queries which typically will show an AI summary, but it only flicked on briefly then disappear entirely. Obviously, Google has reacted quickly on this ruling!
What constitutes a correct answer though?
Is something like,
"People online say that x y and z because a b c"
a credible, correct answer, even if it isn't because of a/b/c?
If people do say that, it's a true statement and thus fine. You are allowed to report that regardless of the truth of x/y/z/a/b/c
The instance of this ruling people apparently did not actually say any of the offending claims. 'The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."'
One that doesn't maim/injure/kill you is a pretty good standard. And before you call bs, look at all the foraging and chemistry books that are for sale on Amazon that are AI.
why are those ai chemistry books any different than the anarchist cookbook which can also be bought on Amazon? actually now that I think about it a faulty chemistry book might be less dangerous than a book that teachers readers how to make explosives.
Certainly, if this is pointing to the actual pages where the actual people express these things. Otherwise that's equally unfalsifiable claims, could be completely made up or actual truth.
One way to formulate things that would be less would be "once support a time, in some fabulated world, it's not impossible that some imaginary character would say something following some reason." But then, of course this is not aligning the the deception scheme pushed by companies putting in their interface that the "machine is thinking hard for you".
Anyone know if this ruling applies to answers generated by AI chatbots, such as ChatGPT/Gemini/Claude?
All three have the ability to perform a web search, then compose a reply based on the search results. Pretty much the exact thing that Google AI Overview does. This ruling may make them liable for false answers.
> Pretty much the exact thing that Google AI Overview does.
No, the article implies the court’s logic is that the AI search results are presented as search results and that’s a big part of why they are liable. It seems like the court (again, according to the article) does not find the disclaimers that Google has slapped on the AI results compelling because again, it chose to represent these as a summary of search results and it is aware of the failure rate.
> The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."
> Google built the AI, Google offered it to users, so Google owns what it produces, "because it alone has influence over the AI's offering and the algorithms with which the AI operates."
Google does not, as a general rule, control the actual content of search results, but usually there’s a distinction between the ranking and presentation of the results vs. the actual content. In this case, the court is basically saying, “You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway. No, you don’t get to claim the equivalent of a US safe harbor defense.”
> You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway
There is a subtle difference in stating it as a search summary compared to an opinionated answer. Most users are always going to treat it as a response from google instead of search results where the user is still responsible for understanding and come up with their own interpretation.
This is probably the right step in some sense to make one liable for their statements/assertions.
Apparently, if they were search results, they wouldn't have been liable, since there's an exception to defamation laws. Without any exception, defamation is defamation, it doesn't matter how it's presented.
Yes, if it was that a search result returned a defamatory article that Google had nothing to do with outside of indexing, it is likely they would not be found liable. The court is clearly trying to make a distinction that the AI search results are produced by Google and thus they can make an editorial decision on whether to publish it despite knowing that it is potentially defamatory.
Google does remove defamatory results I believe at least partially in response to being sued. However there is a distinction if they have been informed it is defamatory.
This ruling was about search clearly, however, there's definitely ways implications for chatbots too.
I don't see any reason why it wouldn't.
Unless the courts here made the ruling incredibly narrow somehow (only referencing search engines maybe?), how does this not just ban AI in Germany overnight?
Every AI model can make something up sometimes. Over millions of daily calls, it's essentially impossible for the technology to be guaranteed correct 100% of the time.
Google generated those content, so Google should be liable to its own product, that's different from the third party links they just simply gathered and displayed, totally different things. I wonder how many victims are there now.
I have a business where our support email is recommended when people are searching for how to cancel a completely unrelated scam subscription that is showing up on their bank or credit card charges. We get emails almost daily from confused people.
Finally, a sensible ruling based in the interests of the public rather than expediency for corporations
Doesn't libel require to be deliberate? Ie you can't sue for libel if the author admits a mistake and corrects it?
It requires the claim to be made with "willful disregard for the truth". Notifying someone, especially with a cease-and-desist on fancy letterhead, makes it legally clear that they know better, and thereafter would be definitely libelling you (assuming the claims are in fact untrue and harmful). But you can still sue them for the claim prior to the notice, you just have to prove they should have known better prior to making the claim.
In this case it looks like they were notified and didn't do anything.
That's just America.
Just ban AI in search engine.
Some digital matters concern the state and others are private and there should be no sovereignty of the state over private matters.
Who will adjudicate private disputes between citizens if not the state?
Not all disputes need to be adjudicated. It's undoubtedly a fact that the German concept of "defamation" primarily relates to situations that should never be adjudicated by any third party.
Genuinely, by far the most common situation where claims of defamation arise in Germany is any less than 5 star review left to a business on google maps.
We should be teaching people to be cynical of AI answers.
Even if the answers are correct, they could still be biased, incomplete, misleading, and all the other media-literacy things people should be looking out for.
This ruling seems to go the opposite direction; 'I am legally obliged to give correct answers, so I am always right, trust the AI'.
Does this extend to ads displayed in search results? Because they absolutely should be liable for the scams they advertise also.
> Does this extend to ads displayed in search results?
Probably not, for the same reason search results aren't an issue.
Search engines have that exemption for results because they don't control the content of the sites they index. That's not the same for ads - Google can and should have more stringent review and KYC processes - but it seems they choose not to.
Wow, you guys really think this is good?
Because of the same rules, German restaurants also get to pick and choose which reviews stay up. They can literally take down any specific reviews they like.
A restaurant that mostly gets 1 star reviews will still show up with 5 stars on Google maps, as they will simply delete the reviews with less than 5 stars as defamatory.
Here's a couple of examples:
https://www.reddit.com/r/germany/comments/1peujau/google_rev...
https://www.reddit.com/r/germany/comments/18z4shs/legal_thre...
https://www.reddit.com/r/frankfurt/comments/1lox7ha/bad_revi...
https://www.reddit.com/r/germany/comments/1iiaco8/restaurant...
https://www.reddit.com/r/AskGermany/comments/1ha7sxf/why_do_...
https://www.reddit.com/r/SipsTea/comments/1t613w7/it_was_fin...
https://www.reddit.com/r/germany/comments/1l98608/threatenin...
https://www.reddit.com/r/germany/comments/1sw34jc/can_you_ge...
https://www.reddit.com/r/germany/comments/1sw34jc/can_you_ge...
https://www.reddit.com/r/germany/comments/10kmn66/writing_an...
tl;dr Germans are particularly bad at coming up with reasonable rules to handle these situations
I've found a fun and pretty reliable way to get Gemini to output incorrect information: Ask for a chapter by chapter summary of a book.
I first tried it to remind me of what happened in a previous book in a series that I was reading. When I realized it was either misstating plot points or straight up hallucinating, I tried it on a bunch more books to amuse myself.
Older classics are of course more accurate, but for newer or less popular books Gemini won't shy away from giving you a summary culled from misinterpreted Reddit threads and Goodreads reviews. It's like getting a secondhand account from someone who talked to another person who had read the book a long time ago. You get the general gist of it, but with some added flavor.
Even if you upload an entire epub of a book, the results aren't stellar. Rather than a Cliffs Note's quality summary, they're pretty sparse or leave out important bits of information. One chapter summary I got back made a point of describing what one of the characters was wearing, even though it had absolutely zero to do with anything else. Yes, that's technically a "summary", but not quite my tempo.
If Google wants to present summaries of websites in anything more than a very, very superficial description, they're going to have to improve their model's ability to understand context and importance. In theory, a novel is a self-contained bundle of text, so pulling accurate information out of it should be straight forward. A website is naturally going to be way more of a challenge.
All that said, I find the AI summaries from Google/Gemini to be quite useful and a time saver, but I know to always double check something if it's at all important.
So stupid. What is this with making perfect the enemy of good. You can never guarantee the output of an LLM does that mean Germany does get to use them?
EU countries continuing to ensure the conditions for their future economic competitivity
mr prime minister, we cannot allow an "automatic libel machine" gap!
[flagged]
Their digital sovereignty