I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 (https://www.ai.se/en/project/gpt-sw3) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by its people, basically for free. Nations should mirror what Cursor has done with Composer 2.5 for example.
Do we know for sure how much national corpus of knowledge (like dutch) goes into these "global" models and how that affects "localized" model biases? What's wrong with specialized models?
If open frontier models start closing up and states start more export controls on AI services and hardware, it might be good to ensure the supply chain is there to reproduce the SotA, or even a couple generations behind it.
And what happens once the "solid baselines" become unavailable for a reason or the other?
You keep building on the last available version? Fine tuning is a whole lot cheaper, easier and more useful than pretraining a model from scratch. It's a complete no brainer.
> You keep building on the last available version?
yes but a sovereign can allocate some resources and a few people to stay in the loop from a first principles level. No need to wait for a rug pull.
Of course, it can not compete with the frontier labs. But good to have researchers and professors "in-house". LLMs are here for the long-term.
[deleted]
Seems like you don’t understand.
You take current version and build on top of it. You have the weights.
You might not get some n+1 version at some point but the n version you will have will be still most likely much better than whatever you come up with burning good will money of people believing in „sovereignty”.
You are not getting ahead in this game by being „true to your local values” capital expenditure is insane in this game.
Kimi and Qwen come out of China, which means that their training material may be biased e.g. relating to Taiwan [1]. In addition, there is no way to determine what input went into the training, if it was properly licensed, if it was legal (e.g. not contaminated by CSAM), or how the human component of RLHF was sourced - in US models, for example, stories about exploitation like [2] have been floating for years.
Assuming us Europeans finally get our act together, I think it is better for our long-term future (and the ethical problems) if we manage to get a baseline of training input and data ourselves, from scratch, with everything being ethically sourced.
Oh and, while we're at it, the EU has 24 official languages plus a host of minority languages. Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best. An European model with actual funding and proper data sources might be able to significantly reduce that.
There is something north of 8% OCR error rates.. that will hurt model quality!
> Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best.
that is not true, so please read before make an opinion. The French Mistral project shipped seven+ years ago with 140 languages for example.. language translation was the first LLM task from 2015
Uh, some would say it's easy to determine what input went into the training for kimi and qwen.. since they were caught stealing it from American labs. Some cultural cliches may never change.
It's well-known that all commercial models are based on stolen content. That doesn't mean there is no filtering/censoring, just that the censoring likely depends on where it's happening…
> since they were caught stealing it from American labs. Some cultural cliches may never change.
Has a formal lawsuit been brought to bear? Given, Anthropic & OpenAI are being dragged through courts for copyright violation (or stealing, as you'd call it, if the companies involved were culturally Chinese) by newspapers, publishing houses etc; one'd think they'd pass on some of that medicine to Alibaba, which does have business entities registered in the US.
It is crazy that anything Europe gets so much hate. IMO it is important to build models within the boundaries of smaller nations, using their own language. Research has to continue even if it is outside of US and China.
It’s not that it gets hate so much as it’s akin to watching them make announcements that they’re going to make a European google/facebook/tiktok.
Sure… they can, except at the end of the day it’s a bit late, regulatory burden will make it comparatively useless, and because of that nobody will ever use it. It will be spending a bunch of taxpayer dollars for press releases.
The running joke is that when these “sovereign” EU models launch, they’re going to refuse to answer anything that might involve personal information such as Elon Musk’s birthday.
That’s on Wikipedia, it’s not PII, it’s also not going to be relevant to any meaningful IRL work.
I challenge the assumption you can do meaningful work in this field without blatant disregard for intellectual property.
The idea that it’s all down to training size is clearly incorrect, as every expert human learned their craft without nearly the sum total information of the internet. Clearly there are architectural wins to be found.
Besides that, why would everyone just be fine with Opus level AI at best, as that’s all the US is willing to export, and I doubt China will share beyond that.
Sovereign AI is more important than ever after Friday.
I kinda agree, the best use of taxpayer money should be in reducing taxes to corporation that would like to compete in the market vs US and China, rather than making governments playing the game (since they very obviously can’t).
So good to see these developments. Every country should do this. I'd even say every person should gave their own personalized AI running on their own computers. If only the costs involved were not so astronomical.
I think it will be cost effective at some point. Computers were limited to research institutes before the personal computer arrived.
I hope you're right. I really don't want a future where only corporations and governments have computers.
Nvidia will certainly be pleased.
Interesting that this got posted now: the project is receiving increasingly more skepticism lately in the Dutch tech scene [0], and I think that’s fully justified.
What is the exact skepticism? The only thing I could get from that was from some "tech entrepreneur":
> GPT-NL was never built to compete with Claude or ChatGPT. It was trained exclusively on licensed data, and is intended more for governments and companies where privacy and compliance matter more than raw performance.”
That's it? That it didn't aim to compete with SOTA models? Maybe this is something you have to start with something, then ramp up, rather do what only a select few labs been able to do, start with really big models. Especially if you're resource constrained, which since this is a government project, I really hope for the sake of the tax payers it was.
I mean if you are wasting funds kind of knowing it's nowhere near remote competitive, then it's kind of a fraud.
TNO is something like semi-DARPA. It gets a lot of stuff tax free and a lot of gov funding, but a lot of their budget is from getting businesses to hire their R&D teams.
They do really good R&D on a lot of stuff. This is just their attempt at public credibility/internal skill building to enter the LLM business.
Doubt its going to be successful, but they "waste" a lot more money on other things that you never heard of. Its not fraud, its just R&D dressed up a little too much too early.
But why is "competing against remote SOTA models on quality" the only thing that matters here?
What the hell else is there? All the other stuff can be done by an intern with an 8 euro HF Pro subscription.
Other than actual research, which is in a different camp.
Common approach I've seen is having workflows with paid/larger/hosted models for some workflow where you don't quite know exactly how it'll be when you first put it together, then with time you've locked down how things more or less work yet you still need free-form text parsing of some kind, so you end up replacing the bigger models with carefully post-trained small models.
Besides that, there is a ton of use cases for smaller models for a bunch of different things. We'll be unlikely to be able to run LLMs (actually Large) on smartphones for a while, while the smaller LLMs seem to run already on-device in experiments.
Targeting a niche audience with specific requirements is not fraud.
[deleted]
I don't understand countries (especially governments) wanting to have their own models when there are already pretty solid open source (weights) models out there.
Countries should want control over _where_ the compute is happening rather than _what code_ is running.
What's wrong with a country hosting a Kimi, Qwen or GPT-Oss on their hardware for their government work purpose?
It is not about the country but the language. Most llms have poor or no support for Dutch.
Idk which models you refer to, but I tested a bunch recently, and they performed well on Dutch. Only the smallest, such as qwen 3.6 27B, made up words and switched languages.
There would be a bunch of value in having, say, a good 30B-class model that used my local language as well as it does English. There's lots of cases, especially in the government sphere, where local processing is a requirement and frontier-level capabilities aren't required. Making those cheap to run seems like a fine goal.
Can you provide some examples of these use cases?
There's a large gap between making up words and an actually native text distribution. LLMs have a clear pattern, clear tells, a "feel" in English, and it's normally even more pronounced in non-English languages.
Lots of bias towards English sentence structure, idioms, etiquette, etc.
I don't understand this. Even if that were true (and it isn't in my experience), a model that is trained on a Dutch corpus and arguably "knows Dutch well" but has the reasoning and comprehension abilities of a three year old is useless in any case. I'd rather use a model that can only speak English and put an automatic translator around it.
An LLM is an encoding of a culture, a way of viewing the world.
They are not neutral technology, they are a direct representation of the training set that has been chosen and how they are aligned.
In many ways, they are ideology made code.
If we leave building them to the US and China, only their way of seeing things will be digitized.
I don't like the idea of that.
Yes and also, US and Chinese models are censored in different ways. US models are way too prudish for personal use in Europe because they're afraid to piss off religious investors. Chinese models are too censored on history and current affairs, eg the tiananmen massacre never happened stuff like that.
Really? Because I'm pretty sure that at least every two days there's a active post with a top voted comment along the lines of "The EU isn't doing AI themselves, they are so hosed".
Why should Dutch people be expected to make do with models 99% trained on American/Chinese cultural context and language?
Oh, it's all fine with cultural context here -- we don't even dub English language movies here because we are that cheap
Understood, but they could fine tune base models on their own cultural context and language. Why reinventing the wheel?
I thought finetuning data can't contradict foundation models, and anything that are inconsistent with the standard LLM American-Chinese split personality would be rejected?
Fine tuning happens on top of pretraining, so of course it can "forget" pretrained defaults when warranted by the new data it's being fine tuned on.
But you have to have more data than used for pretraining for the added knowledge to take precedent over pretraining, no? If that would be the case, you practically contradict the knowledge in the base model.
I mean ... LLMs are sort of an extreme and living proof of linguistic determinism. Their behaviors are dictated almost entirely by disorganized language data, primarily English and Chinese. So you can't just add a language as native primary language in a quick post training, I think. There's no way that it would work.
They could apply the Polder Model of consensus decision making with a mixture of experts.
Funny, that's what I thought when PewDiePie set up his monster AI rig and what he called a 'council'. Quote:
"PewDiePie has built a custom web UI for self-hosting AI models called "ChatOS" that runs on his custom PC with 2x RTX 4000 Ada cards, along with 8x modded RTX 4090s with 48 GB of VRAM. Running open-source models from Baidu and OpenAI, PewDiePie made a "council" of bots that voted on the best responses, and then built "The Swarm" for data collection that will become the foundation of his own model coming next month."
This gets better short-term results for a fraction of the cost, for sure, but what do you when China places an export control banning the release of open weight models? If you don't have your own talent, you're then relegated to using a base model from 2026 or whatever the cutoff date is, forever. That defeats the purpose of a 'sovereign' model made for and by your people.
>Countries should want control over _where_ the compute is happening
Yeah but Europe doesn't build any computer hardware, and EU Green eco-communists and NIMBVYs don't want to have data centers built in their backyard, so the only way left for EU consultancies to milk taxpayer money for the AI bubble, is shipping a sovereign AI model for each country/language.
Watch out US tech sector, we're coming for you. Feel our wrath.
Have you heard of ASML? NXP?
Ignorant comment
Please don't move the goalposts. What computer parts does ASML or NXP make?
ASML only makes the lithography machines, 85% of which go outside the EU (let that sink in). And then fabs in Taiwan, Korea or the US use those ASML machines to etch US IP for computer chips. EU doesn't make any computer parts domestically.
And NXP mostly makes various microcontrollers and small chips, not high margin IP decenter centric parts like ASICS, FPGAs, CPUs or GPUs.
So not only are you the ignorant one here, but you also have the audacity to insult others with so much confidence.
@dwa3592 below. Firstly, why are you moving the goalposts in bad faith again just to stir an argument? What does that have to do with my original comment?
And secondly, there's other lithography machines out there, not just ASML.
And thirdly, the IP Nvidia, AMD, etc develop to etch on silicone via ASML machines makes them more valuable than ASML.
Fourthly, repeating my "let that sink in" phrase is just childish and low-IQ trolling, unworthy of this platform.
>>ASML only makes the lithography machines
Woah! only lithography machines???? it is literally impossible to make any device capable of running anything close to AI without ASML. Let that sink in.
Funnily ASML owes its current success partly to US funded research (straight from Wikipedia):
> Two years later, it joined a consortium, which included Intel and two other U.S. chipmakers, in order to exploit fundamental research conducted by the US Department of Energy. Because the Cooperative Research and Development Agreement (CRADA) it operates under is funded by the US government, licensing must be approved by Congress.[12]
Why are you acting childish and petty? I said EU hasn't got AI compute manufacturing(aka no equivalent IP to Nvidia and AMD and no equivalent to TSMC or Samsung fabs), not that it doesn't have lithography machines manufacturing.
Surely you understand that while you can have the latter, you can also lack the former.
In a recent podcast, it was summarized as:
ASM (International) makes machines that add material to a silicon wafer (deposition).
ASML makes machines that remove material from said wafers (lithography, etching)
(I was a bit surprised that's not combined in 1 machine. But let's move on)
Then Besi makes machines to stack / interconnect / package those ICs into a package. I'm assuming pick & place machines are other companies' turf.
The above are all Dutch companies, operating a pretty important section of the tech stack.
Iirc there were (& probably still are) some IC fabs in Europe, but mostly older nodes (like useful for microcontrollers used by car manufacturers. Wikipedia has a list). So for SOTA smartphone SoCs it's off to Taiwan (TSMC), South Korea (Samsung) or China (who makes everything, including smartphones & the chips going in there).
So as far as EU goes, the capabilities are mostly there. Skilled workforce? Check. Money? This is a rich continent.
What's missing is the guts to say "hey, let's dump €100B into this & make ourselves some laptop & server CPUs!".
But now the important thing: several of such initiatives are starting to bear fruit, and b) confidence that EU can do such things, is growing.
As for bureaucracy / red tape... sigh... (won't be fixed any time soon)
Most important machine ... built on US IP, subject to US export restrictions, used to manufacture high value US IP, in factories outside the EU, so profits of those chips goes to US. A point I have addressed over two times already.
Also ASML even threatened to leave the NL if the Dutch government doesn't do what they want on taxes and labor policies. So having only a single card to play that EU can loose at any time, it's not putting EU tech sovereignty argument in a good light.
The "wahabout ASML" that keeps being spammed by people here, isn't proof of EU compute and AI sovereignty. It's the exception which is why it's the only thing people can mention on EU tech and they DDoS you with it as if that changes anything.
Are people here that petty that they can't stay on topic and argue in good faith and instead need to hijack your argument to go on offtopic whataboutism for a cheap gotcha spamming "whatabout ASML" on unrelated arguments?
>ASML only makes the shovel making machines
>>Yeah but Europe doesn't build any computer hardware,
Well, then this is will be a good start.
EU bureaucrats are too busy trying to keep the welfare/pension system from collapsing, defeating Russia, supporting Ukraine, managing the fossil fuels energy shortages, figuring out how to nerf Chinese EVs while supporting domestic car companies, and restricting social media free speech to make sure the "far right" don't win elections.
So of course, semiconductor manufacturing sovereignty is very low on their priority list.
That's what bureaucrats are supposed to be doing BTW.
How many in that list of things is the US also doing?
The US is a single country. Russia is not on the US' doorstep. The US has its own oil. The US prints the world reserve currency.
Different scale for those problems. Way different scale. US is monetary rich, oil rich, energy rich, manufacturing rich, and doesn't suffer from russian aggression at its borders. US is so bored from how rich and problem free it is compared to Europe, that it can afford to keep starting foreign wars as if nothing ever happens.
Also back on the topic, the US managed to bring TSMC to open a cutting edge fab in the US and has already been operational for a while. Which already puts it way ahead of the EU on this front as well.
The thing is, US is much better on actually making things happen when push comes to shove. It saw it's deficient and vulnerable on domestic semiconductor manufacturing, it then made it happen with TSMC. It's doing the same with domestic ship building with Korean partners.
US might be slow moving, but somehow EU is even way slower at realizing and addressing its vulnerabilities, only waking up when it's far too late, causing it to pay a much more painful price for sleeping at the wheel (Russian invaded Ukraine in 2014 BTW, not in 2022, and they were building another gas pipeline with them), and when this type of own-goaling keeps repeating enough times you see the correlation with EU's decline as their economic rivals keep biting more and more market share from their industries as they sleep on critical changes and developments.
What really matters is the sovereign capability to finetune the LLM models. Any model could be vetted and tested, but you need finetuning/lora training to prevent the model from being outdated.
> GPT‑NL is developed within the Netherlands and Europe. This gives us full control over the model, the data and the choices we make. We avoid dependency on non‑European providers and invest in a sustainable AI ecosystem aligned with our laws, values and societal goals.
I love it! So this is our answer to America and China denying foreigners access to their frontier models.. a massive 13,5M€ founding to develop souvereign european ai, trained exclusively on legally obtained documents and highest moral standards as defined in EU AI Act.
NL could simply say: no more ASML machines, and no more ASM wafers.
The US can only block exports. They cannot force exports. The NL-US relationship is quite toxic, for example the "Dutch America Friendship Treaty". Everything is very one-sided.
Any move like this would be simply by murdering the golden goose. ASMLs stuff is good no doubt but you don’t want to give the world the incentive to develop an alternative. If it was done once it can be done again and once it has been done again say good bye to all those returns on monopoly.
You don't wanna find out how fast american troops would land there..
Faster than in Iran?
[deleted]
They’re building a competitive-quality model, from scratch, with fair compensation to content owners, for €13.5 million? Something’s wrong with this picture.
Being cheap is on brand for inhabitants of the sea floor. Nothing is wrong
I feel that not only is Europe losing its independence to the US and China, but it does not even try to take part in the race.
Unlike the US, Europe has no California-level VCs. I don't expect hundreds of billions of Euros to be poured into long-shot projects.
Unlike China, Europe has neither cohesive public investment at the global level nor the drive to grow. Long-term investments have a lot of words, a lot of regulations, a lot of proxy goals, but there is neither a lot of money nor urgency. It was captured by this post: https://x.com/piotrsankowski/status/2065795919623438546
So yeah, both in economy and warfare, Europe dooms itself to be in the hands of the US, China, or a mix of both.
> Unlike the US, Europe has no California-level VCs.
Some would consider that a good thing. There is a lot to be said for VC in recent years not being beneficial for the economy, certainly on an individual level, other than "number go up".
Sure.
At the same time, it made in many cases EU dependent on the US. A lot of governments are basically dependent on MS Office or Google Cloud.
With AI, it is even more strategic.
My impression is that in Europe much fewer people are convinced that AI-maxxing is necessary or even a net benefit.
And if you ask a bit more, you'll find that those same people are very likely to daily-drive AI models, desktop and phone operating systems and various other software critical to their professional and personal lives from US companies. And buy tons of Chinese products over Chinese or US e-commerce platforms.
What people say matters much less than what they do.
Much fewer people in Europe are convinced maxxing anything besides work/life/life balance and generous social support are a benefit.
All the stuff that doesn't help an economy grow or pay for the future.
Which has nothing to do with VCs, just with sourcing decisions.
If there were European MicroSoft or Google, there would be a preference.
Well those companies exist because of VC's, and the nation's business freedom compared to Europe
[deleted]
> There is a lot to be said for VC in recent years not being beneficial for the economy
What a wild statement, VC's are behind most of the growth in the US economy, and they directly drive up wages in tech. I'd be fascinated to hear a valid complaint of VC's that isn't just money envy
Some people consider it a good thing that communists boiled people's hands as torture. Some people consider it a good think that Iran massacred 10,000s of its own citizens. Some would consider it a good thing that Israel killed all Palestinians in Gaza.
Europe decided to regulate the hell out of foreign AI instead of investing in their own systems. It's sad to see the European continent lost the race to create a decent startup ecosystem (no decent search engines, social networks, cloud, mobile OS) and now it seems to be hellbent in losing this battle.
>It's sad to see the European continent lost the race to create a decent startup ecosystem
What's ironic and sad at the same time is that pre-2022 Russia's Yandex(domestic Russian variant of Google) was lightyears ahead of what EU, a significantly richer and more capable block, had. IIRC, their reverse image search was so good, they had to nerf it because people were using it to find the identity of people from photos.
Same for Israel, their tech sector is probably greater than the EU one combined
Absolutely shameful how the EU kept managing to snatch defeat from the jaws of victory over and over.
I think much of that is because European customers (both private and business) tended to prefer American suppliers over suppliers from European countries that were not their own. That may have something to do with most people in IT being quite fluent in English, while European products were all-to-often half-heartedly translated from German/French/Spanish/Polish/Italian/Ukranian.
In many cases, well-established and well-liked European services have been supplanted by American counterparts that came later and were not really better in any way. They did usually have much more money to burn though, undercutting pricing until competition was dead.
I'm speaking in the past tense, because now for the first time in the couple of decades I can remember, there seems to be a somewhat commonly held preference for European suppliers.
Not surprising. All the Yandex people that moved over to here to the Netherlands that I know of are astounded about the insane difference in the tax burden between what they had in Russia and what they have in Western Europe (when their 5 year tax discount ends, that is). If the government takes the bulk of your income after a certain point, there isn't really that big of a push to create ground-breaking technology.
> If the government takes the bulk of your income after a certain point, there isn't really that big of a push to create ground-breaking technology.
I'm skeptical that high taxes is a large reason to lose to California of all places. Maybe in some important sense CA has "earned" that via talent and funding density while NL hasn't (from the perspective of a company, to be clear)
[deleted]
NL is a corporate tax haven in Europe(and also compared to California), that's why almost all US and foreign companies have their corporate HQs there, Dublin or Luxembourg.
Companies go there because taxes are low for them, not necessarily for their employees(ignoring the NL 5 year tax break for foreigners). It's kinda like the US Delaware of Europe.
We also have much nicer societies than Russia. It's a dictatorship ffs.
And yes having nice things cost money. And a safety net is important.
I would never want to live in America even if I got 3x my wage. Nor Russia of course but that's a foregone conclusion.
Frankly, being Russian myself, I'm also very disappointed by the state of tech in Europe.
But you know what hurts the most? That I know it wasn't always that way.
I'm sitting right now in the same country that invented the Minitel, built out the TGV network and the Grands Projets, and don't even get me started about the weird and wonderful machines they've got in that museum in Mulhouse, hell, you could go back in time to Gustave Eiffel. Industry and ambition used to be here. It was almost physically painful to discover that it seems to be gone now.
> It was almost physically painful to discover that it seems to be gone now.
It's not gone, it just needs to be re-discovered. And the bureaucrats need to flash some € then get out of the way.
You are saying that as if China or the US are completely isolated from the EU.
We live in a globalized world whether you like it or not, and every supply chain spans multiple countries.
Arguably, staying out of the AI "race" is a good thing
Military race isn't a good think either, but you don't want to be on the losing side.
I'll play devil's advocate a little bit - I'm not sure it is losing its "independence" by not taking part in the race. It could very well be that it is gaining independence from tech and choosing a "second mover advantage" to decide how it gets deployed after seeing how it impacts everyone else. Let the US and China experiment on the bleeding edge (and their citizens feel the effect, both good and bad), and then be picky about how you use it.
I don't know if it is the right strategy but there's certainly a legitimate strategy in there.
The problem is recursive self improvement creating a very difficult gap and the fact that power, compute has a lag from when you invest and when data centers come up.
You also can't just spin up a research team out of nowhere.
These things are true, however:
1. The labs in the US and China don't seem to have any problem selling (or even giving) access to these models right now.
2. If some kind of take-off happens which makes that not true, my bet is all bets are off on what that outcome even looks like. What would the economic paradigm even be under a superintelligent AGI? Do you think "it" is going to listen when Trump says "you can't work with Europe"?
There's a whole bunch of grey in between the two, for example only having access to second rate models, but I'm not sure that particularly matters if the strategy is "second mover."
I mean, it might listen to him. We have no clue, which is the problem.
Sure, but my guess is for "true" super intelligence we won't be able to predict whether that is true or not until it happens. I'm not a doomer, but I also don't really think we can "align" people, much less a "super intelligent" AI.
Let’s autonomous Russian drones, and Europe is at mercy of two other empires, who capitalize on this opportunity.
Serious question: what does any of that have to do with the submitted article? Where is the relevance to the topic at hand?
> Unlike the US, Europe has no California-level VCs. I don't expect hundreds of billions of Euros to be poured into long-shot projects.
My ex-neighbor (when I was a teenager, living in Belgium) and very good friend really wanted to make it big. He became a chip engineer, moved to California, raised money for a first startup (it tanked) then raised money for a second startup. He made the world a better place (he created some very specific micro-inverters for solar panels) and made a $$$ exit.
The EU saw exactly zero of the wealth he created and he's never ever coming back to what he considers a failure of a continent.
That's the problem: many of the great minds with the mindset required to do great things already left the EU.
> So yeah, both in economy and warfare, Europe dooms itself to be in the hands of the US, China, or a mix of both.
And in energy (economy is energy and energy is economy, and China really understood that) the EU doomed itself to be in the hands of Russia.
We are a failure of sinking continent.
> economy is energy and energy is economy, and China really understood that
In former times the energy monopoly was called "The Power Company"; we intend to give this name an entirely new meaning."
– CEO Nwabudike Morgan, "The Centauri Monopoly"
Europe is a great place to live if you just want to float through life.
The US is a great place to live if you have talent, want to work, and want to reap the rewards.
Europe is not a country.
Regulations are not even throughout each of the 27 member states. Each country is relatively small in the world stage.
Until EU progresses towards federalization, discussing this is a moot point.
We already had GEITje but it was banned by the courts. Of course it can still be found because the entire internet is not subject to Dutch law. But it did manage to stop development :'(
> A total of €13.5 million has been allocated to the project.
> This public investment underlines the importance of an independent, trustworthy and future‑proof Dutch language model.
It does, but not in the way you think it does.
> It does, but not in the way you think it does.
They're training a model, not funding a startup. €13.5 million is plenty to pre- and post-train a decent model.
If Europe is serious about getting home grown AI fast, three simple steps:
1. Huge tax incentives, let the companies get grossly wealthy while paying minimal taxes. Minimum 10 years with clauses protecting "retribution" taxes there after.
2. Tax incentives for the founders/shareholders, just like above.
3. Drop worker protections to a minimum, make it easy to fire people. You only want serious/dedicated employees anyway.
Within 2-3 years there will be at least a trillion dollars looking to get in.
Don't worry though if reading that made you mad. Its absolutely not going to happen. I can think of few things more antithetical to the European ethos than smart skilled people working 80-100hrs weeks with almost no vacation to gas their founders net worth by tens, hundreds, of billions.
[deleted]
A total of €13.5M has been allocated to the project.
I guess we’re going for GPT2 level capability?
Honestly, I used to think the 'sovereign model' was a waste of money. But recently, with the US logic of restricting model exports, I've come to think that if things go south, they could even cut off allied nations. So now the sovereign model seems reasonable to me. That, in turn, means US influence is deteriorating. And that probably isn't such great news for American businesses.
I wonder with these stories. Why are there so many individual country efforts? We know the scale needed with scaling laws / capital / energy. Most of these countries alone can barely compete (even large groups of them would struggle.
Why don't they work together on it? Companies like Airbus have already been able to do that with aircraft.
This is not an open source model. In that sense I think the sovereign claim is a bit strange. It's the data providers that determine access to the model.
So it's a model that's sovereign as in sovereign kingdom of the Netherlands vs sovereign for the people's?
"sovereign" the marketing term basically means "in-house" now, where "house" depends on who says it.
I'm making a Dutch dictionary and would be interested to see how this model would fair in evals vs non specialized ones. I've tested a variety of models for https://hetnederlands.com content and differences can be big
Is it a proposal or a model?
And if it is a model, how fies it fare on benchmarks?
What are they going to train with 13.5M really? We're a tiny company in Amsterdam in Holland and we've got "only 64x B300 to train on" so we could never make an LLM I thought, since we've got only 4M in compute.
And they're going to train an LLM with all kinds of extra difficulties compared to OpenAI for just 13.5M?
The very first Llama was 16M for one training.
This is too little, too late. Europe really need to start focussing.
All these tiny niche models are perhaps fun as an academic exercise or great for the researchers resume but I highly doubt that they'll add any value or will be used for anything serious.
Even if this becomes a somewhat decent model with a fantastic understanding of "gezellig", "kring verjaardag" or "pannenkoeken", how many people will interact with it before the limits of it will drive them back to a frontier model?
Even if the purpose of this is government & other regulated industries, do we really want our government to use a poor model? Either do it right or don't do it at all.
Prices for training have dropped immensely in terms of research required, code efficiency, algorithmic/sample efficiency, and possibly also hardware (I'm not qualified to say without looking it FLOPS/dollar, or even to be certain that's the right metric here).
[deleted]
How do you use it?
So cute.
I really think countries should build a sovereign _ecosystem_ and sovereign models are an excuse to achieve it.
An ecosystem is the tribal knowledge, revolving door of talent, known processes etc.
If the end goal is to make a half assed Dutch speaking model, I think it won’t cut it. I don’t see anyone using it over Gemma 4b that runs on my laptop.
An ecosystem is more durable and has desirable second order effects.
Supposedly this model also aims to treat publishers of all sizes well. Looking forward to its launch soon :)
Maybe it's time to acknowledge that current copyright laws do more harm than good and put another framework in place.
[dead]
[flagged]
This really isn’t what HN is for. I think you should read the guidelines. I’ve looked at your posting history and it is all like this: relentless negativity with low information content.
Why are you being nasty? And in what world is the Netherlands a second-world nation?
> Excluding harmful content
#define(HARMFUL)
[edit] Downvoters please tell me what the problem is with specifying this?
I didn't down vote you, but you aren't really adding anything to the conversation. This type of pithy comment might be fun on Reddit, but at HN, we try to provide more constructive, and information rich, comments.
I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 (https://www.ai.se/en/project/gpt-sw3) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by its people, basically for free. Nations should mirror what Cursor has done with Composer 2.5 for example.
Do we know for sure how much national corpus of knowledge (like dutch) goes into these "global" models and how that affects "localized" model biases? What's wrong with specialized models?
If open frontier models start closing up and states start more export controls on AI services and hardware, it might be good to ensure the supply chain is there to reproduce the SotA, or even a couple generations behind it.
And what happens once the "solid baselines" become unavailable for a reason or the other?
You keep building on the last available version? Fine tuning is a whole lot cheaper, easier and more useful than pretraining a model from scratch. It's a complete no brainer.
> You keep building on the last available version?
yes but a sovereign can allocate some resources and a few people to stay in the loop from a first principles level. No need to wait for a rug pull.
Of course, it can not compete with the frontier labs. But good to have researchers and professors "in-house". LLMs are here for the long-term.
Seems like you don’t understand.
You take current version and build on top of it. You have the weights.
You might not get some n+1 version at some point but the n version you will have will be still most likely much better than whatever you come up with burning good will money of people believing in „sovereignty”.
You are not getting ahead in this game by being „true to your local values” capital expenditure is insane in this game.
Kimi and Qwen come out of China, which means that their training material may be biased e.g. relating to Taiwan [1]. In addition, there is no way to determine what input went into the training, if it was properly licensed, if it was legal (e.g. not contaminated by CSAM), or how the human component of RLHF was sourced - in US models, for example, stories about exploitation like [2] have been floating for years.
Assuming us Europeans finally get our act together, I think it is better for our long-term future (and the ethical problems) if we manage to get a baseline of training input and data ourselves, from scratch, with everything being ethically sourced.
Oh and, while we're at it, the EU has 24 official languages plus a host of minority languages. Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best. An European model with actual funding and proper data sources might be able to significantly reduce that.
[1] https://www.taiwannews.com.tw/news/6245677
[2] https://www.theguardian.com/technology/2024/apr/16/techscape...
There is something north of 8% OCR error rates.. that will hurt model quality!
> Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best.
that is not true, so please read before make an opinion. The French Mistral project shipped seven+ years ago with 140 languages for example.. language translation was the first LLM task from 2015
Uh, some would say it's easy to determine what input went into the training for kimi and qwen.. since they were caught stealing it from American labs. Some cultural cliches may never change.
It's well-known that all commercial models are based on stolen content. That doesn't mean there is no filtering/censoring, just that the censoring likely depends on where it's happening…
> since they were caught stealing it from American labs. Some cultural cliches may never change.
Has a formal lawsuit been brought to bear? Given, Anthropic & OpenAI are being dragged through courts for copyright violation (or stealing, as you'd call it, if the companies involved were culturally Chinese) by newspapers, publishing houses etc; one'd think they'd pass on some of that medicine to Alibaba, which does have business entities registered in the US.
It is crazy that anything Europe gets so much hate. IMO it is important to build models within the boundaries of smaller nations, using their own language. Research has to continue even if it is outside of US and China.
It’s not that it gets hate so much as it’s akin to watching them make announcements that they’re going to make a European google/facebook/tiktok.
Sure… they can, except at the end of the day it’s a bit late, regulatory burden will make it comparatively useless, and because of that nobody will ever use it. It will be spending a bunch of taxpayer dollars for press releases.
The running joke is that when these “sovereign” EU models launch, they’re going to refuse to answer anything that might involve personal information such as Elon Musk’s birthday.
That’s on Wikipedia, it’s not PII, it’s also not going to be relevant to any meaningful IRL work.
I challenge the assumption you can do meaningful work in this field without blatant disregard for intellectual property.
The idea that it’s all down to training size is clearly incorrect, as every expert human learned their craft without nearly the sum total information of the internet. Clearly there are architectural wins to be found.
Besides that, why would everyone just be fine with Opus level AI at best, as that’s all the US is willing to export, and I doubt China will share beyond that.
Sovereign AI is more important than ever after Friday.
I kinda agree, the best use of taxpayer money should be in reducing taxes to corporation that would like to compete in the market vs US and China, rather than making governments playing the game (since they very obviously can’t).
So good to see these developments. Every country should do this. I'd even say every person should gave their own personalized AI running on their own computers. If only the costs involved were not so astronomical.
I think it will be cost effective at some point. Computers were limited to research institutes before the personal computer arrived.
I hope you're right. I really don't want a future where only corporations and governments have computers.
Nvidia will certainly be pleased.
Interesting that this got posted now: the project is receiving increasingly more skepticism lately in the Dutch tech scene [0], and I think that’s fully justified.
[0]: https://www.quotenet.nl/zakelijk/a71588202/techondernemers-m...
What is the exact skepticism? The only thing I could get from that was from some "tech entrepreneur":
> GPT-NL was never built to compete with Claude or ChatGPT. It was trained exclusively on licensed data, and is intended more for governments and companies where privacy and compliance matter more than raw performance.”
That's it? That it didn't aim to compete with SOTA models? Maybe this is something you have to start with something, then ramp up, rather do what only a select few labs been able to do, start with really big models. Especially if you're resource constrained, which since this is a government project, I really hope for the sake of the tax payers it was.
I mean if you are wasting funds kind of knowing it's nowhere near remote competitive, then it's kind of a fraud.
TNO is something like semi-DARPA. It gets a lot of stuff tax free and a lot of gov funding, but a lot of their budget is from getting businesses to hire their R&D teams.
They do really good R&D on a lot of stuff. This is just their attempt at public credibility/internal skill building to enter the LLM business.
Doubt its going to be successful, but they "waste" a lot more money on other things that you never heard of. Its not fraud, its just R&D dressed up a little too much too early.
But why is "competing against remote SOTA models on quality" the only thing that matters here?
What the hell else is there? All the other stuff can be done by an intern with an 8 euro HF Pro subscription.
Other than actual research, which is in a different camp.
Common approach I've seen is having workflows with paid/larger/hosted models for some workflow where you don't quite know exactly how it'll be when you first put it together, then with time you've locked down how things more or less work yet you still need free-form text parsing of some kind, so you end up replacing the bigger models with carefully post-trained small models.
Besides that, there is a ton of use cases for smaller models for a bunch of different things. We'll be unlikely to be able to run LLMs (actually Large) on smartphones for a while, while the smaller LLMs seem to run already on-device in experiments.
Targeting a niche audience with specific requirements is not fraud.
I don't understand countries (especially governments) wanting to have their own models when there are already pretty solid open source (weights) models out there.
Countries should want control over _where_ the compute is happening rather than _what code_ is running.
What's wrong with a country hosting a Kimi, Qwen or GPT-Oss on their hardware for their government work purpose?
It is not about the country but the language. Most llms have poor or no support for Dutch.
Idk which models you refer to, but I tested a bunch recently, and they performed well on Dutch. Only the smallest, such as qwen 3.6 27B, made up words and switched languages.
There would be a bunch of value in having, say, a good 30B-class model that used my local language as well as it does English. There's lots of cases, especially in the government sphere, where local processing is a requirement and frontier-level capabilities aren't required. Making those cheap to run seems like a fine goal.
Can you provide some examples of these use cases?
There's a large gap between making up words and an actually native text distribution. LLMs have a clear pattern, clear tells, a "feel" in English, and it's normally even more pronounced in non-English languages.
Lots of bias towards English sentence structure, idioms, etiquette, etc.
I don't understand this. Even if that were true (and it isn't in my experience), a model that is trained on a Dutch corpus and arguably "knows Dutch well" but has the reasoning and comprehension abilities of a three year old is useless in any case. I'd rather use a model that can only speak English and put an automatic translator around it.
An LLM is an encoding of a culture, a way of viewing the world.
They are not neutral technology, they are a direct representation of the training set that has been chosen and how they are aligned.
In many ways, they are ideology made code.
If we leave building them to the US and China, only their way of seeing things will be digitized.
I don't like the idea of that.
Yes and also, US and Chinese models are censored in different ways. US models are way too prudish for personal use in Europe because they're afraid to piss off religious investors. Chinese models are too censored on history and current affairs, eg the tiananmen massacre never happened stuff like that.
Really? Because I'm pretty sure that at least every two days there's a active post with a top voted comment along the lines of "The EU isn't doing AI themselves, they are so hosed".
Why should Dutch people be expected to make do with models 99% trained on American/Chinese cultural context and language?
Oh, it's all fine with cultural context here -- we don't even dub English language movies here because we are that cheap
Understood, but they could fine tune base models on their own cultural context and language. Why reinventing the wheel?
I thought finetuning data can't contradict foundation models, and anything that are inconsistent with the standard LLM American-Chinese split personality would be rejected?
Fine tuning happens on top of pretraining, so of course it can "forget" pretrained defaults when warranted by the new data it's being fine tuned on.
But you have to have more data than used for pretraining for the added knowledge to take precedent over pretraining, no? If that would be the case, you practically contradict the knowledge in the base model.
I mean ... LLMs are sort of an extreme and living proof of linguistic determinism. Their behaviors are dictated almost entirely by disorganized language data, primarily English and Chinese. So you can't just add a language as native primary language in a quick post training, I think. There's no way that it would work.
They could apply the Polder Model of consensus decision making with a mixture of experts.
https://en.wikipedia.org/wiki/Polder_model
Funny, that's what I thought when PewDiePie set up his monster AI rig and what he called a 'council'. Quote:
"PewDiePie has built a custom web UI for self-hosting AI models called "ChatOS" that runs on his custom PC with 2x RTX 4000 Ada cards, along with 8x modded RTX 4090s with 48 GB of VRAM. Running open-source models from Baidu and OpenAI, PewDiePie made a "council" of bots that voted on the best responses, and then built "The Swarm" for data collection that will become the foundation of his own model coming next month."
https://www.tomshardware.com/tech-industry/artificial-intell...
This gets better short-term results for a fraction of the cost, for sure, but what do you when China places an export control banning the release of open weight models? If you don't have your own talent, you're then relegated to using a base model from 2026 or whatever the cutoff date is, forever. That defeats the purpose of a 'sovereign' model made for and by your people.
>Countries should want control over _where_ the compute is happening
Yeah but Europe doesn't build any computer hardware, and EU Green eco-communists and NIMBVYs don't want to have data centers built in their backyard, so the only way left for EU consultancies to milk taxpayer money for the AI bubble, is shipping a sovereign AI model for each country/language.
Watch out US tech sector, we're coming for you. Feel our wrath.
Have you heard of ASML? NXP?
Ignorant comment
Please don't move the goalposts. What computer parts does ASML or NXP make?
ASML only makes the lithography machines, 85% of which go outside the EU (let that sink in). And then fabs in Taiwan, Korea or the US use those ASML machines to etch US IP for computer chips. EU doesn't make any computer parts domestically.
And NXP mostly makes various microcontrollers and small chips, not high margin IP decenter centric parts like ASICS, FPGAs, CPUs or GPUs.
So not only are you the ignorant one here, but you also have the audacity to insult others with so much confidence.
@dwa3592 below. Firstly, why are you moving the goalposts in bad faith again just to stir an argument? What does that have to do with my original comment?
And secondly, there's other lithography machines out there, not just ASML.
And thirdly, the IP Nvidia, AMD, etc develop to etch on silicone via ASML machines makes them more valuable than ASML.
Fourthly, repeating my "let that sink in" phrase is just childish and low-IQ trolling, unworthy of this platform.
>>ASML only makes the lithography machines
Woah! only lithography machines???? it is literally impossible to make any device capable of running anything close to AI without ASML. Let that sink in.
Funnily ASML owes its current success partly to US funded research (straight from Wikipedia):
> Two years later, it joined a consortium, which included Intel and two other U.S. chipmakers, in order to exploit fundamental research conducted by the US Department of Energy. Because the Cooperative Research and Development Agreement (CRADA) it operates under is funded by the US government, licensing must be approved by Congress.[12]
Why are you acting childish and petty? I said EU hasn't got AI compute manufacturing(aka no equivalent IP to Nvidia and AMD and no equivalent to TSMC or Samsung fabs), not that it doesn't have lithography machines manufacturing.
Surely you understand that while you can have the latter, you can also lack the former.
In a recent podcast, it was summarized as:
ASM (International) makes machines that add material to a silicon wafer (deposition).
ASML makes machines that remove material from said wafers (lithography, etching)
(I was a bit surprised that's not combined in 1 machine. But let's move on)
Then Besi makes machines to stack / interconnect / package those ICs into a package. I'm assuming pick & place machines are other companies' turf.
The above are all Dutch companies, operating a pretty important section of the tech stack.
Iirc there were (& probably still are) some IC fabs in Europe, but mostly older nodes (like useful for microcontrollers used by car manufacturers. Wikipedia has a list). So for SOTA smartphone SoCs it's off to Taiwan (TSMC), South Korea (Samsung) or China (who makes everything, including smartphones & the chips going in there).
So as far as EU goes, the capabilities are mostly there. Skilled workforce? Check. Money? This is a rich continent.
What's missing is the guts to say "hey, let's dump €100B into this & make ourselves some laptop & server CPUs!".
But now the important thing: several of such initiatives are starting to bear fruit, and b) confidence that EU can do such things, is growing.
As for bureaucracy / red tape... sigh... (won't be fixed any time soon)
The World's Most Important Machine ;)
https://www.youtube.com/watch?v=MiUHjLxm3V0
Most important machine ... built on US IP, subject to US export restrictions, used to manufacture high value US IP, in factories outside the EU, so profits of those chips goes to US. A point I have addressed over two times already.
Also ASML even threatened to leave the NL if the Dutch government doesn't do what they want on taxes and labor policies. So having only a single card to play that EU can loose at any time, it's not putting EU tech sovereignty argument in a good light.
The "wahabout ASML" that keeps being spammed by people here, isn't proof of EU compute and AI sovereignty. It's the exception which is why it's the only thing people can mention on EU tech and they DDoS you with it as if that changes anything.
Are people here that petty that they can't stay on topic and argue in good faith and instead need to hijack your argument to go on offtopic whataboutism for a cheap gotcha spamming "whatabout ASML" on unrelated arguments?
>ASML only makes the shovel making machines
>>Yeah but Europe doesn't build any computer hardware,
Well, then this is will be a good start.
EU bureaucrats are too busy trying to keep the welfare/pension system from collapsing, defeating Russia, supporting Ukraine, managing the fossil fuels energy shortages, figuring out how to nerf Chinese EVs while supporting domestic car companies, and restricting social media free speech to make sure the "far right" don't win elections.
So of course, semiconductor manufacturing sovereignty is very low on their priority list.
That's what bureaucrats are supposed to be doing BTW.
How many in that list of things is the US also doing?
The US is a single country. Russia is not on the US' doorstep. The US has its own oil. The US prints the world reserve currency.
Different scale for those problems. Way different scale. US is monetary rich, oil rich, energy rich, manufacturing rich, and doesn't suffer from russian aggression at its borders. US is so bored from how rich and problem free it is compared to Europe, that it can afford to keep starting foreign wars as if nothing ever happens.
Also back on the topic, the US managed to bring TSMC to open a cutting edge fab in the US and has already been operational for a while. Which already puts it way ahead of the EU on this front as well.
The thing is, US is much better on actually making things happen when push comes to shove. It saw it's deficient and vulnerable on domestic semiconductor manufacturing, it then made it happen with TSMC. It's doing the same with domestic ship building with Korean partners.
US might be slow moving, but somehow EU is even way slower at realizing and addressing its vulnerabilities, only waking up when it's far too late, causing it to pay a much more painful price for sleeping at the wheel (Russian invaded Ukraine in 2014 BTW, not in 2022, and they were building another gas pipeline with them), and when this type of own-goaling keeps repeating enough times you see the correlation with EU's decline as their economic rivals keep biting more and more market share from their industries as they sleep on critical changes and developments.
What really matters is the sovereign capability to finetune the LLM models. Any model could be vetted and tested, but you need finetuning/lora training to prevent the model from being outdated.
> GPT‑NL is developed within the Netherlands and Europe. This gives us full control over the model, the data and the choices we make. We avoid dependency on non‑European providers and invest in a sustainable AI ecosystem aligned with our laws, values and societal goals.
I love it! So this is our answer to America and China denying foreigners access to their frontier models.. a massive 13,5M€ founding to develop souvereign european ai, trained exclusively on legally obtained documents and highest moral standards as defined in EU AI Act.
NL could simply say: no more ASML machines, and no more ASM wafers.
ASML’s EUV technology is partially based on US research and so Congress has a degree of control over it, so it’s not that simple: https://web.archive.org/web/20230116222847/https://www.nytim...
The US can only block exports. They cannot force exports. The NL-US relationship is quite toxic, for example the "Dutch America Friendship Treaty". Everything is very one-sided.
Any move like this would be simply by murdering the golden goose. ASMLs stuff is good no doubt but you don’t want to give the world the incentive to develop an alternative. If it was done once it can be done again and once it has been done again say good bye to all those returns on monopoly.
You don't wanna find out how fast american troops would land there..
Faster than in Iran?
They’re building a competitive-quality model, from scratch, with fair compensation to content owners, for €13.5 million? Something’s wrong with this picture.
Being cheap is on brand for inhabitants of the sea floor. Nothing is wrong
Previously posted on 02-dec-2023 https://news.ycombinator.com/item?id=38497495 3 comments
Two and a half years and still not complete? That's ridiculous.
AMALIA, from Portugal, going the same path!
https://en.wikipedia.org/wiki/Am%C3%A1lia_(LLM)
I feel that not only is Europe losing its independence to the US and China, but it does not even try to take part in the race.
Unlike the US, Europe has no California-level VCs. I don't expect hundreds of billions of Euros to be poured into long-shot projects.
Unlike China, Europe has neither cohesive public investment at the global level nor the drive to grow. Long-term investments have a lot of words, a lot of regulations, a lot of proxy goals, but there is neither a lot of money nor urgency. It was captured by this post: https://x.com/piotrsankowski/status/2065795919623438546
So yeah, both in economy and warfare, Europe dooms itself to be in the hands of the US, China, or a mix of both.
> Unlike the US, Europe has no California-level VCs.
Some would consider that a good thing. There is a lot to be said for VC in recent years not being beneficial for the economy, certainly on an individual level, other than "number go up".
Sure.
At the same time, it made in many cases EU dependent on the US. A lot of governments are basically dependent on MS Office or Google Cloud.
With AI, it is even more strategic.
My impression is that in Europe much fewer people are convinced that AI-maxxing is necessary or even a net benefit.
And if you ask a bit more, you'll find that those same people are very likely to daily-drive AI models, desktop and phone operating systems and various other software critical to their professional and personal lives from US companies. And buy tons of Chinese products over Chinese or US e-commerce platforms.
What people say matters much less than what they do.
Much fewer people in Europe are convinced maxxing anything besides work/life/life balance and generous social support are a benefit.
All the stuff that doesn't help an economy grow or pay for the future.
Which has nothing to do with VCs, just with sourcing decisions.
If there were European MicroSoft or Google, there would be a preference.
Well those companies exist because of VC's, and the nation's business freedom compared to Europe
> There is a lot to be said for VC in recent years not being beneficial for the economy
What a wild statement, VC's are behind most of the growth in the US economy, and they directly drive up wages in tech. I'd be fascinated to hear a valid complaint of VC's that isn't just money envy
Some people consider it a good thing that communists boiled people's hands as torture. Some people consider it a good think that Iran massacred 10,000s of its own citizens. Some would consider it a good thing that Israel killed all Palestinians in Gaza.
Europe decided to regulate the hell out of foreign AI instead of investing in their own systems. It's sad to see the European continent lost the race to create a decent startup ecosystem (no decent search engines, social networks, cloud, mobile OS) and now it seems to be hellbent in losing this battle.
>It's sad to see the European continent lost the race to create a decent startup ecosystem
What's ironic and sad at the same time is that pre-2022 Russia's Yandex(domestic Russian variant of Google) was lightyears ahead of what EU, a significantly richer and more capable block, had. IIRC, their reverse image search was so good, they had to nerf it because people were using it to find the identity of people from photos.
Same for Israel, their tech sector is probably greater than the EU one combined
Absolutely shameful how the EU kept managing to snatch defeat from the jaws of victory over and over.
I think much of that is because European customers (both private and business) tended to prefer American suppliers over suppliers from European countries that were not their own. That may have something to do with most people in IT being quite fluent in English, while European products were all-to-often half-heartedly translated from German/French/Spanish/Polish/Italian/Ukranian.
In many cases, well-established and well-liked European services have been supplanted by American counterparts that came later and were not really better in any way. They did usually have much more money to burn though, undercutting pricing until competition was dead.
I'm speaking in the past tense, because now for the first time in the couple of decades I can remember, there seems to be a somewhat commonly held preference for European suppliers.
Not surprising. All the Yandex people that moved over to here to the Netherlands that I know of are astounded about the insane difference in the tax burden between what they had in Russia and what they have in Western Europe (when their 5 year tax discount ends, that is). If the government takes the bulk of your income after a certain point, there isn't really that big of a push to create ground-breaking technology.
> If the government takes the bulk of your income after a certain point, there isn't really that big of a push to create ground-breaking technology.
I'm skeptical that high taxes is a large reason to lose to California of all places. Maybe in some important sense CA has "earned" that via talent and funding density while NL hasn't (from the perspective of a company, to be clear)
NL is a corporate tax haven in Europe(and also compared to California), that's why almost all US and foreign companies have their corporate HQs there, Dublin or Luxembourg.
Companies go there because taxes are low for them, not necessarily for their employees(ignoring the NL 5 year tax break for foreigners). It's kinda like the US Delaware of Europe.
We also have much nicer societies than Russia. It's a dictatorship ffs.
And yes having nice things cost money. And a safety net is important.
I would never want to live in America even if I got 3x my wage. Nor Russia of course but that's a foregone conclusion.
Frankly, being Russian myself, I'm also very disappointed by the state of tech in Europe.
But you know what hurts the most? That I know it wasn't always that way.
I'm sitting right now in the same country that invented the Minitel, built out the TGV network and the Grands Projets, and don't even get me started about the weird and wonderful machines they've got in that museum in Mulhouse, hell, you could go back in time to Gustave Eiffel. Industry and ambition used to be here. It was almost physically painful to discover that it seems to be gone now.
> It was almost physically painful to discover that it seems to be gone now.
It's not gone, it just needs to be re-discovered. And the bureaucrats need to flash some € then get out of the way.
You are saying that as if China or the US are completely isolated from the EU. We live in a globalized world whether you like it or not, and every supply chain spans multiple countries.
Arguably, staying out of the AI "race" is a good thing
Military race isn't a good think either, but you don't want to be on the losing side.
I'll play devil's advocate a little bit - I'm not sure it is losing its "independence" by not taking part in the race. It could very well be that it is gaining independence from tech and choosing a "second mover advantage" to decide how it gets deployed after seeing how it impacts everyone else. Let the US and China experiment on the bleeding edge (and their citizens feel the effect, both good and bad), and then be picky about how you use it.
I don't know if it is the right strategy but there's certainly a legitimate strategy in there.
The problem is recursive self improvement creating a very difficult gap and the fact that power, compute has a lag from when you invest and when data centers come up.
You also can't just spin up a research team out of nowhere.
These things are true, however:
1. The labs in the US and China don't seem to have any problem selling (or even giving) access to these models right now.
2. If some kind of take-off happens which makes that not true, my bet is all bets are off on what that outcome even looks like. What would the economic paradigm even be under a superintelligent AGI? Do you think "it" is going to listen when Trump says "you can't work with Europe"?
There's a whole bunch of grey in between the two, for example only having access to second rate models, but I'm not sure that particularly matters if the strategy is "second mover."
I mean, it might listen to him. We have no clue, which is the problem.
Sure, but my guess is for "true" super intelligence we won't be able to predict whether that is true or not until it happens. I'm not a doomer, but I also don't really think we can "align" people, much less a "super intelligent" AI.
Let’s autonomous Russian drones, and Europe is at mercy of two other empires, who capitalize on this opportunity.
Serious question: what does any of that have to do with the submitted article? Where is the relevance to the topic at hand?
> Unlike the US, Europe has no California-level VCs. I don't expect hundreds of billions of Euros to be poured into long-shot projects.
My ex-neighbor (when I was a teenager, living in Belgium) and very good friend really wanted to make it big. He became a chip engineer, moved to California, raised money for a first startup (it tanked) then raised money for a second startup. He made the world a better place (he created some very specific micro-inverters for solar panels) and made a $$$ exit.
The EU saw exactly zero of the wealth he created and he's never ever coming back to what he considers a failure of a continent.
That's the problem: many of the great minds with the mindset required to do great things already left the EU.
> So yeah, both in economy and warfare, Europe dooms itself to be in the hands of the US, China, or a mix of both.
And in energy (economy is energy and energy is economy, and China really understood that) the EU doomed itself to be in the hands of Russia.
We are a failure of sinking continent.
> economy is energy and energy is economy, and China really understood that
In former times the energy monopoly was called "The Power Company"; we intend to give this name an entirely new meaning."
– CEO Nwabudike Morgan, "The Centauri Monopoly"
Europe is a great place to live if you just want to float through life.
The US is a great place to live if you have talent, want to work, and want to reap the rewards.
Europe is not a country.
Regulations are not even throughout each of the 27 member states. Each country is relatively small in the world stage.
Until EU progresses towards federalization, discussing this is a moot point.
We already had GEITje but it was banned by the courts. Of course it can still be found because the entire internet is not subject to Dutch law. But it did manage to stop development :'(
> A total of €13.5 million has been allocated to the project.
> This public investment underlines the importance of an independent, trustworthy and future‑proof Dutch language model.
It does, but not in the way you think it does.
> It does, but not in the way you think it does.
They're training a model, not funding a startup. €13.5 million is plenty to pre- and post-train a decent model.
If Europe is serious about getting home grown AI fast, three simple steps:
1. Huge tax incentives, let the companies get grossly wealthy while paying minimal taxes. Minimum 10 years with clauses protecting "retribution" taxes there after.
2. Tax incentives for the founders/shareholders, just like above.
3. Drop worker protections to a minimum, make it easy to fire people. You only want serious/dedicated employees anyway.
Within 2-3 years there will be at least a trillion dollars looking to get in.
Don't worry though if reading that made you mad. Its absolutely not going to happen. I can think of few things more antithetical to the European ethos than smart skilled people working 80-100hrs weeks with almost no vacation to gas their founders net worth by tens, hundreds, of billions.
A total of €13.5M has been allocated to the project.
I guess we’re going for GPT2 level capability?
Honestly, I used to think the 'sovereign model' was a waste of money. But recently, with the US logic of restricting model exports, I've come to think that if things go south, they could even cut off allied nations. So now the sovereign model seems reasonable to me. That, in turn, means US influence is deteriorating. And that probably isn't such great news for American businesses.
I wonder with these stories. Why are there so many individual country efforts? We know the scale needed with scaling laws / capital / energy. Most of these countries alone can barely compete (even large groups of them would struggle.
Why don't they work together on it? Companies like Airbus have already been able to do that with aircraft.
This is not an open source model. In that sense I think the sovereign claim is a bit strange. It's the data providers that determine access to the model.
So it's a model that's sovereign as in sovereign kingdom of the Netherlands vs sovereign for the people's?
"sovereign" the marketing term basically means "in-house" now, where "house" depends on who says it.
I'm making a Dutch dictionary and would be interested to see how this model would fair in evals vs non specialized ones. I've tested a variety of models for https://hetnederlands.com content and differences can be big
Is it a proposal or a model? And if it is a model, how fies it fare on benchmarks?
What are they going to train with 13.5M really? We're a tiny company in Amsterdam in Holland and we've got "only 64x B300 to train on" so we could never make an LLM I thought, since we've got only 4M in compute.
And they're going to train an LLM with all kinds of extra difficulties compared to OpenAI for just 13.5M?
The very first Llama was 16M for one training.
This is too little, too late. Europe really need to start focussing.
All these tiny niche models are perhaps fun as an academic exercise or great for the researchers resume but I highly doubt that they'll add any value or will be used for anything serious.
Even if this becomes a somewhat decent model with a fantastic understanding of "gezellig", "kring verjaardag" or "pannenkoeken", how many people will interact with it before the limits of it will drive them back to a frontier model?
Even if the purpose of this is government & other regulated industries, do we really want our government to use a poor model? Either do it right or don't do it at all.
Prices for training have dropped immensely in terms of research required, code efficiency, algorithmic/sample efficiency, and possibly also hardware (I'm not qualified to say without looking it FLOPS/dollar, or even to be certain that's the right metric here).
How do you use it?
So cute.
I really think countries should build a sovereign _ecosystem_ and sovereign models are an excuse to achieve it.
An ecosystem is the tribal knowledge, revolving door of talent, known processes etc.
If the end goal is to make a half assed Dutch speaking model, I think it won’t cut it. I don’t see anyone using it over Gemma 4b that runs on my laptop.
An ecosystem is more durable and has desirable second order effects.
Supposedly this model also aims to treat publishers of all sizes well. Looking forward to its launch soon :)
Maybe it's time to acknowledge that current copyright laws do more harm than good and put another framework in place.
[dead]
[flagged]
This really isn’t what HN is for. I think you should read the guidelines. I’ve looked at your posting history and it is all like this: relentless negativity with low information content.
Why are you being nasty? And in what world is the Netherlands a second-world nation?
> Excluding harmful content
#define(HARMFUL)
[edit] Downvoters please tell me what the problem is with specifying this?
I didn't down vote you, but you aren't really adding anything to the conversation. This type of pithy comment might be fun on Reddit, but at HN, we try to provide more constructive, and information rich, comments.