74
Launch HN: Onyx (YC W24) – The open-source chat UI
Hey HN, Chris and Yuhong here from Onyx (https://github.com/onyx-dot-app/onyx). We’re building an open-source chat that works with any LLM (proprietary + open weight) and gives these LLMs the tools they need to be useful (RAG, web search, MCP, deep research, memory, etc.).
Demo: https://youtu.be/2g4BxTZ9ztg
Two years ago, Yuhong and I had the same recurring problem. We were on growing teams and it was ridiculously difficult to find the right information across our docs, Slack, meeting notes, etc. Existing solutions required sending out our company's data, lacked customization, and frankly didn't work well. So, we started Danswer, an open-source enterprise search project built to be self-hosted and easily customized.
As the project grew, we started seeing an interesting trend—even though we were explicitly a search app, people wanted to use Danswer just to chat with LLMs. We’d hear, “the connectors, indexing, and search are great, but I’m going to start by connecting GPT-4o, Claude Sonnet 4, and Qwen to provide my team with a secure way to use them”.
Many users would add RAG, agents, and custom tools later, but much of the usage stayed ‘basic chat’. We thought: “why would people co-opt an enterprise search when other AI chat solutions exist?”
As we continued talking to users, we realized two key points:
(1) just giving a company secure access to an LLM with a great UI and simple tools is a huge part of the value add of AI
(2) providing this well is much harder than you might think and the bar is incredibly high
Consumer products like ChatGPT and Claude already provide a great experience—and chat with AI for work is something (ideally) everyone at the company uses 10+ times per day. People expect the same snappy, simple, and intuitive UX with a full feature set. Getting hundreds of small details right to take the experience from “this works” to “this feels magical” is not easy, and nothing else in the space has managed to do it.
So ~3 months ago we pivoted to Onyx, the open-source chat UI with:
- (truly) world class chat UX. Usable both by a fresh college grad who grew up with AI and an industry veteran who’s using AI tools for the first time.
- Support for all the common add-ons: RAG, connectors, web search, custom tools, MCP, assistants, deep research.
- RBAC, SSO, permission syncing, easy on-prem hosting to make it work for larger enterprises.
Through building features like deep research and code interpreter that work across model providers, we've learned a ton of non-obvious things about engineering LLMs that have been key to making Onyx work. I'd like to share two that were particularly interesting (happy to discuss more in the comments).
First, context management is one of the most difficult and important things to get right. We’ve found that LLMs really struggle to remember both system prompts and previous user messages in long conversations. Even simple instructions like “ignore sources of type X” in the system prompt are very often ignored. This is exacerbated by multiple tool calls, which can often feed in huge amounts of context. We solved this problem with a “Reminder” prompt—a short 1-3 sentence blurb injected at the end of the user message that describes the non-negotiables that the LLM must abide by. Empirically, LLMs attend most to the very end of the context window, so this placement gives the highest likelihood of adherence.
Second, we’ve needed to build an understanding of the “natural tendencies” of certain models when using tools, and build around them. For example, the GPT family of models are fine-tuned to use a python code interpreter that operates in a Jupyter notebook. Even if told explicitly, it refuses to add `print()` around the last line, since, in Jupyter, this last line is automatically written to stdout. Other models don’t have this strong preference, so we’ve had to design our model-agnostic code interpreter to also automatically `print()` the last bare line.
So far, we’ve had a Fortune 100 team fork Onyx and provide 10k+ employees access to every model within a single interface, and create thousands of use-case specific Assistants for every department, each using the best model for the job. We’ve seen teams operating in sensitive industries completely airgap Onyx w/ locally hosted LLMs to provide a copilot that wouldn’t have been possible otherwise.
If you’d like to try Onyx out, follow https://docs.onyx.app/deployment/getting_started/quickstart to get set up locally w/ Docker in <15 minutes. For our Cloud: https://www.onyx.app/. If there’s anything you'd like to see to make it a no-brainer to replace your ChatGPT Enterprise/Claude Enterprise subscription, we’d love to hear it!
Interesting product and best of luck with it.
> but I’m going to start by connecting GPT-4o, Claude Sonnet 4, and Qwen to provide my team with a secure way to use them
I did get a little giggle out of that because I've never heard anyone say that hooking up 3rd party llms to anything was any way secure.
Thanks for the kind words!
The key point there is that many would do it through Azure / Bedrock + locally host the open-source models. Also, all chats / indexed data lives on-prem, and there are better guarantees around retention when using the APIs directly.
What is it with these Chat apps having strange and not-real open source licenses? OpenWebUI is the same. Is there something about these chat apps that seems to make them more prone to weird and strange licenses? Just opportunist?
MIT core + "ee" (enterprise edition) commercially licensed extension subdirectory isn't that strange of a occurrence nowadays.
I also wouldn't pin it as chat app specific. Quite a few VC funded open core software has adopted that pattern post ~2020(?): cal.com, Dagster, Gitlab
Yea, the license is modeled after the Gitlab license. All of the core chat/RAG/agent logic is fully MIT, and >99% of deployments of Onyx are using the "community edition"!
Copilotkit is in the same boat. There are parts of the open source codebase that require an enterprise license to use. Basic things like "on error" handlers that are completely offline features. (They might have moved away from this, I haven't checked in a while)
New tech draws new people. New people have new ideas. Also for licenses.
The license on this project is pretty confusing. The license at the root of the project links to backend/cc/LICENSE.md which says you need a subscription license to use the code.
Can you call it open source if you need a subscription license to run / edit the code?
You don't need any subscription to run the code! By default, none of the enterprise code runs (and it can all be completely removed and the app will work as expected). Fully FOSS version here: https://github.com/onyx-dot-app/onyx-foss.
that's fair. What does the enterprise code do vs the FOSS?
All of the core chat UX + "add-ons" is in FOSS!
In the enterprise: - permission syncing - UI-based white labeling - Advanced RBAC - Usage analytics UI
As I see it has whitelisting and enterprise integrations.. as for the OS version maybe you need to roll your own. This is a usual monetization method though.
It's not really confusing at all.
Content under backend/ee requires a license, everything else is MIT Expat. Pretty standard stuff.
> Can you call it open source if you need a subscription license to run / edit the code?
MIT is open source, their other stuff isn't. Pretty clear.
Yes. Open source doesn’t mean free.
It really does, by any definition I've ever heard. I suppose the authoritative one would be [1].
A common "trick" for commercial open source software is to use a copyleft license, which restricts redistribution as part of commercial products, and to offer a paid license to get around that.
[1]: https://opensource.org/osd
GNU disagrees.
> Many people believe that the spirit of the GNU Project is that you should not charge money for distributing copies of software, or that you should charge as little as possible—just enough to cover the cost. This is a misunderstanding.
> Actually, we encourage people who redistribute free software to charge as much as they wish or can. If a license does not permit users to make copies and sell them, it is a nonfree license.
https://www.gnu.org/philosophy/selling.html
No, they must then state that it is source-available, not open source.
> We’re building an open-source chat that works
As long as you have Pricing on your website your product is not open source in the true spirit of open sourceness. It is open code for sure but it is a business and so incentive is to run it like a business which will conflate with how the project is used by the community.
Btw, there is nothing wrong with that but let's be honest here if you get this funded (perhaps it already is) who are you going to align your mission with - the open source community or shareholders? I don't think you can do both. Especially if a strong competitor comes along that simply deploys the same version of the product. We have seen this story many times before.
Now, this is completely different from let's say Onyx being an enterprise search product where you create a community-driven version. You might say that fundamentally it is the same code but the way it is presented is different. Nobody will think this is open-source but more of "the source is available" if you want to check.
I thought perhaps it will benefit to share this prospective here if it helps at all.
Btw, I hear good things about Onyx and I have heard that some enterprises are already using it - the open-source version.
Great to hear that you've heard good things. And yea we have many large (>1k+) teams using just the open-source version (something we love to see).
> As long as you have Pricing on your website your product is not open source in the true spirit of open sourceness.
It's an MIT license. That IS open source.
If they have a commercial strategy - that's a GoodThing. It means they have a viable strategy for staying in business, and keeping the project maintained.
MIT == OpenSource. Pricing == Sustainable. That's a horse worth backing IMO.
Actually - if you have bunch of VCs on your back, you can't even align with your very own user base let alone any other wider community.
Do you know what's completely missing from all of these products like anything LLM and Onyx...
a mobile application that has parity on the same features that ChatGPT and Claude does...
I hope you mean "parity" no?
This is great, the value is there. I work for a F100 company that is trying (and failing) to build this in house because every product manager fundamentally misunderstands that users just want a chat window for AI, not to make their own complicated agents. Your biggest competition in the enterprise space, Copilot, has terrible UI and we only put up with it because it has access to email, SharePoint and Teams.
Haha, yea we've seen that exact story many times! Dissatisfied with Copilot and building a (not great) internal solution that is missing polish + most of the "advanced" feature set.
I immediately thought of Google's Agentspace when I saw this product. The value for me sits in its ability to do RAG via connectors.
RAG + connectors is a huge reason why people deploy Onyx (enterprise search roots means we do a pretty good job there).
Also, open-source works really here, since connectors are a long-tail game. We've tried to make it easy to add connectors (a single python interface), and as a result over half of our connectors are contributed by the community. We expect that percentage to grow over time. This means that compared to something like Agentspace, we'll very likely be connected to all of the key tools at your company (and if we aren't, you can easily add an integration).
Aren't most of the large frontier model providers SOC 2 compliant? I think AWS Bedrock is also SOC 2 compliant. Not sure why you would need to self host anything then as you'll get turnkey secure solutions from the bigger plays
I was pretty excited for Onyx as a way to stand up a useful open source RAG + LLM at small scale but as of two weeks ago it was clearly full of features ticked off a list that nobody has actually tried to use. For example, you can scrape sites and upload docs but you can’t really keep track of what’s been processed within the UI or map back to the documents cleanly.
It’s nice to see an attempt at an end to end stack (for all that it seems this is “obvious” … there are not that many functional options) but wow we’ve forgotten the basis of making useful products. I’m hoping it gets enough time to bake.
Really appreciate the feedback (and glad to hear the core concept resonated with you).
The admin side of the house has been missing a bit of love, and we have a large overhaul coming soon that I'm hoping addresses some (most?) of your concerns. For now, if you'd like to view documents that have been processed, you can check out the `Explorer` panel on the left.
In general, I'd love to hear more about what gives it that "unbaked" feel for you if you're up for a quick chat.
What does this do that OpenWebUI (or one of the many of other solutions) does not?
Broadly, I think other open source solutions are lacking in (1) integration of external knowledge into the chat (2) simple UX (3) complex "agent" flows.
Both internal RAG and web search are hard to do well, and since we've started as an enterprise search project we've spent a lot of time making it good.
Most (all?) of these projects have UXs that are quite complicated (e.g. exposing front-and-center every model param like Top P without any explanation, no clear distinction between admin/regular user features, etc.). For broader deployments this can overwhelm people who are new to AI tools.
Finally trying to do anything beyond a simple back and forth with a single tool calls isn't great with a lot of these projects. So something like "find me all the open source chat options, understand their strengths/weaknesses, and compile that into a spreadsheet" will work well with Onyx, but not so well with other options (again partially due to our enterprise search roots).
As someone building another competitor in the field, I'll relay some reasons why some of our customers ruled out OpenWebUI in their decision-making process:
- Instability when self-hosting
- Hard to get in touch with sales when looking for SLA-based contracts
- Cluttered product; Multiple concepts seemingly serving the same purpose (e.g. function calling vs. MCP); Most pre-MCP tools suffer from this
- Trouble integrating it with OIDC
- Bad docs that are mostly LLM generated
why use a name, although spelled differently onyx vs onnx, that's already used and known in the ML/AI community?
That's a fair point! We were aware of onnx, but felt it was okay since they are very different products so we felt that there wouldn't be too much confusion (people generally know which onyx/onnx they are looking for).
Seems like AnythingLLM
... just because OpenAI call their product "ChatGPT" doesn't mean the term "chat" should now mean "interacting with an LLM".
Congrats on the launch!
We are building a competing open source tool[0] with a very similar focus (strongly relying on interoperable standards like MCP; built for enterprise needs, etc.), though bootstrapping with customers rather than being VC funded. It's nice to see a competitor in the field following similar "OSS Friends" principles, while many of the other ones seem to have strong proprietary tendencies.
(Small heads up: The "view all integrations" button goes to a 404)
[0] https://erato.chat/
how is this different from Librechat?
Some of the key differences:
1/ large connector suite + good RAG. Answers at scale is hard, and from our enterprise search roots, we've spent a lot of time with it. It's something that many teams expect from their chat UI.
2/ deep research + open-source code interpreter.
3/ simpler UX. LibreChat has a lot of customizability exposed front and center to the user, which is great for the power user but can be overwhelming for someone new to using AI systems.
LibreChat has recently been acquired by ClickHouse, so who knows what their future holds.
Onyx?
And no one bothered to say anything to them?
This is awesome and love that its open source!
Thanks! Open source is awesome :)
A bit like mastra.ai - my goto SOTA solution for these kind of LLM flow coordinations (though more dev-focused). (yes I realise this is more user-facing)
Congrats on the launch! Every enterprise deserves to use a beautiful AI chat UI (and Onyx is a fantastic and easy to try option).
Honestly surprised something like this can get funded
My understanding of YC is that they place more emphasis on the founders than the initial idea, and teams often pivot.
That being said, I think there is an opportunity for them to discover and serve an important enterprise use case as AI in enterprise hits exponential growth.
"Chat UI" can "feel" a bit thin from an eng/product when you initially think about, and that's something we've had to grapple with over time. As we've dug deeper, my worry about that has gone down over time.
For most people, the chat is the entrypoint to LLMs, and people are growing to expect more and more. So now it might be basic chat, web search, internal RAG, deep research, etc. Very soon, it will be more complex flows kicked off via this interface (e.g. cleaning up a Linear project). The same "chat UI" that is used for basic chat must (imo) support these flows to stay competitive.
On the engineering side, things like Deep Research are quite complex/open-ended, and there can be huge differences in quality between implementations (e.g. ChatGPTs vs Claude). Code interpreter as well (to do it securely) is quite a tricky task.
w24, those were different times.
Yeah that's like so long ago. But yeah, good luck competing with ChatGPT.
There are many markets (Europe), and highly regulated industries with air-gapped deployments where the typical players (ChatGPT, MS Copilot) in the field are having a hard time.
On another axis, if you are able to offer BYOK deployments and the customers have huge staff with low usage, it's pretty easy to compete with the big players due to their high per-seat pricing.
There are also many teams we work with that want to (1) retain model flexibility and (2) give everyone at the company the best model for the job. Every week? a model from a different provider comes out that is better at some tasks than anyone else. It's not great to be locked out from using that model since you're a "ChatGPT" company.
why?
there's a million other project just like this one, many that are much more advanced and mature, including from Vercel. There's no moat.
Agree that's a lot of other projects out there, but why do you say the Vercel option is more advanced/mature?
The common trend we've seen is that most of these other projects are okay for a true "just send messages to an AI and get responses" use case, but for most things beyond that they fall short / there a lot of paper cuts.
For an individual, this might show up when they try more complex tasks that require multiple tool calls in sequence or when they have a research task to accomplish. For an org, this might show up when trying to manage access to assistants / tools / connected sources.
Our goal is to make sure Onyx is the most advanced and mature option out there. I think we've accomplished that, so if there's anything missing I'd love to hear about it.
[dead]
I've been using Cherry Studio, works great.
Sorry, but why use this over oobabooga/sillytavern?
Why do we have to yet again poorly copy an oversimplified UI?
The value of local models comes from their huge amount of settings/control that they offer. Why must we throw that all away?
Yet again, the world waits for good UI/UX for pro/prosumers with AI systems. No one is learning from ComfyUI, Automatic1111, or SillyTavern. No, LM-Studio is not actually prosumer
We're definitely looking to add back some of that flexibility / customizability. I don't think you have to sacrifice a nice, simple UI to provide what power users are looking for.
For now, the main reasons for a prosumer to use over oobabooga/sillytavern are around the base tool set we provide and the "agent loop". If you ever want to use your single chat interface to do data analysis (code interpreter), multi-step realtime research (deep research), or RAG over large scale data (hybrid search), Onyx would be a particularly good choice.
You are not the target user.
The target customers are enterprises where 70% of the target users have never used an AI system before and have to go through a AI training before being allowed access to the system.
Congrats on the launch!
Can you clarify the license and if this actually meets the definition of Open Source as outlined by the OSI [1] or if this is actually just source available similar to OpenWebUI?
Specifically can / does this run without the /onyx/backend/ee and web/src/app/ee directories which are licensed under a proprietary license?
1 - https://opensource.org/licenses
Yes, it absolutely does! The chat UX, add-ons (deep research, code interpreter, RAG, etc.), and SSO are MIT licensed. Most deployments of Onyx are using the pure FOSS version of Onyx. Many individuals / teams have done extensive white labeling (something that OpenWebUI doesn't allow).
We have https://github.com/onyx-dot-app/onyx-foss, for a fully MIT licensed version of the repo if you want to be safe about the license/feel freedom to modify every file.