TL;DR: AnythingLLM is a free, open-source (MIT-licensed) app from Mintplex Labs that lets you chat with your own documents using AI that runs on your computer. The desktop and self-hosted editions cost nothing and have no usage caps; with a local model it works completely offline. The honest catches: you supply the model (built-in local options make this easy), quality depends on your hardware and model choice, and the vendor’s optional managed cloud hosting starts at $50/month for people who don’t want to run it themselves.
What is AnythingLLM?
AnythingLLM organizes document chat into workspaces. Each workspace holds its own documents, chat history, and settings. Drop in PDFs, text files, and other documents, and the app indexes them (the technique called retrieval-augmented generation, or RAG) so the AI answers from your material — “What did this contract say about renewal?” — instead of from general training data. On top of chat, it includes AI agents that can use tools, a no-code agent flow builder, and a developer API.
The defining choice is where the AI runs. AnythingLLM ships with local model support built in, stores everything on your machine by default, and can run 100% offline. Prefer a cloud model instead? It connects to 20+ LLM providers — just know those providers see your prompts under their own terms, and bill you separately for usage.
What is actually free on AnythingLLM?
- The full desktop app for Mac, Windows, and Linux — free and open source under the MIT license.
- The self-hosted Docker edition — also open source, adding multi-user support and admin controls for teams running their own instance.
- Local models, document chat (RAG), and agents with no message caps, no credit meter, and no account requirement.
- Offline operation — with a local model, embedder, and vector database, nothing needs to leave your device.
What is paywalled?
Convenience, not the core product. Mintplex Labs sells managed cloud hosting (starting at $50/month) for businesses that want AnythingLLM without running a server, and there are optional paid extras around the ecosystem. And if you connect a cloud model provider, that provider’s API usage is billed by them. Run it locally with local models, and the ongoing cost is $0 plus your electricity.
Pros and cons
Pros
- Genuinely free with no usage caps — a rarity among document-chat tools
- Private by design: local-first, offline-capable, open source and auditable
- Model freedom: local models or 20+ cloud providers, your choice per workspace
- Workspaces keep separate document sets cleanly apart
Cons
- Answer quality rides on the model you pick and your computer’s horsepower
- Initial setup (choosing a model and embedder) asks more of you than a hosted tool
- Local models on modest hardware can be slow with large documents
Who is AnythingLLM best for?
Privacy-conscious professionals — lawyers, consultants, researchers, small-business owners — sitting on documents they’d never paste into a cloud chatbot, plus tinkerers who want a capable local AI front-end without a subscription. If you want zero setup and don’t mind the cloud, a hosted document-chat tool is easier. If your files staying on your machine is the point, AnythingLLM is the free answer.
Bottom line: AnythingLLM is what “free” should mean: the complete app, open source, no caps — you bring the model and the machine. For private document chat, nothing hosted at $0 comes close.
Frequently asked questions
Is AnythingLLM really free?
Yes. The desktop app and the self-hosted Docker edition are free and open source (MIT license). Managed cloud hosting and optional extras are the paid parts.
Does AnythingLLM work offline?
Yes — with a local model, embedder, and vector database, it can run 100% offline and your documents never leave your computer.
Do I need an API key or account?
Not for local use. API keys are only needed if you choose to connect a cloud model provider, which then bills its own usage.
What can I chat with?
Documents you add to a workspace — PDFs, text files, and more — plus data connectors such as GitHub and YouTube on the desktop app.