AnythingLLM is the most-starred self-contained local RAG app with a built-in UI on GitHub, bar none. As of late July 2026 it sits at 63,984 stars, 7,015 forks, MIT license, primary language JavaScript, latest release v1.15.0, just over three years on from the first commit on June 4, 2023, maintained by the Mintplex Labs team. What it does in one sentence: install it and you get a GUI-drag in documents and chat, with a built-in vector store and workspace isolation, no need to build a RAG pipeline yourself. The biggest selling point is full-stack local: hook up Ollama for a local model + LanceDB as a local vector store, and data never leaves your machine. The desktop build suits individuals; the Docker build brings multi-user and permissions for teams. The official one-liner is "the all-in-one AI app you were looking for"-read it as a privately-deployable ChatGPT.
What Pain It Solves
Anyone who's tried to feed company docs into an LLM has hit these: dump a whole PDF into the prompt and you blow the context window, burn tokens, and get hallucinations; roll your own RAG with LangChain and you're hand-writing chunking, embedder selection, vector DB wiring, retrieval logic, and source citation-each step more engineering than the business logic itself; use ChatGPT and legal/contracts/internal code can't be uploaded for leakage risk; team use needs multi-user workspace isolation, which open-source RAG scripts don't have; giving an agent web-search, code-execution, or custom-tool capability means wrapping another pile of glue. AnythingLLM packs all of this into one out-of-the-box app: install, pick a model, drag in docs, create a workspace, chat-vector DB built in, source citation built in, multi-user permissions tiered, agent skills pluggable. The core shift is from "assemble your own RAG pipeline" to "install and use"-the engineering complexity is pushed down into the product, you just feed it and ask.
Workspace Isolation: Slice Messy Docs into Separate Knowledge Bases
This is its fundamental split from the "one chat box holds everything" approach. AnythingLLM's core unit of organization is the workspace, each with its own document set, vector slices, embedder, even its own LLM-product team one KB, legal one, R&D one, no cross-contamination, and no dilution of retrieval precision because one department dumped in junk docs. Multi-user is enabled on the Docker build, with permissions tiered-admins, document admins, regular users-down to "this user can only query this workspace, can't edit docs." That's a hard requirement for team deployments-single-user RAG scripts have no user model, but in a company you need to govern who sees contracts and who uploads SOPs. Go further: each workspace's LLM and embedder is independently configurable-legal docs are small, use cheap GPT-4o-mini + OpenAI embedder; the R&D codebase is large, run local Ollama with qwen2.5 + a local embedder, saving money and keeping data on-domain. There's also a "memories" mechanism: the LLM remembers important info about you or the workspace, carrying context across sessions without re-stating background each time.
RAG Full-Stack Built In: Docs to Vector Store in One Pipeline
It owns the whole RAG chain. On the ingestion side, it natively handles PDF / Word / Excel / PPT / web / CSV / Markdown / TXT-drag in, auto-chunk, vectorize, index; at chat time it carries source citation, telling you which doc and which passage an answer came from, not made up. On the vector DB side, LanceDB is the default-zero config, out of the box, enough for individuals and small teams; scale up and you can swap to Pinecone, Chroma, Weaviate, Qdrant, Milvus, Zilliz, Astra DB, or PGVector-nine options listed officially, pick by your existing infrastructure. The model layer is its most flexible part: LLMs cover dozens of providers-closed-source OpenAI / Anthropic / Google Gemini / Azure / AWS Bedrock / xAI, open-source Ollama / LM Studio / LocalAI / KoboldCPP, and China-side DeepSeek / Moonshot / Z.AI / Gitee AI, domestic channels well-covered. Embedders work the same-OpenAI / Voyage / Cohere / local all swappable. Then there's Dynamic Model Routing-route different conversations to the best provider and model by rules you define, e.g. simple questions to a cheap model, heavy reasoning to Claude. For large document sets the team claims optimizations that lower cost and speed up responses vs. peer chat UIs. Voice is covered too: built-in browser TTS/STT, OpenAI TTS, ElevenLabs, plus audio transcription for documents.
Agent Skills and MCP: Beyond Just Reading Docs
A RAG that only reads docs is a "Q&A bot"; AnythingLLM wants to go one step further as an agent platform. The workspace ships with built-in agents that can browse the web, run code, and call custom tools, not just retrieve the local KB. A few capabilities worth flagging. One, the No-code AI Agent builder: visually drag-and-drop to compose agent flows, no code needed to wire up complex workflows. Two, Intelligent Skill Selection: hang "unlimited" tools on the model while the team claims up to 80% lower token usage per query-by smart-filtering which tool to call this turn rather than stuffing every tool description into the prompt. Three, MCP compatibility (Model Context Protocol): tap into the Anthropic-pushed tool protocol ecosystem and reuse existing MCP servers. Four, Scheduled Tasks: run recurring tasks or prompts on a cron schedule with full agent capability-e.g. every morning auto-summarize the workspace's new docs and push to Slack. Five, the Custom Embeddable Chat widget: embed the whole chat window into your site as a customer-service bot (Docker build only). Add Dynamic Model Routing and the cross-session memories from the RAG section above, and the picture is not "a chat box" but "an AI app platform that schedules tools, runs on a timer, and can be embedded outward."
Three-Minute Setup
# 1. Docker one-liner (team pick)
docker run -d -p 3001:3001 \
-v /your/storage:/app/server/storage \
mintplex-labs/anything-llm
# 2. Wait 30s, open browser
open http://localhost:3001 # or visit server-ip:3001
# 3. First-run UI: pick LLM provider
# - personal offline: pick Ollama, set http://host.docker.internal:11434
# - cloud: paste OpenAI / Anthropic API key
# 4. Create workspace -> drag PDF -> chatThe desktop build is even faster (individual pick): download the Mac / Windows / Linux installer from anythingllm.com/download, double-click to install, get guided to pick a model on startup (Ollama is the default push). Running from source works too-the monorepo splits into frontend (Vite + React), server (Node Express, manages vector DBs and LLMs), collector (Node Express, parses documents), docker, embed, and browser-extension-six sections; yarn setup fills the .env files, then yarn dev:server / yarn dev:frontend / yarn dev:collector to boot the three processes. If that sounds like work, just use Docker or the desktop build-most people never need to touch source.
Who It's For + Five Pitfalls
For: teams building a privately-deployable internal knowledge base; privacy-sensitive scenarios needing data to stay on-domain (medical, legal, internal code); individuals wanting one entry point to query all their PDFs, notes, and web pages as a second brain; teams needing a RAG backend to embed into a site as a customer-service bot.
Five pitfalls. One, only the Docker build has full multi-user and the embed widget-the desktop build is single-user single-machine; pick the right form factor for team use. Two, the default LanceDB is fine for small teams, but at hundreds of thousands of docs you'll want a serious vector DB like Pinecone or Milvus-LanceDB isn't built for large-scale retrieval, and when you pick an embedder match its dimensions to the vector DB or you'll misconfigure. Three, running Ollama locally needs real hardware-a 7B model wants at least 8GB VRAM, 13B wants 16GB; don't expect a thin laptop to run a big model smoothly, fall back to a cloud API if needed. Four, manage agent-skill and MCP-tool permissions carefully-agents can run code and call custom tools, meaning they can touch your real environment; in production narrow the scope of sensitive tools and sandbox them so the LLM doesn't delete files or fire off API calls by mistake. Five, telemetry is on by default-anonymized reporting of install type, doc add/remove events, vector DB type, LLM provider, and chat events (no content); in privacy-sensitive setups turn it off in the sidebar's Privacy section or set DISABLE_TELEMETRY=true. Even with it off, using external LLMs / vector DBs still sends outbound traffic to those providers-read their ToS.
vs. the Competition
Against Dify, both are privately-deployable LLM app platforms, but the emphasis differs: Dify leans "workflow orchestration," visually wiring nodes to chain prompts / tools / branches, good for building complex workflows and external APIs; AnythingLLM leans "out-of-the-box doc Q&A + workspace knowledge base," with a smoother drag-doc-and-chat experience, with agent capability added on top. Dify is stronger on workflows, AnythingLLM thicker on out-of-the-box doc RAG. Against LangChain, LangChain is a library not a product-it gives you RAG components to assemble, you write code, build the UI, manage the vector DB yourself, flexible but heavy engineering; AnythingLLM packages all of that into an app, install and go, less flexible but faster to land-go LangChain for deep customization, AnythingLLM to ship fast. Against ChatGPT (including its file-upload feature), ChatGPT is closed-source SaaS-docs hit the cloud, enterprise-sensitive data can't go there; AnythingLLM is MIT-licensed, fully local-stack offline-capable, multi-user with permissions, privately deployable. Against fellow "ChatGPT-with-your-data" open-source options (PrivateGPT, Quivr), AnythingLLM has the broadest LLM / vector DB / embedder support, the most features (memories, scheduled tasks, MCP, embed widget, model routing), and the most active community (63k stars). One line: for workflow orchestration pick Dify, for building a custom pipeline from scratch pick LangChain, for install-and-chat that's still privately deployable, pick AnythingLLM.
References
- AnythingLLM GitHub repo (63,984 stars, MIT, JavaScript): https://github.com/Mintplex-Labs/anything-llm
- Official site (desktop downloads): https://anythingllm.com/download
- Official docs (features / agent / model router / MCP): https://docs.anythingllm.com
- Docker deployment guide: https://github.com/Mintplex-Labs/anything-llm/blob/master/docker/HOW_TO_USE_DOCKER.md
- Bare-metal deploy (no Docker): https://github.com/Mintplex-Labs/anything-llm/blob/master/BARE_METAL.md