# Open-source DeepWiki: comparison

> Self-hosted generators that turn a code repository into a browsable, AI-written wiki.

Canonical page: https://llms-technical-reviews.com/compare/open-source-deepwiki/

## How is a repository ingested and chunked?

Both tools work on a full checkout, but they decide very differently what the model gets to see.

[deepwiki-open](/p/deepwiki-open/) does a shallow `--depth=1` clone for GitHub, GitLab and Bitbucket (tokens injected into the URL) or reads a local path in place. `iterate_files()` keeps files whose extension is on an allow-list and drops the excluded dirs in `repo.json`. Every remaining file is split into 350-word chunks with 100 words of overlap, each tagged with its line range, and embedded. Files above about 82K tokens are skipped. There is no file-count cap, so cost and indexing time grow with the repo.

[OpenDeepWiki](/p/opendeepwiki/) imports Git (LibGit2Sharp, with retries), ZIP uploads or allow-listed local directories. It does not chunk anything. A scan plan limits only the directory tree shown to the catalog agent: depth 2 to 4 and about 500 files (350 for repos over 20,000 files), with `.gitignore` and hidden paths filtered. The agent can still open any file later with `ReadFile` or `Grep`.

Choose deepwiki-open if you want every file indexed up front and searchable by similarity. Choose OpenDeepWiki if your repos are large and you would rather have the model pick a few files than pay to embed all of them. OpenDeepWiki also handles ZIP uploads and keeps branches and commits for incremental runs.

Per-project answers: https://llms-technical-reviews.com/open-source-deepwiki/q/ingestion/index.md

## How is retrieval (RAG) implemented?

This is the core split in the category.

[deepwiki-open](/p/deepwiki-open/) is classic RAG. It uses adalflow with a `FAISSRetriever` (`top_k: 20`) over embeddings from OpenAI `text-embedding-3-small` (256 dims) by default, or Google, Ollama or Bedrock through `DEEPWIKI_EMBEDDER_TYPE`. The index is a pickle on disk. In `research_chat()`, the retrieved chunks are grouped by file, labelled with `[lines a-b]`, and placed between `<START_OF_CONTEXT>` tags. The query is the last user message, which for wiki pages is the whole page prompt. If that message is over 7,500 tokens, retrieval is skipped completely.

[OpenDeepWiki](/p/opendeepwiki/) has no embeddings and no vector store. The agent reads code through tool calls: `ListFiles`, `Grep` and `ReadFile` on `GitTool`, plus `ReadDoc` over pages it has already generated when it answers chat. During generation a budget wrapper limits this to 6 calls per page (4 for the catalog) and 240 lines per read.

RAG is cheaper per page and finds scattered snippets that match a question, but the model never sees whole files. Agentic reading gives real file context and precise source lists, but it depends on the model searching well within a small budget. For Q&A over very large codebases, deepwiki-open's index finds more. For pages that explain one subsystem in depth, OpenDeepWiki's approach usually reads more relevant code.

Per-project answers: https://llms-technical-reviews.com/open-source-deepwiki/q/retrieval/index.md

## How is the wiki structure (table of contents) determined?

Both tools ask an LLM for the table of contents. They differ in what the model sees and how the answer comes back.

[deepwiki-open](/p/deepwiki-open/) makes one plain completion call. `build_structure_prompt()` includes the complete filtered file list and the README, and asks for `<wiki_structure>` XML with 4 to 6 pages (concise) or 8 to 12 pages in sections (comprehensive). Each page lists its relevant files. `parse_wiki_structure()` tolerates code fences, stray `&` and truncated output. The model cannot check any file, and in this version an argument-order bug turns request exclusions into inclusions for this step.

[OpenDeepWiki](/p/opendeepwiki/) runs an agent with the `catalog-generator` prompt. Its input is a TOON-encoded tree limited by the scan plan, the README (up to 4,000 chars), key files and entry points. It can make up to 4 confirmation calls (`ListFiles`, `Grep`, `ReadFile`) and then saves a nested JSON catalog with `WriteCatalog`. Page count is not fixed. The prompt asks for "right-sized" coverage, so big repos get many more pages (61 for OpenBot in our runs, against 12 from deepwiki-open).

deepwiki-open gives a predictable, small wiki at a known cost. OpenDeepWiki gives a catalog that grows with the codebase and is checked against a few real files.

Per-project answers: https://llms-technical-reviews.com/open-source-deepwiki/q/structure/index.md

## How are individual pages generated?

[deepwiki-open](/p/deepwiki-open/) builds each page with one LLM call. `build_page_prompt()` passes the title and the page's file links. The model must open with a `<details>` source list, use Mermaid diagrams, and cite as `Sources: [path:10-25]()`. Context is the top-20 retrieved chunks. `post_process_wiki_content()` then turns those citations into GitHub, GitLab or Bitbucket links with line anchors. Pages run with `DEEPWIKI_WIKI_PAGE_CONCURRENCY` (default 1) and two retries, then an error placeholder. The whole wiki is saved as one JSON cache file. A cached wiki is never updated, only deleted and rebuilt.

[OpenDeepWiki](/p/opendeepwiki/) runs an agent per page with the `content-generator` prompt. It has 6 source-tool calls and writes with `WriteDoc` plus up to 4 `AppendDoc` calls, so a page can be longer than one response. Links use a File Reference Base URL, and `DocTool` stores the files actually read as `SourceFiles`. The prompt includes detailed Mermaid syntax rules, and two normalizers clean the output. The first page runs alone to warm the prompt cache, then 5 pages run in parallel, each with a 60-minute timeout. Pages that already exist are skipped, and new commits trigger `IncrementalUpdateAsync()` on the affected pages only.

deepwiki-open is simpler and has stronger line-level citation repair. OpenDeepWiki is better for long pages, large repos and wikis that must follow the code over time.

Per-project answers: https://llms-technical-reviews.com/open-source-deepwiki/q/page-generation/index.md

## How are model providers configured?

[deepwiki-open](/p/deepwiki-open/) reads providers from `api/config/generator.json`. Each provider has a `ChatStreamer` subclass that registers itself: Google (default, `gemini-2.5-flash`), OpenAI, OpenRouter, Ollama, Bedrock, Azure, DashScope, LiteLLM and Anthropic. Each request names one provider and model, and that pair is used for the outline, every page and chat, so you cannot use a different model per stage. The embedder is set separately with `DEEPWIKI_EMBEDDER_TYPE`. For OpenAI-compatible endpoints you set a base URL, or go through LiteLLM. Ollama allows fully local runs. Its OpenRouter client is non-streaming with a 60 s timeout, which this site had to raise for whole-repo outlines.

[OpenDeepWiki](/p/opendeepwiki/) keeps providers and models as database rows managed in an admin console. It seeds them from 29 built-in presets. `AiProviderResolver` reduces them to five request types (OpenAI, OpenAI Responses, Azure OpenAI, Anthropic, DeepSeek), and any OpenAI-compatible URL works. Catalog, content and translation each have their own model and provider binding. Per-model overrides cover thinking modes and request parameters. One gotcha: a built-in preset such as `openrouter` starts disabled, and the env-var migration does not enable it.

Pick deepwiki-open for quick, file-based setup and local models. Pick OpenDeepWiki if you need a cheap model for the catalog and a stronger one for content, or if you manage models centrally for a team.

Per-project answers: https://llms-technical-reviews.com/open-source-deepwiki/q/providers/index.md

## How is interactive Q&A / chat implemented?

[deepwiki-open](/p/deepwiki-open/) serves chat over a WebSocket at `/ws/chat`, with an HTTP streaming fallback. Each turn runs the same `research_chat()` RAG path as page generation: FAISS top-20 chunks plus the history the client sends, wrapped in `<turn>` blocks. The server stores no history. Deep Research is a five-step prompt sequence (plan, updates, conclusion) that the frontend drives turn by turn. A separate Codemap mode builds a step-by-step code guide in two LLM calls and streams it as NDJSON. There is no MCP server.

[OpenDeepWiki](/p/opendeepwiki/) serves chat at `POST /api/v1/chat/stream` as Server-Sent Events, with typed `content`, `thinking`, `tool_call`, `tool_result` and `done` events. The agent reads the generated pages first (`ReadDoc`) and then the source (`ReadFile`, `Grep`, `ListFiles`). It can also call configured MCP servers and skills. Sessions and messages are saved in the database. There is no separate deep-research loop; the system prompt asks for step-by-step reasoning instead. It also runs an MCP server. The global `/api/mcp` endpoint lists repositories, routes a question to them by token scoring, and searches or reads generated pages. The per-repo endpoint `/api/mcp/{owner}/{repo}` offers `SearchDoc`, `GetRepoStructure` and `ReadFile`, but its `SearchDoc` uses a literal SQL `LIKE`.

For a chat that you can use in an IDE agent, or that must read actual source files, OpenDeepWiki is the stronger choice. For quick similarity-based answers and the guided Deep Research and Codemap modes in a single-user setup, use deepwiki-open.

Per-project answers: https://llms-technical-reviews.com/open-source-deepwiki/q/qa/index.md

## Projects

- [AsyncFuncAI/deepwiki-open](https://llms-technical-reviews.com/p/deepwiki-open/index.md) — FastAPI service and Next.js UI that embed a repo into a FAISS index, then generate a cited, Mermaid-heavy wiki with one prompt per page.
- [AIDotNet/OpenDeepWiki](https://llms-technical-reviews.com/p/opendeepwiki/index.md) — ASP.NET Core service in which tool-calling LLM agents read a repo and write its wiki, mind map and translations into SQLite or PostgreSQL.