# SciPhi-AI/R2R

> Python RAG server on Postgres/pgvector: hybrid and graph search, cited streaming answers, and RAG and research agents behind a REST API.

- Category: [RAG engines](https://llms-technical-reviews.com/rag/)
- Repository: https://github.com/SciPhi-AI/R2R (reviewed at commit `9c5a94d151f90876bd7eb860f300a8fd662dc481`, 2025-11-06)
- Stars: 8010 · Language: Python · License: MIT
- Canonical page: https://llms-technical-reviews.com/p/r2r/

## Overview

R2R ("RAG to Riches") from SciPhi is a self-hosted RAG server. It is a FastAPI application with a `/v3` REST API, Python and JavaScript SDKs, and a Next.js dashboard. It covers the whole loop: upload documents, parse and chunk them, embed them into PostgreSQL with pgvector, search with vector, full-text or hybrid retrieval plus an optional knowledge graph, and answer with an LLM that cites its sources. On top of plain `rag` it offers an `agent` endpoint with two modes. "rag" is a tool-using retrieval agent. "research" adds reasoning, critique and Python-execution tools.

The architecture is a provider registry. Every concern (auth, database, embeddings, LLM, ingestion, OCR, orchestration, file storage, email, scheduler) is an abstract provider. A TOML config selects the concrete one, and a factory builds it at startup. PostgreSQL does almost everything: chunks and vectors, a generated `tsvector` column for keyword search, documents, users, collections, conversations, prompts and graph tables. There is no separate vector database. LLM and embedding calls go through LiteLLM by default, so most providers work by changing a model string.

R2R suits teams that want a ready multi-user RAG backend with document management, collections-based access control and an API they can build a product on. The pinned commit is from November 2025.

## Architecture

```mermaid
flowchart LR
  C["SDK / dashboard / MCP"] --> API["FastAPI /v3 routers"]
  API --> SVC["Services: ingestion, retrieval, graph, management"]
  SVC --> ORCH["Orchestration: simple (inline) or Hatchet"]
  ORCH --> ING["R2RIngestionProvider: parsers + splitter"]
  ING --> OCR["Mistral OCR / VLM / Unstructured"]
  ING --> EMB["Embedding provider (LiteLLM)"]
  EMB --> PG["Postgres + pgvector"]
  SVC --> PG
  SVC --> LLM["LLM provider (LiteLLM)"]
  SVC --> AG["Agents: RAG / research"]
  AG --> LLM
  PG --> CL["Graph clustering service (Leiden)"]
```

| Component | Path | Role |
|---|---|---|
| App and routers | `py/core/main/app.py`, `py/core/main/api/v3/` | Documents, chunks, collections, graphs, indices, prompts, retrieval, conversations, users, system |
| Config | `py/r2r/r2r.toml`, `py/core/configs/*.toml`, `py/core/main/config.py` | Default config plus named variants (`full`, `ollama`, `azure`, ...) |
| Provider factory | `py/core/main/assembly/factory.py` | Builds each provider from config |
| Services | `py/core/main/services/` | `IngestionService`, `RetrievalService`, `GraphService`, `ManagementService`, `AuthService` |
| Orchestration | `py/core/main/orchestration/{simple,hatchet}/` | Ingestion and graph workflows, inline or as Hatchet tasks |
| Ingestion | `py/core/providers/ingestion/`, `py/core/parsers/` | Per-type parsers, text splitters, Unstructured alternative |
| Database | `py/core/providers/database/` | Postgres handlers for chunks, documents, graphs, filters, limits |
| Agents | `py/core/agent/`, `py/core/base/agent/tools/` | RAG and research agents, streaming, citations, tool registry |
| Clients | `py/sdk/`, `js/`, `py/r2r/mcp.py` | Python and JS SDKs, a small MCP server |

## How a request flows

Upload one PDF, then ask a question with `POST /v3/retrieval/rag`:

1. **Upload.** `create_document` checks the per-type size limit, stores the raw bytes with the file provider, and registers the document with `ingest_file_ingress`. With `run_with_orchestration`, it calls `orchestration.run_workflow("ingest-files")`. If that fails, it falls back to the inline ingestor ([documents_router.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/api/v3/documents_router.py#L455-L585)).
2. **Orchestrate.** The default `simple` provider has no worker. `run_workflow` simply awaits the workflow function, so ingestion happens inside the HTTP request ([simple.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/orchestration/simple.py#L1-L61)). The `full` config switches to Hatchet for background workers.
3. **Parse, summarize, embed, store.** `ingest_files` moves the document through PARSING, AUGMENTING (an LLM document summary unless `skip_document_summary`), EMBEDDING, STORING and SUCCESS. It then assigns the document to the user's default collection ([ingestion_workflow.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/orchestration/simple/ingestion_workflow.py#L25-L100)). Parsing picks a parser from `DEFAULT_PARSERS` by type, and chunking defaults to a recursive character splitter of 1,024 characters with 512 overlap ([base.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/ingestion/r2r/base.py#L31-L76), [L160-L215](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/ingestion/r2r/base.py#L160-L215)).
4. **Search.** `RetrievalService.rag` first calls `search`, which dispatches on `search_strategy` to basic, HyDE or RAG-Fusion ([retrieval_service.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/services/retrieval_service.py#L259-L325)). `_vector_search_logic` embeds the query, runs semantic, full-text or hybrid search on `PostgresChunksHandler`, then calls `arerank` and prefixes each chunk with its document title ([L651-L730](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/services/retrieval_service.py#L651-L730)).
5. **Fuse.** `hybrid_search` runs the vector query and the `websearch_to_tsquery` query separately. It merges them in Python with weighted reciprocal rank fusion, using `semantic_weight`, `full_text_weight` and `rrf_k` ([chunks.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/database/chunks.py#L538-L640)).
6. **Generate.** The results go into a `SearchResultsCollector` and are formatted with short source ids. The `system` and `rag` prompts are loaded from the database and filled with `query` and `context`. Without streaming, one `aget_completion` call is made, and bracketed short ids in the answer become `Citation` objects with full payloads ([L985-L1080](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/services/retrieval_service.py#L985-L1080)). With `stream: true`, the same flow yields SSE events instead.

## Key components

### Ingestion and parsing

`R2RIngestionProvider` maps about 35 document types to parser classes: PDF, Office formats, email (EML/MSG), EPUB, HTML, Markdown, code files, images and MP3. PDFs have extra parsers (`zerox` VLM and `ocr`) configured in the default TOML. Images and audio go through the configured VLM and the Whisper model. The `unstructured` provider is a drop-in alternative for layout-aware partitioning. Only `recursive` and `character` splitting are implemented. `basic` and `by_title` raise `NotImplementedError` in the R2R provider. Optional chunk enrichment rewrites each chunk with neighbouring context, and it is off by default ([r2r.toml](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/r2r/r2r.toml#L82-L132)).

### Storage and indexes

One chunks table per project schema holds `vec vector(N)`, an optional `vec_binary bit(N)` for INT1 quantization, `text`, `metadata JSONB`, a `collection_ids UUID[]` with a GIN index, and a generated English `tsvector` ([chunks.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/database/chunks.py#L140-L182)). HNSW or IVFFlat vector indexes are created explicitly through the indices API. The default embedding is `openai/text-embedding-3-small` at 512 dimensions via LiteLLM.

### Reranking

Reranking is a method on the embedding provider. The LiteLLM provider supports only a HuggingFace TEI rerank endpoint, configured with `rerank_model` and `rerank_url` or `HUGGINGFACE_API_BASE`. Without them, `arerank` just truncates the results ([litellm.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/embeddings/litellm.py#L40-L62), [L258-L288](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/embeddings/litellm.py#L258-L288)).

### Knowledge graph

Graph extraction is an explicit step per document or collection. An LLM extracts entities and relationships, describes them and embeds the descriptions. Community detection is delegated to a separate clustering container (`ragtoriches/cluster-prod`) that runs Leiden. Communities are then summarized by LLM and become searchable next to chunks ([graphs.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/database/graphs.py#L2450-L2481)). `automatic_extraction` is true in the default config, but the `simple` workflow only logs that it is "not yet implemented" ([ingestion_workflow.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/orchestration/simple/ingestion_workflow.py#L150-L200)).

### Agents and streaming citations

`AgentFactory` picks a RAG or research agent, with streaming or XML-tool variants. `R2RAgent.arun` loops LLM calls and tool calls up to `max_iterations`, then summarizes if the loop never completed ([base.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/agent/base.py#L100-L151)). The streaming agent emits `message` events. It scans the accumulated text for bracketed short ids, and sends a `citation` event with the source payload the first time each id appears ([L440-L510](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/agent/base.py#L440-L510)). Default RAG tools are `search_file_descriptions`, `search_file_knowledge` and `get_file_content`. Web search (Serper), scraping (Firecrawl) and Tavily tools are opt-in.

## Extending it

- **Swap providers by config.** Pick a TOML with `R2R_CONFIG_NAME` or `R2R_CONFIG_PATH`. Bundled variants cover Ollama, LM Studio, Azure, Gemini and the Hatchet `full` stack.
- **Prompts.** System, task, graph and enrichment prompts live in Postgres and are editable through `/v3/prompts` without a redeploy.
- **Custom agent tools.** `ToolRegistry` imports every Python file in a `user_tools` directory, and the Docker image mounts `docker/user_tools`. Add a tool name to `rag_tools` or `research_tools` to enable it.
- **Search tuning per request.** `search_settings` toggles semantic, full-text, hybrid and graph search, sets the HyDE or RAG-Fusion strategy, and applies Mongo-style metadata filters (`$eq`, `$in`, `$overlap`, ...).
- **Clients.** Python (`R2RClient`), JavaScript (`r2r-js`), and an MCP server in `py/r2r/mcp.py` that wraps search and RAG for MCP clients.

## Running it

- **Light mode.** `pip install r2r` and `python -m r2r.serve`, with an `OPENAI_API_KEY` and a reachable Postgres with pgvector. Ingestion runs inline.
- **Docker.** `docker/compose.yaml` runs Postgres, MinIO, the graph-clustering service, R2R on port 7272 and the dashboard ([compose.yaml](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/docker/compose.yaml#L40-L78)). `compose.full.yaml` adds Hatchet (engine, dashboard, RabbitMQ, its own Postgres) and an Unstructured service.
- **Defaults to change.** The default config ships `require_authentication = false` and a well-known default admin password ([r2r.toml](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/r2r/r2r.toml#L37-L44)). Set both before exposing the server.

## Strengths and caveats

- **Strength: one database.** Vectors, keyword search, graph, users and conversations all sit in Postgres, so backup and multi-tenant filtering (`collection_ids`, `owner_id`) stay simple.
- **Strength: complete API surface.** Document versioning and status, collections, user management, conversation history, prompt management and streaming citations are already built. That is the boring 70% of a RAG product.
- **Strength: flexible retrieval.** Hybrid RRF, HyDE, RAG-Fusion, graph and community search, plus web search tools, all switchable per request.
- **Caveat: inline ingestion by default.** Without Hatchet, an upload blocks its HTTP request through parsing, an LLM summary and embedding. Large files or bursts need the heavier `full` stack.
- **Caveat: hybrid fusion in Python.** Two separate SQL queries are fused in application code, and English-only `tsvector` keyword search is weak for other languages.
- **Caveat: research agent's Python tool.** `python_executor` writes model-generated code to a temp file and runs it with the server's interpreter in a subprocess, with a 10 s timeout and no sandbox ([research.py](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/agent/research.py#L358-L392)). Keep it out of `research_tools` on shared deployments.
- **Caveat: limited reranking.** Only a TEI endpoint is supported, and it is off by default.

*Sources: code at 9c5a94d, deepwiki-open wiki (12 pages), verified Q&A.*

## How SciPhi-AI/R2R answers the RAG engines questions

### How are documents parsed and chunked? (answered)

**Supported formats.** R2R supports 40+ document types via a parser registry at `py/core/providers/ingestion/r2r/base.py:40-76`. DEFAULT_PARSERS maps each `DocumentType` enum (`py/shared/abstractions/document.py:18-93`) to a dedicated parser class: BMPParser, CSVParser, DOCParser, DOCXParser, EMLParser, EPUBParser, HTMLParser, JSONParser, MDParser, MSGParser, ORGParser, BasicPDFParser (plus OCRPDFParser, VLMPDFParser, PDFParserUnstructured as extras), PPTParser, PPTXParser, RTFParser, TextParser, TSVParser, XLSParser, XLSXParser, ImageParser (GIF/JPEG/JPG/PNG/HEIC/SVG/TIFF), AudioParser (MP3), P7SParser, RSTParser, PythonParser, JSParser, TSParser, CSSParser.

**OCR and layout parsing.** The `MistralOCRProvider` (`py/core/providers/ocr/mistral.py:13-53`) wraps the Mistral OCR API for document image processing. For PDFs, alternative parsers include OCRPDFParser (local OCR via pdf2image + Mistral) and VLMPDFParser (zero-shot VLM-based). The `UnstructuredIngestionProvider` (`py/core/providers/ingestion/unstructured/base.py`) integrates with Unstructured's API or a local Unstructured service for layout-aware partitioning with options like `pdf_infer_table_structure`, `hi_res_model_name`, and `ocr_languages`. Table handling is further supported through XLSXParserAdvanced and CSVParserAdvanced.

**Chunking strategy.** The `R2RIngestionProvider` (`py/core/providers/ingestion/r2r/base.py:160-215`) builds a text splitter based on `ChunkingStrategy` (RECURSIVE, CHARACTER, BASIC, BY_TITLE). Default is `RecursiveCharacterTextSplitter` with `chunk_size=1024` and `chunk_overlap=512` (`py/core/providers/ingestion/r2r/base.py:31-35`). The recursive splitter (`py/shared/utils/splitter/text.py:1219-1289`) walks through `["

", "
", " ", ""]` separator levels. The character splitter (`py/shared/utils/splitter/text.py:620-646`) uses a configurable separator. For OCR/VLM parsers, an optional `vlm_ocr_one_page_per_chunk` mode creates one `DocumentChunk` per page rather than re-splitting.

> **Editor's note.** Correction: only `recursive` (default, 1,024 characters with 512 overlap) and `character` splitting are implemented in the R2R provider; `basic` and `by_title` raise NotImplementedError.

Citations: [py/core/providers/ingestion/r2r/base.py:31-76](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/ingestion/r2r/base.py#L31-L76) · [py/core/providers/ingestion/r2r/base.py:160-215](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/ingestion/r2r/base.py#L160-L215) · [py/core/providers/ocr/mistral.py:13-53](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/ocr/mistral.py#L13-L53) · [py/core/providers/ingestion/unstructured/base.py:86-111](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/ingestion/unstructured/base.py#L86-L111) · [py/shared/utils/splitter/text.py:1219-1289](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/shared/utils/splitter/text.py#L1219-L1289) · [py/shared/abstractions/document.py:18-93](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/shared/abstractions/document.py#L18-L93)

### How are embeddings and indexes built and stored? (answered)

**Embedding models.** Three embedding providers: `OpenAIEmbeddingProvider` (`py/core/providers/embeddings/openai.py:21-60`) supporting text-embedding-ada-002, text-embedding-3-small, and text-embedding-3-large with configurable dimensions; `LiteLLMEmbeddingProvider` (`py/core/providers/embeddings/litellm.py:25-65`) routing through LiteLLM for any provider (Amazon, HuggingFace, etc.); and `OllamaEmbeddingProvider` for local models. EmbeddingConfig (`py/core/base/providers/embedding.py:21-44`) configures provider, base_model, base_dimension, batch_size, concurrent_request_limit, and vector quantization settings. Retry with exponential backoff is built in.

**Vector store.** All vectors are stored in PostgreSQL using pgvector. `PostgresChunksHandler` (`py/core/providers/database/chunks.py:75-182`) creates tables with a `vec` column (configurable dimension), optional `vec_binary bit(N)` column for INT1 quantization, `text TEXT`, and `metadata JSONB`. A full-text search column (`fts tsvector`) is auto-generated as `to_tsvector('english', text)`. Index methods supported: IVFFlat (`IndexArgsIVFFlat` with `n_lists`) and HNSW (`IndexArgsHNSW` with `m` and `ef_construction`) via `py/shared/abstractions/vector.py:27-107`. Distance measures: cosine, L2, max-inner-product, L1, hamming, jaccard. Vector quantization types: FP32, FP16, INT1 (binary), SPARSE.

**Hybrid / keyword (BM25).** Full-text search uses PostgreSQL's `websearch_to_tsquery('english', ...)` with `ts_rank` ordering (`py/core/providers/database/chunks.py:484-536`). Hybrid search (`py/core/providers/database/chunks.py:538-640`) combines semantic and full-text via weighted Reciprocal Rank Fusion with configurable `semantic_weight`, `full_text_weight`, and `rrf_k`.

**Metadata.** Stored as JSONB alongside each vector entry. All document metadata (title, version, chunk_order, page_number, parser_generated) is carried through to the metadata column. Filtered at query time via the filter system in `py/core/providers/database/filters.py`.


Citations: [py/core/providers/embeddings/openai.py:21-60](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/embeddings/openai.py#L21-L60) · [py/core/providers/embeddings/litellm.py:25-65](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/embeddings/litellm.py#L25-L65) · [py/core/base/providers/embedding.py:21-44](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/base/providers/embedding.py#L21-L44) · [py/core/providers/database/chunks.py:75-182](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/database/chunks.py#L75-L182) · [py/core/providers/database/chunks.py:484-640](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/database/chunks.py#L484-L640) · [py/shared/abstractions/vector.py:27-107](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/shared/abstractions/vector.py#L27-L107)

### How is retrieval performed? (answered)

**Dense / sparse / hybrid search.** The `RetrievalService` (`py/core/main/services/retrieval_service.py:248-280`) dispatches to three strategies: basic (vanilla semantic + optional graph), HyDE (LLM-generated hypothetical documents), and RAG Fusion (multi-query with RRF). Internally, `_vector_search_logic` (`py/core/main/services/retrieval_service.py:651-729`) routes to `semantic_search`, `full_text_search`, or `hybrid_search` on `PostgresChunksHandler` depending on `SearchSettings` flags. Semantic search (`py/core/providers/database/chunks.py:327-482`) computes cosine distance via `vec <=> $1::vector(N)` with a two-stage binary re-ranking path for INT1 quantization. Full-text search uses `ts_rank(fts, websearch_to_tsquery('english', $1))`. Hybrid search (`py/core/providers/database/chunks.py:538-640`) fuses both via weighted RRF.

**Search strategies.** HyDE (`py/core/main/services/retrieval_service.py:554-614`) generates N hypothetical documents via LLM, embeds each, runs parallel searches, and re-ranks results against the original query. RAG Fusion (`py/core/main/services/retrieval_service.py:326-403`) generates `num_sub_queries` alternative phrasings via LLM, runs each independently, then fuses via RRF and optionally re-ranks.

**Reranking.** The embedding provider's `arerank()` method is called after initial search. `OpenAIEmbeddingProvider` passes through (`py/core/providers/embeddings/openai.py:227-243`). `LiteLLMEmbeddingProvider` optionally uses a HuggingFace TEI reranking endpoint (`py/core/providers/embeddings/litellm.py:47-59`).

**Graph search.** `_graph_search_logic` (`py/core/main/services/retrieval_service.py:732-794`) searches entities, relationships, and communities via embedding similarity on their description vectors. Graph entities have their own vector tables with pgvector.

**Filters.** Rich filter system (`py/core/providers/database/filters.py`) supporting `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin`, `$like`, `$ilike`, `$overlap`, `$contains`, `$and`, `$or`. Applied to the `metadata JSONB` column and top-level columns like `document_id`, `owner_id`, `collection_ids`.


Citations: [py/core/main/services/retrieval_service.py:248-280](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/services/retrieval_service.py#L248-L280) · [py/core/main/services/retrieval_service.py:326-403](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/services/retrieval_service.py#L326-L403) · [py/core/main/services/retrieval_service.py:554-614](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/services/retrieval_service.py#L554-L614) · [py/core/main/services/retrieval_service.py:651-729](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/services/retrieval_service.py#L651-L729) · [py/core/providers/database/chunks.py:327-482](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/database/chunks.py#L327-L482) · [py/core/providers/database/chunks.py:538-640](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/database/chunks.py#L538-L640) · [py/core/providers/database/filters.py:1-42](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/database/filters.py#L1-L42)

### How are answers generated and grounded? (answered)

**Prompt assembly.** System prompts are stored in the database via `PostgresPromptsHandler`. The `Agent` base class (`py/core/base/agent/agent.py:81-97`) loads a static prompt (e.g., `static_rag_agent`) and a dynamic reasoning prompt at startup. The `R2RAgent.arun()` (`py/core/agent/base.py:100-151`) assembles messages from the conversation history, passes them to `llm_provider.aget_completion()`, and iterates up to `max_iterations` (default 10). `GenerationConfig` (`py/shared/abstractions/llm.py`) controls model, temperature, max_tokens, streaming, and extended thinking.

**Citations / source attribution.** During streaming (`py/core/agent/base.py:351-638`), the `R2RStreamingAgent` tracks a `SearchResultsCollector` and `CitationTracker`. As the LLM streams tokens, `find_new_citation_spans()` detects bracket references like `[abc1234]` in the text. Each citation emits an SSE `citation_event` with the short ID, span boundaries, and full source payload on first occurrence. On `finish_reason=stop`, consolidated citations with all spans and payloads are emitted in a `final_answer_event` (`py/core/agent/base.py:599-633`). Citations are stored in `Message.metadata` for persistence.

**Streaming.** The `R2RStreamingAgent` (`py/core/agent/base.py:351-638`) yields SSE-formatted events: `thinking` (model chain-of-thought), `message` (partial text tokens), `tool_call` (tool name/arguments), `citation` (source reference), and `final_answer` (complete answer with structured citations). The endpoint returns `text/event-stream` content type via FastAPI's `StreamingResponse` (`py/core/main/api/v3/retrieval_router.py:408-427`).

**Agentic / multi-step.** `AgentFactory` (`py/core/main/services/retrieval_service.py:60-245`) creates one of 10 agent types based on mode (RAG or research), streaming preference, and XML tool format. `R2RRAGAgent` and `R2RResearchAgent` support tool usage via `ToolRegistry` — tools include `web_search`, `web_scrape`, `search_file_knowledge`, `get_file_content`, `reasoning`, `critique`, `python_executor`. The research agent adds its own reasoning/critique/Python tools (`py/core/agent/research.py:35-80`).


Citations: [py/core/base/agent/agent.py:81-98](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/base/agent/agent.py#L81-L98) · [py/core/agent/base.py:100-151](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/agent/base.py#L100-L151) · [py/core/agent/base.py:351-638](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/agent/base.py#L351-L638) · [py/core/main/services/retrieval_service.py:60-245](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/services/retrieval_service.py#L60-L245) · [py/core/main/api/v3/retrieval_router.py:408-427](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/api/v3/retrieval_router.py#L408-L427) · [py/core/agent/research.py:35-80](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/agent/research.py#L35-L80)

### How is quality evaluated or observed? (answered)

R2R has **no built-in evaluation framework** — no scoring, no LLM judge, no integrated metrics like faithfulness or answer relevancy. The project ships with two observability mechanisms instead:

- **Sentry integration** (`py/core/utils/sentry.py`). Initialized at app startup via `init_sentry()`, reads `R2R_SENTRY_DSN` from the environment. Supports configurable traces_sample_rate and profiles_sample_rate. This provides error tracking and performance tracing but no RAG-specific quality metrics.

- **Structured logging** (`py/core/utils/logging_config.py`). Configures python-json-logger with HTTP status filters for uvicorn.access logs. Health endpoint requests are filtered out. There is no RAG-specific logging (no retrieval latency, generation quality, or user feedback capture).

- **External evaluation guide.** The project provides a cookbook at `docs/cookbooks/evals.md` demonstrating how to evaluate R2R outputs using the Ragas framework externally. It shows collecting R2R's RAG responses via `client.retrieval.rag()` and feeding them into Ragas for metrics like faithfulness, answer_relevancy, context_precision, and context_recall. This is an optional, bring-your-own-eval approach — Ragas is not a dependency and no integration code exists in the source tree.

There are no built-in evaluation endpoints, no A/B testing infrastructure, no LLM-as-judge calls, and no quality monitoring dashboards. Users must integrate external frameworks (Ragas, TruLens, LangFuse, etc.) themselves.


Citations: [py/core/utils/logging_config.py:1-50](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/utils/logging_config.py#L1-L50) · [docs/cookbooks/evals.md:1-50](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/docs/cookbooks/evals.md#L1-L50)

### How is it deployed and operated? (answered)

**Library vs service.** R2R is both a Python library (`pip install r2r`) and a FastAPI web service. The library exposes `R2RClient` and `R2RAsyncClient` (`py/r2r/__init__.py:1-19`) for programmatic access. The service runs via uvicorn on the `core.main.app_entry:app` FastAPI application (`py/core/main/app_entry.py:109-139`).

**API.** Versioned REST API under `/v3/` with routers for search, RAG, agent conversations, ingestion, documents, collections, users, conversations, prompts, graphs, chunks, indices, and system health (`py/core/main/app.py:14-23`). The `/retrieval/search`, `/retrieval/rag`, and `/retrieval/agent` endpoints are the primary retrieval/RAG surface. Authentication is pluggable via `AuthProvider` (Supabase, JWT, Clerk). CORS is wide open (`allow_origins=["*"]`).

**Configuration.** Multiple toml config files (`py/core/main/config.py:29-43`) with environment variable overrides. Key env vars: `R2R_POSTGRES_HOST/PORT/USER/PASSWORD/DBNAME`, `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `MISTRAL_API_KEY`, `R2R_SENTRY_DSN`, `R2R_CONFIG_NAME`, `R2R_PROJECT_NAME`.

**Required infrastructure.** PostgreSQL 15+ with pgvector extension is mandatory. Optional: Unstructured API (or self-hosted Unstructured service) for advanced document parsing; Mistral API for OCR; Redis for APScheduler jobs; Sentry DSN for error tracking.

**Deployment.** Docker image (`py/Dockerfile`) runs uvicorn on port 8000 (configurable via `R2R_PORT`). The Docker build uses multi-stage with Rust compilation for tokenizer dependencies. A CMD of `uvicorn core.main.app_entry:app --host $R2R_HOST --port $R2R_PORT` launches the service.

**Scaling and multi-tenancy.** No horizontal scaling infrastructure — the `SimpleOrchestrationProvider` (`py/core/providers/orchestration/simple.py`) is a stub with no background worker. A `HatchetOrchestrationProvider` exists for orchestration via an external Hatchet server. Multi-tenancy is handled via **collection-based scoping**: users and documents belong to collections, and all queries filter by `collection_ids` (`py/core/providers/database/chunks.py:178-179` creates a GIN index on `collection_ids`). The database schema uses PostgreSQL schemas (one per project via `R2R_PROJECT_NAME`) for logical isolation.

> **Editor's note.** Correction: the default `simple` orchestration is not a stub; it runs ingestion and graph workflows inline inside the HTTP request (Hatchet is used only by the `full` configs). Redis is not used: the scheduler is in-process APScheduler.

Citations: [py/core/main/app_entry.py:109-139](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/app_entry.py#L109-L139) · [py/core/main/app.py:14-78](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/main/app.py#L14-L78) · [py/core/providers/database/chunks.py:165-182](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/core/providers/database/chunks.py#L165-L182) · [py/Dockerfile:1-47](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/Dockerfile#L1-L47) · [py/r2r/__init__.py:1-19](https://github.com/SciPhi-AI/R2R/blob/9c5a94d151f90876bd7eb860f300a8fd662dc481/py/r2r/__init__.py#L1-L19)