plastic-labs/honcho

Overview

Honcho is Plastic Labs’ memory server for agents. You run it as a FastAPI service backed by Postgres with pgvector. Your application writes conversation messages to it, and you later ask it questions about the participants. It is not an in-process library. The Python and TypeScript SDKs in sdks/ are HTTP clients for the /v3 REST API.

The data model is the main idea. Everything in a workspace is a peer, and humans and agents are treated the same way. Peers exchange messages in sessions. Memory is stored per (observer, observed) pair, not per user. Each pair owns a Collection of Document rows. The code calls these rows observations, and the API calls them conclusions. By default every peer observes only itself. A peer with observe_others enabled in a session also builds its own model of the other peers in that session. So “what Alice believes about Bob” is stored separately from Honcho’s own view of Bob.

The work happens in the background. A separate deriver process reads a Postgres-backed queue, runs one structured LLM call per batch of messages to extract atomic facts, and deduplicates them on write. A dreamer later runs tool-using agents that write deductive and inductive conclusions on top of those facts. At read time, the dialectic agent answers a natural-language question with a tool loop over conclusions and raw messages. Honcho is aimed at teams that want a long-lived user model shared by several agents, and that can run Postgres and two Python processes for it.

Architecture

flowchart LR
  APP["App / SDK / MCP"] --> API["FastAPI /v3 routers"]
  API --> MSG["crud.create_messages"]
  API --> CHAT["peers.chat route"]
  MSG --> PG[("Postgres + pgvector")]
  MSG --> ENQ["deriver.enqueue"]
  ENQ --> Q[("queue table")]
  Q --> QM["QueueManager (deriver process)"]
  QM --> DER["minimal deriver: one LLM call"]
  QM --> SUM["summarizer"]
  QM --> DRM["dreamer: deduction + induction agents"]
  DER --> RM["RepresentationManager"]
  DRM --> RM
  RM --> DOCS["documents per observer/observed"]
  DOCS --> PG
  DOCS -.-> VS["optional external vector store"]
  CHAT --> DIA["DialecticAgent tool loop"]
  DIA --> DOCS
  DIA --> SRCH["hybrid message search (RRF)"]
  SRCH --> PG

Component	Path	Role
API app	src/main.py, src/routers/	FastAPI app; workspaces, peers, sessions, scopes, messages, conclusions, keys and webhooks under /v3
Data model	src/models.py	Workspace, Peer, Session, Message, Collection, Document, DocumentSource, QueueItem tables
Enqueue	src/deriver/enqueue.py	Turns new messages into representation, summary, dream and deletion queue rows
Queue worker	src/deriver/queue_manager.py, consumer.py	Claims work units from Postgres, batches messages by token budget, dispatches by task type
Deriver	src/deriver/deriver.py, prompts.py	One structured-output LLM call per batch that extracts explicit facts
Representation store	src/crud/representation.py, src/crud/document.py	Embeds, deduplicates and saves conclusions; builds the “working representation” for reads
Dreamer	src/dreamer/	Scheduled consolidation: deduction and induction specialists, optional surprisal sampling
Dialectic	src/dialectic/core.py, chat.py	Tool-using agent behind POST /peers/{id}/chat
Agent tools	src/utils/agent_tools.py	Tool schemas and handlers shared by the dialectic and dreamer agents
Search	src/utils/search.py	Semantic plus full-text message search fused with Reciprocal Rank Fusion
Vector stores	src/vector_store/	pgvector by default; Turbopuffer, LanceDB, Qdrant, ChromaDB adapters
LLM layer	src/llm/	Provider-neutral call wrapper with OpenAI, Anthropic and Gemini backends and a tool loop
Clients	sdks/python, sdks/typescript, mcp/, honcho-cli/	SDKs, MCP server, CLI that can start a local Docker stack

How a request flows

Two paths matter: writing a message, and asking a question.

Write path

1. POST /v3/workspaces/{w}/sessions/{s}/messages calls crud.create_messages. It then schedules enqueue(payloads) as a FastAPI background task, plus an immediate message-embedding task when EMBED_MESSAGES is on (messages.py).
2. enqueue first cancels any pending dreams for the sender, because the peer is active again. handle_session then resolves the configuration hierarchy (message, then session, then workspace) and inserts QueueItem rows (enqueue.py).
3. generate_queue_records adds a summary task every N messages. It also builds one representation task whose observers list holds the sender (if observe_me is on) and every other active peer with observe_others (enqueue.py).
4. In the deriver process, QueueManager.get_and_claim_work_units claims a representation work unit only when its unprocessed messages total at least REPRESENTATION_BATCH_WORK_UNIT_TARGET_TOKENS (default 512) or the oldest item is older than REPRESENTATION_BATCH_MAX_AGE_SECONDS (default 1800), unless FLUSH_ENABLED is set (queue_manager.py, config.py).
5. process_work_unit drains the unit in token-capped batches and calls process_representation_batch (queue_manager.py).
6. process_representation_tasks_batch wraps each message in a <message ... target="true|false"> tag and builds minimal_deriver_prompt. It makes a single honcho_llm_call with response_model=PromptRepresentation (deriver.py, prompts.py).
7. The same observations are saved into every observer’s collection through RepresentationManager.save_representation (deriver.py). That method batch-embeds the texts, calls crud.create_documents with deduplication, and then checks whether a dream is due (representation.py).

Read path

1. POST /v3/workspaces/{w}/peers/{p}/chat validates scope and session options, checks that a peer-scoped key may read the named sessions, and calls agentic_chat (or agentic_chat_stream for SSE). The observer is the path peer, and the observed peer is target or the observer itself (peers.py).
2. agentic_chat loads the configuration and both peer cards, closes its DB session, and builds a DialecticAgent (chat.py).
3. The agent prefetches relevant observations: separate searches for explicit and higher-level conclusions, 25 of each, or 10 at minimal. With a session id, it also injects recent session history into the system prompt (core.py).
4. answer runs honcho_llm_call with the level’s tool set, tool choice and iteration cap, then returns plain text, or JSON when the caller passed a response_format (core.py).

Key components

Deriver (extraction)

The deriver is intentionally small. It makes one LLM call per batch and asks only for explicit facts. The prompt tells the model to extract facts only from target="true" messages and to use other peers’ messages only as context. It also asks for absolute dates and resolved pronouns, and it escapes <message inside content so a message cannot forge its own tag (prompts.py). Workspace or session custom_instructions are inserted into this prompt. Peer cards, deductions and summaries are handled elsewhere.

Deduplication and reinforcement

create_documents always runs exact-match deduplication on normalized content, level and session. A duplicate inside the batch is dropped. A duplicate of an existing row increments that row’s times_derived instead of inserting a new one (document.py). With DERIVER.DEDUPLICATE (on by default), it also looks up near neighbours. _semantic_dup_decision then keeps whichever text carries more unique tokens, so it either replaces the old row or rejects the new one (document.py). times_derived is later used as a retrieval signal.

Storage

A Collection is unique on (observer, observed, workspace_name). Composite foreign keys tie both names to real peers (models.py). A Document carries level (explicit, deductive, inductive, contradiction), times_derived, a pgvector embedding, a soft-delete deleted_at, and a sync_state for external vector stores (models.py). Embeddings use an HNSW cosine index. Reasoning-tree edges live in document_sources, which deliberately has no foreign key on source_id (models.py). The defaults are OpenAI text-embedding-3-small at 1536 dimensions.

Working representation

_get_working_representation_internal fills a budget of up to 100 observations. It takes a semantic slice (default a third of the budget), then a most-reinforced slice ordered by times_derived, and then fills the remainder with the most recent documents. An empty session allowlist returns nothing instead of everything (representation.py).

Dialectic agent

Reasoning levels map to tool sets and iteration caps. minimal gets only search_memory and search_messages. The other levels add get_observation_context, grep_messages, two date-based message tools and get_reasoning_chain (agent_tools.py). Under a session allowlist, get_reasoning_chain is removed, because reasoning chains cross session boundaries (core.py). The default iteration caps are 1, 5, 2, 4 and 10 for minimal, low, medium, high and max. Every level defaults to gpt-5.4-mini over the OpenAI transport (config.py). So by default, “higher” levels differ mostly in tool rounds and tool choice, not in model.

Hybrid message search

search combines pgvector (or external-store) similarity with Postgres full-text search and merges the two ranked lists with reciprocal_rank_fusion (k=60) (search.py). A peer_perspective filter limits results to sessions the peer was in, and to the time window of its membership (search.py).

Dreamer

check_and_schedule_dream counts only explicit documents, so dream output cannot trigger more dreams. It schedules a dream after 50 new explicit facts, at least 8 hours after the last dream, delayed by an idle timeout (dream_scheduler.py). run_dream runs the deduction specialist, then the induction specialist. Each is a tool loop with a hardcoded cap, 12 rounds for deduction and 10 for induction (specialists.py, L740-L745). Both can search memory and messages and create conclusions. Only deduction can delete conclusions and rewrite the peer card (agent_tools.py). Surprisal sampling over k-d trees and other tree types can seed both with “hints”, but SURPRISAL.ENABLED defaults to false (orchestrator.py, config.py).

Scoping and auth

JWT claims form a hierarchy: admin, workspace w, peer p, session s (security.py). Scopes group sessions into a recall boundary. Under a session allowlist, only explicit conclusions are served. The reason is that dreamer output reads across all sessions but is stamped with a single session, and the code says so (representation.py).

Extending it

• Configuration hierarchy. Reasoning, summary, dream and peer-card settings resolve from message to session to workspace. custom_instructions steers the deriver and the dreamer without code changes.
• Models. Each stage (deriver, each dialectic level, summary, deduction, induction, embeddings) has its own MODEL_CONFIG with a transport of openai, anthropic or gemini. These can be set in config.toml or through DERIVER_, DIALECTIC_LEVELS__<level>__ and similar environment variables. An overrides block sets a custom base_url and key, so OpenAI-compatible local endpoints work through the openai transport.
• Vector store. Set VECTOR_STORE_TYPE to turbopuffer, lancedb, qdrant or chromadb, and implement VectorStore (upsert_many, query, delete_many, delete_namespace) for anything else (vector_store/init.py). Namespaces are hashed per (workspace, observer, observed).
• Structured answers. Pass response_format (JSON Schema) to chat. It is converted to a Pydantic model, and the answer comes back as JSON.
• Webhooks and clients. Webhook endpoints receive queue-driven deliveries. The mcp/ server runs as a Cloudflare Worker, over stdio or over Streamable HTTP. session.context() in both SDKs renders history plus summary for to_openai / to_anthropic. Coding-agent plugins (Claude Code, Codex, Cursor, OpenCode, OpenClaw, Hermes) are separate packages built on the same API.

Running it

• Required services. Postgres with the pgvector extension, plus an API key for the configured provider (OpenAI by default, for both chat models and embeddings). Redis is optional and serves as a cache.
• Two processes. The API (fastapi on src/main.py) and the deriver (python -m src.deriver). Both share the database, which also serves as the job queue. docker-compose.yml.example wires api, deriver, database and redis, and binds ports to localhost.
• CLI. honcho start in honcho-cli brings up the same four services from a published image and can run a config wizard (--setup basic|advanced).
• Auth. AUTH_USE_AUTH defaults to false. Set it and AUTH_JWT_SECRET before exposing the API (config.py).

Strengths and caveats

• Strength: a real multi-party model. Observer/observed collections and per-session observe_others allow different agents to hold different views of the same person. Few memory systems store this.
• Strength: cheap, careful ingestion. One structured call per token-budgeted batch, target-only extraction, injection-safe tagging, and exact plus semantic deduplication with a reinforcement counter.
• Strength: honest scoping. Empty allowlists fail closed, and synthesized conclusions are excluded from session-scoped reads. The code says openly that per-conclusion provenance is not there yet.
• Strength: operational depth. A Postgres queue with work-unit ownership, retries, a reconciler for vector sync and soft-delete cleanup, plus Prometheus metrics and Sentry.
• Caveat: memory is not immediate. Facts appear only after a work unit reaches 512 tokens or 30 minutes pass, unless you enable FLUSH_ENABLED. Dreams need 50 facts and an 8-hour gap.
• Caveat: every read costs an LLM call. Chat is an agent loop, so latency and cost scale with the reasoning level. The default levels share one model, so “high” means more tool rounds, not a stronger model.
• Caveat: heavy footprint. You run Postgres, pgvector and two Python services, and the codebase is large (for example, a 2,800-line agent_tools.py). This is infrastructure, not a drop-in library.
• Caveat: insecure by default, and AGPL. Auth is off until you configure a JWT secret. The AGPL-3.0 license matters if you modify Honcho and offer it as a network service.

Sources: code at 11b22bf, verified Q&A.