How are memories stored?

Storage is layered with three backends. SQLite is the primary local store using bun:sqlite. The memory_items table (src/storage/sqlite/schema.ts:85-105) stores each memory with columns for id, project_id, kind (observation/summary/prompt/manual), type, title, subtitle, text, narrative, facts (JSON), concepts (JSON), files_read, files_modified, and metadata. An FTS5 virtual table (memory_items_fts, src/storage/sqlite/schema.ts:171-182) indexes title/subtitle/text/narrative/facts/concepts with porter unicode61 tokenization. The MemoryItemsRepository (src/storage/sqlite/memory-items.ts:111) implements CRUD operations and FTS5-based search via the MATCH keyword (src/storage/sqlite/memory-items.ts:260-291). Postgres is the server store (src/storage/postgres/schema.ts:223-238). The observations table adds team_id for multi-tenancy, a generated content_search TSVECTOR column using to_tsvector('english', content). Observations use embeddings stored as JSONB, and the generation_key column has a UNIQUE partial index (src/storage/postgres/schema.ts:298-300) for idempotent write deduplication. An observation_sources table links each observation to its upstream events. Chroma is an optional vector database (src/services/sync/ChromaSync.ts) that stores observations as embeddings for semantic search. The embedding function is configurable via the CLAUDE_MEM_CHROMA_EMBEDDING_FUNCTION setting (src/services/sync/ChromaSync.ts:356-362). Chroma runs as a local MCP subprocess (ChromaMcpManager). The memory schema is defined in Zod at src/core/schemas/memory-item.ts:5-44 with full runtime validation — a MemoryItem carries fields for id, projectId, serverSessionId, kind, type, title, subtitle, text, narrative, facts[], concepts[], filesRead[], filesModified[], and metadata.

Memories are stored primarily in a pluggable vector store with auxiliary SQLite storage for change history and per-session messages.

Vector store — the core persistence layer. Each memory is stored as a vector embedding plus a payload dict. The payload fields (mem0/memory/main.py:1029-1041) include: data (the extracted memory text), hash (MD5 for dedup), text_lemmatized (for BM25 keyword search), created_at/updated_at (ISO timestamps), user_id/agent_id/run_id (scoping), actor_id (who spoke), role (user/assistant), attributed_to, and expiration_date (optional TTL). Any caller-supplied metadata keys survive as additional payload fields. The vector store interface (mem0/vector_stores/base.py) standardizes insert, search, delete, update, list, and optionally keyword_search for BM25. The VectorStoreFactory (mem0/utils/factory.py:180-218) supports 25+ backends: Qdrant, Pinecone, Chroma, Weaviate, Milvus, MongoDB, Redis, Elasticsearch, pgvector, Supabase, Faiss, S3Vectors, and more.

Embeddings — configured via EmbedderFactory (mem0/utils/factory.py:152-177) with 12 providers (OpenAI, Azure OpenAI, Gemini, HuggingFace, FastEmbed, Together, AWS Bedrock, Ollama, etc.). The embedding model is called for query encoding and for each memory text at write time. embed_batch is used for batch efficiency.

Entity store — a second vector store collection (named {collection}_entities or {collection}-entities) storing entities extracted from memories. Each entity record has: data (entity text), entity_type (PROPER, QUOTED, TOPIC, IDENTIFIER), and linked_memory_ids (list of memory UUIDs that mention this entity). This is lazily initialized on first entity operation.

SQLite — local database (mem0/memory/storage.py) with two tables: history tracking every ADD/UPDATE/DELETE on each memory (with old/new text, timestamps, actor_id, role), and messages storing recent conversation messages per session scope (trimmed to 10 most recent).

Memories are stored in PostgreSQL across several tables created by the initial migration (5a366d414dce_initial_schema.py). The central table is memory_units (line 266), which stores each fact as a row with columns: id (UUID PK), bank_id, text, embedding (Vector(384) by default), context, event_date, occurred_start, occurred_end, mentioned_at, fact_type (world/experience/observation), confidence_score, access_count, metadata (JSONB), tags (TEXT[]), plus timestamps. The embedding dimension is auto-detected from the model. Vector indexes support pgvector HNSW, pgvectorscale DiskANN, vchord, or alloydb_scann (initial_schema.py:363-368). For keyword search, a search_vector column is maintained — native PostgreSQL tsvector, vchord_bm25 vector, or TEXT for extension-backed BM25 (initial_schema.py:310-332). Entity storage lives in entities (canonical_name, bank_id, metadata JSONB, first_seen, last_seen, mention_count) with a unique index on (bank_id, LOWER(canonical_name)). unit_entities joins memory_units to entities. memory_links stores precomputed semantic (kNN-graph), causal, and entity co-occurrence links. chunks stores document chunks with their own embeddings. The documents table records source documents. The PostgresMemories class (memories/postgres.py) implements the pluggable MemoriesExtension interface (memories/base.py); alternative storage backends can be loaded via HINDSIGHT_API_MEMORIES_EXTENSION env var. Embeddings are generated by a pluggable Embeddings class (embeddings.py) supporting local (SentenceTransformers, ONNX) and remote providers (OpenAI, Cohere, Google, TEI, LiteLLM).

The system uses a property-graph model stored in one of four backends: Neo4j (primary), FalkorDB, Kuzu, or Neptune. Node and edge base classes live in nodes.py and edges.py. EntityNode (nodes.py:499) stores: uuid, name (with name_embedding for semantic search), summary (accumulated entity description), group_id (partition key), labels (e.g. 'Person', 'Entity'), and attributes (arbitrary key-values from custom Pydantic models). EntityEdge (edges.py:263) stores: source_node_uuid, target_node_uuid, name (relation type), fact (textual fact with fact_embedding), episodes (provenance list), temporal fields valid_at/invalid_at/expired_at, and typed attributes.

Episodic input is stored as EpisodicNode (nodes.py:318) with content, source, valid_at, entity_edges, and optional episode_metadata for custom filtering. SagaNode groups ordered episodes into longer conversation threads. A MENTIONS edge (EpisodicEdge) links episodes to entities; a NEXT_EPISODE edge chains sequential episodes within a saga; a HAS_EPISODE edge attaches episodes to a saga.

Embeddings are generated by pluggable EmbedderClient implementations: OpenAI, Voyage, Gemini, Azure OpenAI (embedder/ directory). EntityNode.name_embedding and EntityEdge.fact_embedding are stored directly as properties on the nodes/edges in the graph database. The default dimension is read from EMBEDDING_DIM in embedder/client.py. Community nodes aggregate entity clusters with their own name_embedding for search.

The GraphDriver ABC (driver/driver.py) defines the storage abstraction with execute_query(), session(), transaction support, and specialized operation interfaces (entity_node_ops, search_ops, etc.). Concrete drivers (Neo4jDriver, FalkorDriver, KuzuDriver, NeptuneDriver) map Cypher/SQL-like queries to their respective backends. Fulltext indexes are created on entity names, edge facts, episode content, and community names. Customization is possible in CLAUDE.md: set index names via ENTITY_INDEX_NAME, EDGE_INDEX_NAME, etc.

Memories are stored in three parallel pluggable backends. Graph DB through GraphDBInterface (graph_db_interface.py:36): nodes are DataPoints with deterministic UUID5 ids from identity_fields; edges are (source_id, target_id, relationship_name) triples, all upsert-keyed on id. Default is Ladybug (embedded Kuzu) with Neo4j, Neptune, Turso, and Postgres alternatives. Vector DB through VectorDBInterface (vector_db_interface.py:11): one collection per DataPoint type and embedded field pair (e.g. DocumentChunk_text), with idempotent upserts and embedding via the configured engine. Default is LanceDB with PGVector, Neptune Analytics, and Turso alternatives. Relational DB (SQLite default, PostgreSQL) holds users, datasets, permissions, ACLs, and pipeline run status. The DataPoint base (DataPoint.py:28) adds version, created_at, updated_at, bi-temporal valid_to, feedback_weight, importance_weight, ontology_uri, and provenance stamps. The session cache (CACHE_BACKEND=sqlite/postgres/redis) stores QA turns, agent traces, and distilled context as a fast layer.

Storage is primarily API-driven through the supermemory SDK. The SupermemoryClient (apps/mcp/src/server/client/index.ts:138-160) wraps the SDK and exposes methods that call REST endpoints. The memory schema is defined in apps/mcp/src/shared/types.ts:139-152: each MemoryEntry has id, memory (string), version, isLatest, isForgotten, isStatic, isInference, createdAt, updatedAt, sourceCount, documentIds, plus a history array of prior versions (line 125-137). Container tags (apps/mcp/src/shared/types.ts:39-53) scope memories into named spaces with documentCount and memoryCount. The SDK search (apps/mcp/src/server/client/index.ts:252-256) uses searchMode: 'hybrid', indicating a combination of vector (embeddings) and keyword retrieval. Memory entries have parent/root tracking via parentMemoryId and rootMemoryId (line 85-86), enabling version chains. The upstream document schema (documentsApiResponseSchema, line 113-118) includes type, title, summary, and content fields. No pluggable store interface exists in this repo — the MCP server delegates fully to the hosted API.

Memori stores memories in relational databases (PostgreSQL, MySQL, SQLite, TiDB, OceanBase, Oracle, CockroachDB) via SQLAlchemy/dbapi adapters, or in MongoDB via a document adapter (memori/storage/_registry.py:18-67). The schema is defined through migration files — for PostgreSQL the full DDL is in memori/storage/migrations/_postgresql.py. The core tables are:

• memori_entity — one row per entity (user/person/thing), keyed by external_id (your application's user ID).
• memori_process — one row per agent process, similarly keyed.
• memori_session — ties an entity + process to a session UUID.
• memori_conversation — a session-scoped conversation with an optional summary TEXT populated by augmentation.
• memori_conversation_message — individual message rows with role, type and content.
• memori_entity_fact — the primary memory table: stores content (the fact as text), content_embedding (BYTEA for PostgreSQL/MySQL, BLOB for others), a numeric num_times counter, and a uniq hash for deduplication. A UNIQUE constraint on (entity_id, uniq) prevents duplicates; ON CONFLICT increments the counter.
• memori_knowledge_graph — stores semantic triples via FK references to memori_subject, memori_predicate, and memori_object tables, plus its own num_times frequency counter.
• memori_process_attribute — key-value attributes for the process, also with dedup via uniq.
• memori_entity_fact_mention — links facts back to the conversation where they were mentioned.

Embeddings default to all-MiniLM-L6-v2 (memori/_config.py:62-63), computed either via a native Rust core (memori_python.NativeEmbedder, memori/native/_embeddings.py:24-32) or via a TEI (Text Embeddings Inference) server (memori/embeddings/_tei_embed.py). The store is pluggable: Registry.register_driver() maps dialect strings to driver classes (e.g., postgresql and cockroachdb share the same Postgres driver, memori/storage/drivers/postgresql/_driver.py:792-793), and Registry.register_adapter() maps connection objects to the right adapter (SQLAlchemy, MongoDB, dbapi, Django). Builders auto-detect the dialect and run migrations on startup (memori/storage/_builder.py, memori/storage/_manager.py:28-35).

Pluggable backends across 3 tiers. Vector DB: Qdrant/Milvus with collection_name, vector_dimension, cosine/euclidean/dot. VecDBItem payload holds TextualMemoryItem + embedding (general.py:81-104). Graph DB: MemoryManager (organize/manager.py:54-88) uses Neo4j/PolarDB/PG+pgvector (graph_dbs/factory.py:14-19) as labeled nodes with typed edges. EmbedderFactory (embedders/factory.py:16-36): Ollama, Sentence Transformers, Ark, OpenAI; CachingEmbedder adds LRU+TTL. TextualMemoryItem schema (item.py:94-212): id, memory text, TreeNodeTextualMemoryMetadata with memory_type (9 categories), embedding, sources, tags, status, user_id, session_id, version, history, confidence. ActivationMemory = KV-cache; ParametricMemory = LoRA. Env-driven config with local-first defaults.

Memories (called observations internally, conclusions in the API) are stored as Document rows in Postgres (src/models.py:390-521), each with fields: content (the fact text), level (explicit / deductive / inductive / contradiction), embedding (pgvector column at configured dimensionality), times_derived (integer reinforcement counter), session_name, observer, observed, workspace_name, and deleted_at for soft-delete. Documents belong to a Collection (src/models.py:342-382) uniquely keyed by (observer, observed, workspace_name) — so every observer+observed pair gets its own collection, enabling self-representation (observer == observed) and cross-peer modeling. The Documents table has an HNSW index on the embedding column for cosine-distance ANN search (src/models.py:511-519). An orthogonal MessageEmbedding table (src/models.py:284-335) stores chunk-level embeddings of raw messages with sync_state tracking, decoupled from message creation. Embeddings are generated via embedding_client (src/embedding_client.py) using configurable providers (OpenAI by default for embeddings). The pgvector path embeds + queries directly in Postgres. For external vector stores, the VectorStore ABC (src/vector_store/__init__.py:53-195) defines an interface with upsert_many, query, delete_many, and delete_namespace. Implementations include Turbopuffer (src/vector_store/turbopuffer.py), LanceDB (src/vector_store/lancedb.py), Qdrant (src/vector_store/qdrant.py), and ChromaDB (src/vector_store/chroma.py). Namespaces isolate collections per (type, workspace, observer, observed) with a hashed suffix. Both stores are dual-written to in migration mode (settings.VECTOR_STORE.MIGRATED); a sync_state column (pending/synced/failed) tracks consistency and a reconciliation backstop heals missed writes.

MemMachine uses a layered, pluggable storage architecture. Episodes (conversation turns) are persisted in EpisodeStorage — a SQLAlchemy-backed relational store (episode_sqlalchemy_store.py) using PostgreSQL or SQLite. Long-term episodic memory has two backend options: a declarative backend using VectorGraphStore, which wraps Neo4j (neo4j_vector_graph_store.py) or NebulaGraph Enterprise (nebula_graph_vector_graph_store.py), storing episodes as graph nodes with vector embeddings and DERIVED_FROM edges; or an event backend using a VectorStore (Qdrant, Milvus, SQLite+sqlite-vec, or SQLite+usearch/hnswlib) for vector embeddings plus a SegmentStore for time-ordered segment data. Short-term memory is an in-memory deque with capacity-based eviction and persistence hooks to SessionDataManager (short_term_memory.py:123). Profile/semantic memory stores extracted facts — each a SemanticFeature with fields (set_id, category, tag, feature_name, value, embedding, citations) — in SemanticStorage, which has implementations for PostgreSQL+pgvector (sqlalchemy_pgvector_semantic.py) and for vector stores with a separate citation table (vector_store_semantic_storage.py). Embeddings are generated by the Embedder abstraction, with concrete classes for OpenAI (openai_embedder.py), Sentence Transformers (sentence_transformer_embedder.py), and Amazon Bedrock (amazon_bedrock_embedder.py). The DatabasesConf class (database_conf.py:587) registers eight storage backends — Neo4j, PostgreSQL, SQLite, NebulaGraph, Qdrant, Milvus, SQLiteVectorStore, and SQLiteVecVectorStore — all configurable from a single YAML configuration file. Re-ranking is handled by the Reranker abstraction with BM25, cross-encoder, Cohere, and identity implementations.