LLMs Technical Reviews
Home / Open-source personal assistants / rakazo

elie222/rakazo

Self-hosted persistent AI bots on a Pi agent runtime, with Postgres job queue, pluggable sandboxes and rule-based tool approval.

GitHub ↗★ 3.4kTypeScriptApache-2.0commit 4cdf3e2 · 2026-10-06homepage ↗

Overview

Rakazo is a self-hosted platform for persistent AI “bots”: each bot has its own threads, memory, routines and a computer (browser, terminal, files, optional graphical desktop). People reach bots from a React web app, an Electron desktop app, an Expo mobile app, or chat platforms (Slack, Telegram, WhatsApp, Sendblue). It is a TypeScript monorepo: apps/ holds deployable surfaces and packages/ holds domain logic and adapters.

The design is adapter-first. packages/adapter-kit defines interfaces for the agent runtime, sandbox, connectors, memory, jobs, realtime and more, and packages/adapters implements them. The default agent runtime is Pi (@earendil-works/pi-agent-core and pi-ai), so model choice is broad: OpenRouter by default, plus Anthropic, OpenAI, Google, ChatGPT/Claude subscription OAuth, OpenAI-compatible and local endpoints.

Work is asynchronous. The API only writes a queued run and enqueues a job. A separate Graphile Worker process leases the run in Postgres and drives the model loop, so runs survive restarts and can pause for approval or a human takeover.

Architecture

flowchart LR
  WEB["web / desktop / mobile"] --> API["api (Hono + oRPC)"]
  CHAT["Slack / Telegram / WhatsApp"] --> API
  API --> PG["Postgres (Prisma)"]
  API --> Q["Graphile jobs"]
  Q --> WK["worker"]
  WK --> EX["Run executor"]
  EX --> PI["PiAgentRuntime"]
  PI --> LLM["Model providers"]
  EX --> GATE["Approval gate"]
  EX --> CON["Connectors: Composio, Pipedream, MCP, OpenAPI"]
  EX --> MEM["Memory: Markdown + semantic"]
  EX --> SBX["Sandbox provider"]
  SBX --> SUP["supervisor"]
  SUP --> COMP["computer container"]
  EX --> PG
  PG --> RTF["Realtime fanout"]
  RTF --> WEB
Component Path Role
API server apps/api/src/app.ts, apps/api/src/router.ts Better Auth sessions, oRPC procedures, webhooks, voice routes
Worker apps/worker/src/index.ts Wires adapters and runs Graphile Worker job handlers
Run executor packages/adapters/src/executor.ts Leases a run, builds context and tools, gates and records each tool call
Pi runtime packages/adapters/src/pi-runtime.ts Streams the model and executes tool calls
Approval rules packages/core/src/action-approval.ts Which tools need approval; rule resolution; auto-review plan
Adapter interfaces packages/adapter-kit/src/interfaces.ts Contracts for runtime, sandbox, connectors, memory, jobs
Memory packages/memory/src/index.ts Markdown documents with revisions in Postgres
Sandboxes packages/adapters/src/*-sandbox.ts, infra/sandboxes/ Docker (via supervisor), E2B, Daytona, CreateOS, Box, desktop
Clients apps/web, apps/desktop, apps/mobile React + Vite, Electron, Expo
Deploy infra/compose/ Compose files, installer, Caddy and hardening scripts

How a request flows

  1. The client calls the threads.send oRPC procedure. The /rpc/* middleware resolves the Better Auth session and the space membership first (app.ts).
  2. threads.send refuses if no model is connected, then calls sendThreadMessage (router.ts). A run row is created with status queued and runContinueJob(run.id) is enqueued (router.ts).
  3. The worker’s run.continue handler calls executor.continueRun (background-job-handlers.ts).
  4. continueRun takes a fenced lease on the run with a conditional updateMany, so only one worker owns it (executor.ts). It then resolves the model, loads memory and scratchpad context, recalls semantic memories and discovers connector tools.
  5. It calls deps.runtime.run(...) with the prompt and instructions; memory and scratchpad context pass through redactSecrets first (executor.ts).
  6. For each tool call, the executor checks toolRequiresApproval, user rules and webhook-trigger rules, then planActionGate returns ask, allow or judge (executor.ts). An ask writes an approval block and the run waits for the person.
  7. Effects are recorded with idempotency keys (recordEffect), message blocks are persisted, and Postgres realtime fanout pushes updates to clients. If chat platforms are configured, bot replies are mirrored to them.

Key components

Worker wiring

main() in the worker is the clearest map of the system. It picks PiAgentRuntime or ScriptedAgentRuntime from AGENT_RUNTIME, builds the sandbox from SANDBOX_PROVIDER, the MCP connector with stdio disabled unless MCP_STDIO_ENABLED=true, Composio and Pipedream when keys exist, and a connector stack of installed connectors, managed catalogues and MCP (worker/index.ts).

Approval gate

Tools fall into sets: approval-exempt (computer, browser, file, shell, remember, scheduling), always-approve builtins (delete_bot, forget_memory, cloud_agent_launch and others), and create_space, which always asks (action-approval.ts). Connector tools are judged by name: a declared write or a mutating or compound name needs approval, and anything not clearly read-only does too (action-approval.ts). Auto Review can only escalate to ask, and judge errors fail closed for consequential tools (action-approval.ts).

Memory

MarkdownMemoryStore stores documents and revisions per space, user, scope and bot, with plain search (memory/index.ts). Optional semantic providers (Serenity, Supermemory) implement SemanticMemoryProvider with recall, save and purgeHistory (interfaces.ts). Old thread history is compacted into an LLM summary. Semantic recall runs only after a thread has been compacted, and returns at most five memories.

Models

resolveDeploymentModel sets the deployment default: PI_DEFAULT_PROVIDER (default openrouter) with OPENROUTER_API_KEY or ANTHROPIC_API_KEY, and PI_DEFAULT_MODEL (deployment-model.ts). Users add their own credentials (API key or OAuth) in the UI, and bots can pin a model.

Sandboxes

resolveSandboxProvider defaults to docker and falls back to none when a remote provider has no API key, or when production Docker has no supervisor token (sandbox-provider-env.ts).

Extending it

  • New adapter. Implement an interface from adapter-kit, such as AgentRuntime (interfaces.ts) or ConnectorProvider (discoverTools), and wire it in the worker and API.
  • Tools without code. Users add a remote MCP server, a Treg endpoint or an OpenAPI JSON document from the Integrations screen; operators enable Composio or Pipedream with env keys.
  • Skills. Bots load skills through skill-tools.ts; builtin-skills.ts ships defaults.
  • Approval rules. Per-tool, per-connector or per-category (email, purchase) rules decide always-allow versus require-approval.

Running it

The fastest path is infra/compose/install-images.sh: it downloads the Compose files, writes .env with random secrets, pulls the edge images and starts postgres (16), supervisor, api, worker and web on port 5173. No Redis is used; jobs and realtime both run on Postgres. For development you need Node 22.22.2+/24/26+, pnpm 9 and Docker, then pnpm install, pnpm db:generate, pnpm db:migrate, pnpm sandbox:build and pnpm dev. Set SANDBOX_PROVIDER to e2b, daytona, createos or box to use a hosted computer. Production overlays, a Caddyfile, harden-host.sh and restrict-computer-egress.sh live in infra/compose/.

Strengths and caveats

  • Strength: durable runs. Fenced leases, a job reconciler and idempotent effect records mean a crashed worker does not double-send.
  • Strength: model and sandbox choice. Pi gives a wide provider catalogue including local and subscription OAuth; six sandbox backends share one interface.
  • Strength: few hard vendor dependencies. Only Postgres and a model key are required.
  • Caveat: computer actions are not gated in normal runs. shell, browser_act, computer_act and write_file are approval-exempt (webhook-triggered runs are stricter). A shell guard blocks commands that would break a bot’s desktop, but otherwise safety on the computer relies on the sandbox, not per-action approval.
  • Caveat: name-based connector gating. Read-only detection uses tool-name prefixes; an oddly named write tool is caught only because unknown names default to approval.
  • Caveat: very large core file. executor.ts is over 8,000 lines and router.ts over 6,000, which makes changes to the run loop costly to review.

Sources: code at 4cdf3e2, deepwiki-open wiki (10 pages), OpenDeepWiki wiki (17 pages), verified Q&A.

How it answers the Open-source personal assistants questions

Each answer was drafted by a code-reading agent at commit 4cdf3e2. Its citations were checked mechanically. Compare with the other open-source personal assistants →

How is the assistant architected?

answered

Architecture overview. Rakazo is structured as a monorepo (packages/ for domain logic, apps/ for deployable surfaces) built in TypeScript. The backend is two processes: a Hono/oRPC API server (apps/api/src/app.ts) that handles HTTP requests, authentication (Better Auth), WebSocket realtime fanout, and RPC endpoints, and a Graphile Worker (apps/worker/src/index.ts) that runs the actual agent loops as background jobs. The API enqueues run jobs; the worker picks them up and calls the executor.

Agent loop and runtime. The core execution lives in packages/adapters/src/executor.ts in createRunExecutor(). The exported continueRun() method (line 3166) leases the run from Postgres, sets up a computer execution lease, resolves the model credential, builds the system prompt, assembles conversation history (including compacted summaries and semantic recall), and calls deps.runtime.run() (line 6134). The runtime is either the Pi agent runtime (PiAgentRuntime, which uses @earendil-works/pi-agent-core for tool-calling LLM loops) or a ScriptedAgentRuntime for deterministic replay. The Pi runtime (packages/adapters/src/pi-runtime.ts) orchestrates the model stream, tool execution, and approval pauses.

Frontend/backend split. The three frontends (web React/Electron via apps/web/, and Expo mobile via apps/mobile/) are pure clients of the API. The web app uses shadcn/ui components from packages/ui-web and semantic tokens from @rakazo/ui-tokens. Desktop Electron hosts the web UI with added native setup/sandbox management. Mobile is native-first with Expo Router and StyleSheet, diverging only where native patterns are stronger.

User request flow. A user sends a message via the web/mobile composer → API RPC handler (apps/api/src/router.ts enqueues a run via runContinueJob) → Graphile Worker picks up the job → continueRun() in executor.ts leases the run, resolves the bot's model and tools, and calls PiAgentRuntime.run() → the LLM streams tokens and tool calls → results are persisted as MessageBlock rows and fanned out via Postgres realtime to the frontend.

apps/worker/src/index.ts:60-90apps/api/src/app.ts:1-100packages/adapters/src/executor.ts:2897-2970packages/adapters/src/executor.ts:3166-3245packages/adapters/src/executor.ts:6134-6195

How are integrations (email, calendar, chat, docs) implemented?

answered

Integration services and API clients. Rakazo supports four integration tool sources, all accessed through a unified ConnectorProvider interface (packages/adapter-kit/src/interfaces.ts). The connector stack is built in apps/worker/src/index.ts line 145: InstalledConnectorProvider (user-installed connectors), Composio, Pipedream Connect, and MCP servers. Each resolves to a ConnectorTool with a name, description, and inputSchema.

Composio (packages/adapters/src/composio-connector.ts): wraps the Composio SDK to provide 200+ SaaS integrations (Google Calendar, Gmail, Slack, GitHub, etc.). It maps Composio tool definitions to the connector interface. Enabled via COMPOSIO_API_KEY env var. Tools are fetched from Composio's catalog and cached.

Pipedream Connect (packages/adapters/src): triggered via PIPEDREAM_CLIENT_ID/CLIENT_SECRET/PROJECT_ID env vars. Runs user-configured Pipedream workflows as tool invocations. Connector credentials are encrypted server-side using EncryptedSecretStore (packages/adapters/src/secrets.ts).

MCP (packages/core/src/mcp.ts and packages/adapters/src): supports both stdio-based and remote HTTPS MCP servers. Users add MCP servers through the Integrations settings UI. Stdio MCP is opt-in via MCP_STDIO_ENABLED=true with an allowlist of commands (MCP_STDIO_ALLOWED_COMMANDS). Remote MCP can target private endpoints when MCP_ALLOW_PRIVATE_ENDPOINT=true. OAuth for MCP goes through McpOAuthBroker.

OpenAPI integration: users can point at any OpenAPI JSON document to expose those endpoints as tools.

OAuth flow. Model providers use OAuth tokens stored in ModelCredential rows via encrypted ciphertext fields. packages/adapters/src/pi-oauth.ts handles OAuth device-code flows for ChatGPT Plus/Pro (openai-codex), GitHub Copilot, and xAI SuperGrok, plus auth-url flow for Claude Pro/Max (anthropic). Refresh tokens are cycled automatically; failed refreshes retire the credential.

Sync vs on-demand. Connector tools are discovered at run start via connector.discoverTools(). Composio connections are synced lazily on first use; Pipedream and MCP tools are fetched per run. There is no background sync — tools are resolved when a run begins.

apps/worker/src/index.ts:134-155packages/adapters/src/composio-connector.ts:1-80packages/adapters/src/pi-oauth.ts:1-60packages/core/src/mcp.ts:1-10

How is memory and user context stored and retrieved?

answered

Storage layers. Rakazo has three memory systems:

  1. Conversation history — stored as Message rows in PostgreSQL (packages/db/src/messages.ts), scoped to a thread. Each message has blocks (text, tool calls, images) and a sequential seq. To manage context windows, history is compacted asynchronously via packages/adapters/src/history-compaction.ts: messages beyond a sliding window of 50 are removed and replaced with an LLM-generated summary stored in thread.historyCompactionSummary. Compaction is triggered when the uncompacted window exceeds the batch size threshold (shouldEnqueueCompaction, line 20). The summary is injected as a user-role message in subsequent runs (line 6047-6054 of executor.ts).

  2. Markdown memory store (packages/memory/src/index.ts — MarkdownMemoryStore): a user-and-bot-scoped key-value store for structured facts. Implements the MemoryStore interface from @rakazo/adapter-kit. Documents are stored in memoryDocument and memoryRevision tables. Supports read (list docs by scope/path), commit (upsert with optimistic concurrency via revision numbers), and search (simple substring match on content/path). The agent calls remember/recall_memory tools to write/read from this store. It uses Prisma transactions with Serializable isolation (line 117).

  3. Semantic (vector) memory — optional providers accessed via SpaceMemoryProviderResolver (packages/adapters/src/memory-provider-factory.ts). Supports Serenity (a self-hostable vector memory service) and Supermemory (external SaaS). These providers expose recall(), save(), and forget() operations for semantic similarity search. The memory scope (isolated vs shared, resolved in packages/db/src/memory-config.ts) controls whether a bot sees only its own memories or the whole space's.

Injection into prompts. At run start, the executor calls semanticMemory.recall() (line 3459 of executor.ts) with the user's prompt as query, getting up to 5 relevant memories. The recalled text is formatted and injected into the prompt's historicalContext array (line 6056-6060). Similarly, compacted history summaries are injected as a synthetic user message. All memory content passes through redactSecrets() before injection.

packages/memory/src/index.ts:28-120packages/adapters/src/history-compaction.ts:20-80packages/adapters/src/executor.ts:3454-3500packages/adapters/src/memory-provider-factory.ts:1-80

How are actions on the user's behalf gated?

answered

Approval / human-in-the-loop. All tool calls by the agent go through a gating pipeline in packages/core/src/action-approval.ts. The toolRequiresApproval() function (line 92) classifies each tool into one of four buckets:

  1. Exempt tools (APPROVAL_EXEMPT_TOOLS — line 3-23): computer_observe, browser_navigate, shell, remember, write_file, etc. — run immediately.
  2. Builtin tools requiring approval (APPROVAL_REQUIRED_BUILTIN_TOOLS — line 25-35): delete_bot, archive_bot, forget_secret, forget_memory, cloud_agent_launch, etc. — must be approved.
  3. Explicit-approval tools (EXPLICIT_APPROVAL_BUILTIN_TOOLS — line 36): create_space — always prompts the user; cannot be pre-allowed.
  4. Connector (integration) tools — the connectorToolRequiresApproval() function (line 85) uses naming heuristics: tools with get_, list_, search_, find_, read_ prefixes are read-only (no approval); tools with create_, delete_, send_, update_, pay_, etc. require approval. A declared readOnly=false override always requires approval. Compound action patterns (_and_, _or_, _then_) are always gated.

Permission scopes. User-defined ActionApprovalRule records allow fine-grained overrides: rules match by exact tool name, connector kind, or category (email or purchase). The resolveActionApprovalDetail() function (line 182) applies deterministic resolution: most-specific matching rule wins. Always-allow and require-approval both beat the default. The planActionGate() function (line 220) adds a third outcome — "judge" — routing consequential default-rule tools to an Auto Review LLM judge when configured.

Auto Review. packages/adapters/src/auto-review.ts defines an LLM-based judge that reviews tool calls before execution. Configured via RAKAZO_AUTO_REVIEW=true env var, plus optional TypeSafe Jev integration (TYPESAFE_API_KEY). applyJudgeDecision() (line 240) maps verdicts: pass → allow, ask → human prompt, error → fail closed for consequential tools.

Unattended / webhook runs. unattendedTriggerToolRequiresApproval() (line 110) applies stricter rules: webhook-triggered runs can only use UNATTENDED_SAFE_BUILTIN_TOOLS (read-only: browser_snapshot, recall_memory, web_fetch, etc.) without approval. Any connector tool in a webhook run needs approval.

Audit trail. Every tool execution records an ExternalEffect via recordEffect() in executor.ts (line 7433) with idempotency keys to prevent replay. Effects transition through statuses (executing → intended → approved, or denied) and are the durable audit log of all actions.

packages/adapters/src/auto-review.ts:1-80packages/adapters/src/approval-effect.ts:1-50packages/adapters/src/executor.ts:3300-3400

How are LLM providers selected and configured?

answered

Supported providers. Rakazo uses the Pi AI framework (@earendil-works/pi-ai) for model access. packages/core/src/model-providers.ts lists popular providers: openrouter, openai-codex (ChatGPT Plus/Pro OAuth), anthropic, openai, google, vercel-ai-gateway, and deepseek. The POPULAR_MODEL_PROVIDER_IDS array (line 46) controls the order in the UI. The full list comes from Pi's builtin model catalog (@earendil-works/pi-ai/providers/all), which is supplemented by CodexCatalogCache for OpenAI Codex models and OpenRouter's real-time catalog.

Configuration surface. Model configuration flows through four layers:

  1. Deployment defaults (packages/adapters/src/deployment-model.ts): PI_DEFAULT_PROVIDER (default openrouter) and OPENROUTER_API_KEY or ANTHROPIC_API_KEY env vars. The default model is openai/gpt-6-luna on OpenRouter, or claude-sonnet-5 for Anthropic.
  2. User credentials stored in ModelCredential rows (DB). Users connect providers through the UI, either by pasting an API key or via OAuth (device-code for ChatGPT/Copilot/xAI, auth-url for Claude Pro/Max). packages/adapters/src/pi-oauth.ts manages these flows.
  3. Per-bot overrides: each bot can pin a specific provider/model/thinking level.
  4. Runtime fallback: if no credential resolves, the Pi runtime's built-in models serve as catch-all (runtimeFallbackModel).

The selectConfiguredModel() function in executor.ts (line 3507) merges all layers, preferring bot override > user default > deployment default > runtime fallback.

Tool-calling / structured output. The Pi runtime (PiAgentRuntime in packages/adapters/src/pi-runtime.ts) drives the LLM stream with declarative tool definitions. Tools are assembled from three sources: built-in tools (packages/adapters/src/builtin-tools.ts), connector tools from the integration stack, and agent skills. The model responds with tool calls, which the executor routes through applyTool() — handling approval gates, secret injection, computer sandbox commands, and payment connectors. Structured output is not explicitly used in the application layer; it relies on function-calling schemas via Pi.

Local-model support. A local provider (pi-local-provider.ts) registers Ollama-compatible endpoints set via RAKAZO_LOCAL_MODELS env var. OpenAI-compatible providers can be added through the UI with a custom base URL. The PiOpenAiCompatibleCatalog and probeOpenAiCompatibleModels() support probing unknown endpoints for their model list. Thinking levels are clamped via clampCatalogThinkingLevel() (line 24 of model-providers.ts) to what the model actually supports.

Editor's note. Correction: POPULAR_MODEL_PROVIDER_IDS is openrouter, openai-codex, anthropic, openai, google and vercel-ai-gateway. It does not include deepseek.
packages/core/src/model-providers.ts:46-75packages/adapters/src/pi-oauth.ts:23-60packages/adapters/src/pi-runtime.ts:1-60packages/adapters/src/executor.ts:3505-3590

How is it deployed and self-hosted?

answered

Runtime dependencies. Rakazo requires PostgreSQL 16 (primary datastore), Docker Engine (for sandboxed computer containers), and optionally Redis (for the supervisor queue). Three processes run: the API server (Hono on port 3100), the Worker (Graphile Worker background job runner), and the Web server (Vite preview serving the React frontend on port 5173). A supervisor service manages Docker-based computer sandboxes, listening on port 7091.

Docker / one-click paths. The primary deployment mechanism is Docker Compose (infra/compose/docker-compose.yml). Services defined: postgres (PostgreSQL 16 with healthcheck), supervisor (sandbox manager), computer (sandbox computer image builder), data-init (permissions fixup), api, worker, and web. All images are built from the monorepo via infra/compose/Dockerfile. A bootstrap script infra/compose/install-images.sh downloads pre-built images from GitHub Container Registry, creates .env with random secrets, and starts everything.

Published images. Images are tagged edge for main branch builds (multi-arch linux/amd64 + linux/arm64). The installer at install-images.sh handles the full workflow: download Compose files, create secrets, pull images, docker compose up. For restricted networks, RAKAZO_DOWNLOAD_BASE overrides the download base URL and --local skips existing files.

Development setup. Requires Node.js 22.22.2+/24.x/26+ and pnpm 9. After cloning, copy .env.example, start Postgres via Docker Compose, then pnpm install && pnpm db:generate && pnpm db:migrate && pnpm sandbox:build && pnpm dev. The pnpm dev command runs the API, worker, web, and sandbox supervisor concurrently via Turbo.

Required external accounts. Rakazo is designed to work without any hosted vendor. Core functionality (model access) requires at minimum an OpenRouter API key (OPENROUTER_API_KEY) or another provider's key. Optional accounts: Composio (COMPOSIO_API_KEY) for the curated integration catalog, Pipedream (PIPEDREAM_CLIENT_ID/SECRET/PROJECT_ID) for Pipedream workflows, and TypeSafe (TYPESAFE_API_KEY) for AI-powered tool review. For voice, ElevenLabs, OpenAI, Cartesia, or Fish Audio keys are optional. Email notification providers (SMTP) are self-hosted.

Production hardening. The infra/compose/ directory includes a Caddyfile.prod for HTTPS termination, harden-host.sh for system hardening, restrict-computer-egress.sh to sandbox computer egress, and backup-prod.sh for database backups. Production Docker Compose overlays (docker-compose.prod.yml, docker-compose.prod.docker.yml) are provided for deployment.

Editor's note. Correction: Redis is not used anywhere in the stack. Background jobs run on Graphile Worker over Postgres, and the local compose file has no Redis service.
infra/compose/install-images.sh:1-10package.json:12-35apps/worker/src/index.ts:60-100