How are LLM providers selected and configured?

Supported providers. Three providers are hardcoded in TypeScript and Python: openai, anthropic, google (shared/model-providers.ts:42). Each has a spec row in shared/model-providers.json (shared/model-providers.json:1-38) defining its API key environment variable, base URL override variable, and default model. The default models are gpt-5.5 (OpenAI), claude-sonnet-4-5 (Anthropic), and gemini-2.5-flash (Google). Each agent-framework adapter chooses which SDK it can drive — adding Google's spec doesn't make agent-mastra (which loads no Google module) able to call Gemini.

Configuration surface. Provider and model are selected through environment variables (server/src/copilot.ts:205-220): BOT_PROVIDER and BOT_MODEL override the defaults from the spec file. An empty provider string defaults to OpenAI (matching what the desktop writes when switching back). Base URLs can be overridden per provider (OPENAI_BASE_URL, ANTHROPIC_BASE_URL, GOOGLE_GENERATIVE_AI_BASE_URL) — the Anthropic variant automatically versions to /v1 if missing. Per-Bot defaults are defined in the bots section of model-providers.json, keyed by agent directory name (e.g., agent-bot defaults to OpenAI/gpt-5.5, agent-langgraph to OpenAI/gpt-5.5).

Tool-calling and structured output. The built-in agents use the AG-UI client's AbstractAgent interface for tool calling, with middleware for memory injection and learning. The approval auto-review uses structured JSON schema output (server/src/approvals/policy.ts:188-194) via Zod: { verdict, preApproved, reason, safety }. The Composio integration passes action schemas (as published by the vendor) directly to the model as function definitions. MCP tools are listed with their JSON Schema parameters and returned as model-callable tools.

Local model support. There is no built-in local model support beyond the base URL override mechanism. Any provider that speaks an OpenAI-compatible API can be pointed at by setting OPENAI_BASE_URL to a local endpoint (e.g., Ollama, vLLM). The Google provider similarly supports GOOGLE_GENERATIVE_AI_BASE_URL. No quantized, local-first, or on-device model is shipped.

Supported providers. Rakazo uses the Pi AI framework (@earendil-works/pi-ai) for model access. packages/core/src/model-providers.ts lists popular providers: openrouter, openai-codex (ChatGPT Plus/Pro OAuth), anthropic, openai, google, vercel-ai-gateway, and deepseek. The POPULAR_MODEL_PROVIDER_IDS array (line 46) controls the order in the UI. The full list comes from Pi's builtin model catalog (@earendil-works/pi-ai/providers/all), which is supplemented by CodexCatalogCache for OpenAI Codex models and OpenRouter's real-time catalog.

Configuration surface. Model configuration flows through four layers:

1. Deployment defaults (packages/adapters/src/deployment-model.ts): PI_DEFAULT_PROVIDER (default openrouter) and OPENROUTER_API_KEY or ANTHROPIC_API_KEY env vars. The default model is openai/gpt-6-luna on OpenRouter, or claude-sonnet-5 for Anthropic.
2. User credentials stored in ModelCredential rows (DB). Users connect providers through the UI, either by pasting an API key or via OAuth (device-code for ChatGPT/Copilot/xAI, auth-url for Claude Pro/Max). packages/adapters/src/pi-oauth.ts manages these flows.
3. Per-bot overrides: each bot can pin a specific provider/model/thinking level.
4. Runtime fallback: if no credential resolves, the Pi runtime's built-in models serve as catch-all (runtimeFallbackModel).

The selectConfiguredModel() function in executor.ts (line 3507) merges all layers, preferring bot override > user default > deployment default > runtime fallback.

Tool-calling / structured output. The Pi runtime (PiAgentRuntime in packages/adapters/src/pi-runtime.ts) drives the LLM stream with declarative tool definitions. Tools are assembled from three sources: built-in tools (packages/adapters/src/builtin-tools.ts), connector tools from the integration stack, and agent skills. The model responds with tool calls, which the executor routes through applyTool() — handling approval gates, secret injection, computer sandbox commands, and payment connectors. Structured output is not explicitly used in the application layer; it relies on function-calling schemas via Pi.

Local-model support. A local provider (pi-local-provider.ts) registers Ollama-compatible endpoints set via RAKAZO_LOCAL_MODELS env var. OpenAI-compatible providers can be added through the UI with a custom base URL. The PiOpenAiCompatibleCatalog and probeOpenAiCompatibleModels() support probing unknown endpoints for their model list. Thinking levels are clamped via clampCatalogThinkingLevel() (line 24 of model-providers.ts) to what the model actually supports.