# How are LLM providers selected and configured?

> Open-source personal assistants — a good answer covers: Supported providers; config surface; tool-calling / structured-output usage; local-model support.

Canonical page: https://llms-technical-reviews.com/personal-assistants/q/models/

## Verdict

[OpenBot](/p/openbot/) supports exactly three provider IDs: `openai`, `anthropic` and `google`, defined in `shared/model-providers.json` and checked by both TypeScript and Python loaders. `BOT_PROVIDER` and `BOT_MODEL` override the defaults, and per-provider base URL variables allow an OpenAI-compatible endpoint, which is the only route to a local model. Remote AG-UI Bots choose their own models, so the platform-level setting mostly affects built-in Bots. The approval auto-review uses structured JSON output.

[Rakazo](/p/rakazo/) delegates to the Pi framework, which brings a large built-in catalogue. The deployment default is OpenRouter (`PI_DEFAULT_PROVIDER`, `PI_DEFAULT_MODEL`), with Anthropic as an alternative deployment key. Users connect their own credentials in the UI, by API key or by OAuth for ChatGPT, Claude, GitHub Copilot and xAI subscriptions. Bots can pin a model and thinking level, and local Ollama-style or OpenAI-compatible endpoints are first-class.

Choose Rakazo if users should bring their own models or subscriptions, or you want local models. OpenBot's narrow list is enough when an admin sets one provider for the deployment and agent frameworks handle the rest.

More projects in this category are being researched.

## Per-project answers

### CopilotKit/OpenBot (answered)

**Supported providers.** Three providers are hardcoded in TypeScript and Python: `openai`, `anthropic`, `google` (`shared/model-providers.ts:42`). Each has a spec row in `shared/model-providers.json` (`shared/model-providers.json:1-38`) defining its API key environment variable, base URL override variable, and default model. The default models are `gpt-5.5` (OpenAI), `claude-sonnet-4-5` (Anthropic), and `gemini-2.5-flash` (Google). Each agent-framework adapter chooses which SDK it can drive — adding Google's spec doesn't make agent-mastra (which loads no Google module) able to call Gemini.

**Configuration surface.** Provider and model are selected through environment variables (`server/src/copilot.ts:205-220`): `BOT_PROVIDER` and `BOT_MODEL` override the defaults from the spec file. An empty provider string defaults to OpenAI (matching what the desktop writes when switching back). Base URLs can be overridden per provider (`OPENAI_BASE_URL`, `ANTHROPIC_BASE_URL`, `GOOGLE_GENERATIVE_AI_BASE_URL`) — the Anthropic variant automatically versions to `/v1` if missing. Per-Bot defaults are defined in the `bots` section of `model-providers.json`, keyed by agent directory name (e.g., `agent-bot` defaults to OpenAI/gpt-5.5, `agent-langgraph` to OpenAI/gpt-5.5).

**Tool-calling and structured output.** The built-in agents use the AG-UI client's `AbstractAgent` interface for tool calling, with middleware for memory injection and learning. The approval auto-review uses structured JSON schema output (`server/src/approvals/policy.ts:188-194`) via Zod: `{ verdict, preApproved, reason, safety }`. The Composio integration passes action schemas (as published by the vendor) directly to the model as function definitions. MCP tools are listed with their JSON Schema parameters and returned as model-callable tools.

**Local model support.** There is no built-in local model support beyond the base URL override mechanism. Any provider that speaks an OpenAI-compatible API can be pointed at by setting `OPENAI_BASE_URL` to a local endpoint (e.g., Ollama, vLLM). The Google provider similarly supports `GOOGLE_GENERATIVE_AI_BASE_URL`. No quantized, local-first, or on-device model is shipped.

> **Editor's note.** Addition: `gpt-5.5` is the default in the JSON spec, but when the provider is switched through environment variables, `runtimeModelForEnvironment` falls back to `gpt-5.6-terra`.

Citations: [shared/model-providers.ts:1-45](https://github.com/CopilotKit/OpenBot/blob/f4bc60bf9b12c65eb3d7640173f1b3432f6b2a60/shared/model-providers.ts#L1-L45) · [shared/model-providers.ts:80-145](https://github.com/CopilotKit/OpenBot/blob/f4bc60bf9b12c65eb3d7640173f1b3432f6b2a60/shared/model-providers.ts#L80-L145) · [shared/model-providers.json:1-38](https://github.com/CopilotKit/OpenBot/blob/f4bc60bf9b12c65eb3d7640173f1b3432f6b2a60/shared/model-providers.json#L1-L38) · [server/src/copilot.ts:205-220](https://github.com/CopilotKit/OpenBot/blob/f4bc60bf9b12c65eb3d7640173f1b3432f6b2a60/server/src/copilot.ts#L205-L220) · [server/src/approvals/policy.ts:188-194](https://github.com/CopilotKit/OpenBot/blob/f4bc60bf9b12c65eb3d7640173f1b3432f6b2a60/server/src/approvals/policy.ts#L188-L194)

### elie222/rakazo (answered)

**Supported providers.** Rakazo uses the **Pi AI framework** (`@earendil-works/pi-ai`) for model access. `packages/core/src/model-providers.ts` lists popular providers: `openrouter`, `openai-codex` (ChatGPT Plus/Pro OAuth), `anthropic`, `openai`, `google`, `vercel-ai-gateway`, and `deepseek`. The `POPULAR_MODEL_PROVIDER_IDS` array (line 46) controls the order in the UI. The full list comes from Pi's builtin model catalog (`@earendil-works/pi-ai/providers/all`), which is supplemented by `CodexCatalogCache` for OpenAI Codex models and OpenRouter's real-time catalog.

**Configuration surface.** Model configuration flows through four layers:
1. **Deployment defaults** (`packages/adapters/src/deployment-model.ts`): `PI_DEFAULT_PROVIDER` (default `openrouter`) and `OPENROUTER_API_KEY` or `ANTHROPIC_API_KEY` env vars. The default model is `openai/gpt-6-luna` on OpenRouter, or `claude-sonnet-5` for Anthropic.
2. **User credentials** stored in `ModelCredential` rows (DB). Users connect providers through the UI, either by pasting an API key or via OAuth (device-code for ChatGPT/Copilot/xAI, auth-url for Claude Pro/Max). `packages/adapters/src/pi-oauth.ts` manages these flows.
3. **Per-bot overrides**: each bot can pin a specific provider/model/thinking level.
4. **Runtime fallback**: if no credential resolves, the Pi runtime's built-in models serve as catch-all (`runtimeFallbackModel`).

The `selectConfiguredModel()` function in `executor.ts` (line 3507) merges all layers, preferring bot override > user default > deployment default > runtime fallback.

**Tool-calling / structured output.** The Pi runtime (`PiAgentRuntime` in `packages/adapters/src/pi-runtime.ts`) drives the LLM stream with declarative tool definitions. Tools are assembled from three sources: built-in tools (`packages/adapters/src/builtin-tools.ts`), connector tools from the integration stack, and agent skills. The model responds with tool calls, which the executor routes through `applyTool()` — handling approval gates, secret injection, computer sandbox commands, and payment connectors. Structured output is not explicitly used in the application layer; it relies on function-calling schemas via Pi.

**Local-model support.** A local provider (`pi-local-provider.ts`) registers Ollama-compatible endpoints set via `RAKAZO_LOCAL_MODELS` env var. OpenAI-compatible providers can be added through the UI with a custom base URL. The `PiOpenAiCompatibleCatalog` and `probeOpenAiCompatibleModels()` support probing unknown endpoints for their model list. Thinking levels are clamped via `clampCatalogThinkingLevel()` (line 24 of model-providers.ts) to what the model actually supports.

> **Editor's note.** Correction: `POPULAR_MODEL_PROVIDER_IDS` is openrouter, openai-codex, anthropic, openai, google and vercel-ai-gateway. It does not include deepseek.

Citations: [packages/core/src/model-providers.ts:46-75](https://github.com/elie222/rakazo/blob/4cdf3e23315c2633b6b3fde8977f197b986f06fa/packages/core/src/model-providers.ts#L46-L75) · [packages/adapters/src/pi-oauth.ts:23-60](https://github.com/elie222/rakazo/blob/4cdf3e23315c2633b6b3fde8977f197b986f06fa/packages/adapters/src/pi-oauth.ts#L23-L60) · [packages/adapters/src/pi-runtime.ts:1-60](https://github.com/elie222/rakazo/blob/4cdf3e23315c2633b6b3fde8977f197b986f06fa/packages/adapters/src/pi-runtime.ts#L1-L60) · [packages/adapters/src/executor.ts:3505-3590](https://github.com/elie222/rakazo/blob/4cdf3e23315c2633b6b3fde8977f197b986f06fa/packages/adapters/src/executor.ts#L3505-L3590)