How are API keys, users and tenants managed?

API keys are managed through src/lib/db/apiKeys.ts which stores keys in a SQLite api_keys table with fields for id, key prefix/hash, provider/model permissions, usage limits, spend tracking, rate limits, access schedules, IP allowlists, and tenant/user associations. Key lookup uses constant-time comparison (timingSafeCompare from src/shared/utils/timingSafeCompare). Virtual keys allow multiple users/teams to share upstream credentials while maintaining distinct rate/spend limits. Upstream credentials (OAuth tokens, API keys for downstream providers) live in the provider_connections table managed by src/lib/db/providers.ts, encrypted at rest with AES-256-GCM via the encryptConnectionFields()/decryptConnectionFields() pipeline. Credentials are fetched by getProviderCredentialsByUserIdAndConnectionId(). Auth methods are diverse: API-key auth (bearer token validated by extractApiKey()/isValidApiKey() in route handlers, with requireApiKey: true flag), OAuth flows via src/lib/oauth/, JWT session tokens (src/lib/auth/jwt.ts with signJwt/verifyJwt), and refresh tokens (src/lib/db/tokens.ts). The user/team/tenant model is multi-tenant with src/lib/db/tenants.ts, src/lib/db/teams.ts, and src/lib/db/users.ts. Each API key carries fine-grained permissions: allowed models (with wildcard support via modelPermissions.ts), allowed combos, allowed connections, allowed quotas, access schedules, per-minute/per-day rate limits, spend limits (daily/weekly USD caps), and key groups (apiKeyGroups.ts). The dashboard at src/app/(dashboard)/ provides a UI for key management, while the /v1/* API routes serve the programmatic interface. Auth method discovery is also driven by src/shared/constants/providers/ which classifies providers as OAuth (25), No-Auth (10), Local (14), API-key (bulk), WebCookie (~50+), and Cloud Agent types.

Virtual keys (Verification Tokens). The core auth model is LiteLLM_VerificationToken in the Prisma schema (schema.prisma), which stores hashed API keys with associated permissions (models, budgets, teams, end-users, metadata). The user_api_key_auth.py:3487+ function is the FastAPI dependency called on every proxied request — it looks up the bearer token, validates it against the DB (with Redis cache backing via UserApiKeyCache), and returns a UserAPIKeyAuth object that downstream handlers use for access decisions.

Multi-tenant hierarchy. The data model supports a full RBAC hierarchy: Organizations → Teams → Users → API Keys, with Team Memberships linking Users to Teams with role/permission scoping. Budgets cascade: an Organization budget limits its Teams, a Team budget limits its Members and Keys. The auth check in _can_object_call_model() (litellm/proxy/auth/auth_checks.py) traverses this chain.

Upstream credential storage. Provider API keys are stored in LiteLLM_CredentialsTable (schema.prisma) and can be resolved from environment variables, AWS Secrets Manager, Google Cloud KMS, HashiCorp Vault, or custom secret managers (litellm/secret_managers/).

Auth methods. The proxy accepts API keys via the Authorization: Bearer header (OpenAI-compatible), x-api-key (Anthropic-compatible), or api-key (Azure-compatible) — see litellm/proxy/auth/user_api_key_auth.py:183-199. It also supports JWT auth (litellm/proxy/auth/handle_jwt.py), OAuth2 proxy hooks, and custom SSO.

Admin UI. The /ui route serves a React-based admin dashboard. Management endpoints under /key/, /user/, /team/, /organization/ (litellm/proxy/management_endpoints/) allow CRUD for all auth entities. A master key (LITELLM_MASTER_KEY) bootstraps the first admin.

The system has a three-tier credential model: Users → Tokens (virtual API keys with quotas) → Upstream channel keys.

Virtual API keys are represented by the Token model (model/token.go:14-33). Each Token belongs to a User, has its own Key string (the sk- prefix bearer token clients use), RemainQuota / UnlimitedQuota, ModelLimits (per-model allowlist), AllowIps (IP restriction), ExpiredTime, Group assignment, and CrossGroupRetry / AutoGroups for multi-group routing. Tokens are sent to clients; the gateway looks up the token by its key, identifies the user, and then maps it to an upstream channel.

Upstream credential storage is in the Channel model (model/channel.go:23-60). Each Channel stores its Key (the upstream API key, possibly multi-key via JSON array or newline-separated), BaseURL, API Type, and ModelMapping. Multi-key Channels support MultiKeyModeRandom or MultiKeyModePolling for key rotation (model/channel.go:249-289).

Users and teams use the User model (model/user.go:79-115) with fields for Username, Password (hashed), Role (common/admin/root), Status, Email, Quota/UsedQuota, Group assignment, and OAuth bindings (GitHub, Discord, WeChat, OIDC, Telegram, LinuxDo). User sessions use JWT access tokens with refresh token rotation (model/user_session.go:42+, service/auth_token.go:19+).

Authentication methods: (1) Relay API auth (middleware/auth.go:524-564) — TokenAuth() middleware extracts the bearer token from Authorization: Bearer sk-..., or from x-api-key (Anthropic) or x-goog-api-key (Gemini) headers for protocol compatibility. (2) Dashboard auth — browser sessions (cookies + JWTs) via UserAuth()/AdminAuth()/RootAuth() (middleware/auth.go:132-148), plus Personal Access Tokens (PAT) (model/user_access_token.go:38-52) with scoped permissions. (3) Step-up verification (service/security_verification.go:18-48) for sensitive actions (password change, 2FA setup, admin user management) using password, passkey, TOTP, or OAuth challenge. (4) WebAuthn/Passkeys and TOTP are fully supported.

Admin UI and API are served by the React frontend at the / path, with management endpoints under /api/* (routed in router/main.go). The admin dashboard manages channels, tokens, users, groups, pricing, and log viewing.

User model. Users have roles (model/user.go:19-24): RoleGuestUser (0), RoleCommonUser (1), RoleAdminUser (10), RoleRootUser (100). Fields include username, password (bcrypt-hashed), quota, used_quota, request_count, group (for routing), and OAuth IDs for GitHub, WeChat, Lark, OIDC (model/user.go:34-53). The root user is auto-created on first run with password 123456 (model/main.go:24-64).

Auth methods. The system supports password-based login (session cookie via gin-contrib/sessions), Bearer access tokens (for API/management), and OAuth (GitHub, WeChat, Lark, OIDC). The middleware/auth.go provides three auth middleware for the admin API: UserAuth(), AdminAuth(), RootAuth() checking minRole (auth.go:15-71). For the relay API, TokenAuth() (auth.go:91-151) validates user tokens sent as Authorization: Bearer sk-... headers.

Virtual keys (Tokens). Each token (model/token.go:23-37) is a 48-char Key linked to a UserId. Tokens have statuses (enabled/disabled/expired/exhausted), RemainQuota, UnlimitedQuota, Models (comma-separated whitelist), Subnet (CIDR restriction), and ExpiredTime. The token prefix sk- is stripped during parsing, and the remaining key is validated via Redis cache or DB (model/cache.go:28-56).

Token scoping. On validation, middleware/auth.go:125-131 checks if the requested model is in the token's allowed model list. The Subnet field is verified against c.ClientIP() (auth.go:104-109). An advanced sk-{key}-{channelId} format allows admins to pin a request to a specific channel (auth.go:135-142).

Admin UI/API. The REST API (router/api.go) exposes CRUD for channels (/api/channel), tokens (/api/token), users (/api/user), redemptions (/api/redemption), and system options (/api/option). Role-based access control gates each route. Three React frontends (default, berry, air) provide the web UI.

Upstream credential storage. Each Channel stores its provider API key in the Key field (plaintext in DB, model/channel.go:21-41). Additional config (region, SK, AK for AWS/Vertex) goes in the Config JSON field. The channel's key is injected as Authorization: Bearer {key} on outbound requests (middleware/distributor.go:73).

Upstream credential storage is handled entirely via request headers, not a server-side secrets store. API keys are passed through the Authorization: Bearer <key> header or an x-portkey-api-key header (src/globals.ts:14). The constructConfigFromRequestHeaders function at src/handlers/handlerUtils.ts:836 extracts the key and spreads it into the provider options object. For provider-specific authentication (Azure, AWS/Bedrock, GCP Vertex, Oracle), additional headers like x-portkey-azure-resource-name, x-portkey-aws-access-key-id, x-portkey-vertex-service-account-json etc. are parsed and merged into the config at lines 839-1179.

Virtual keys are referenced via the x-portkey-virtual-key header or virtualKey field in the config (src/types/requestBody.ts:49, src/globals.ts:27). They are passed through to the provider config but the repository itself does not store or manage virtual key mappings — they are resolved externally by the Portkey SaaS or a pre-request hook.

Users and teams have no built-in model in the gateway code. The metadata field (x-portkey-metadata header, src/handlers/services/requestContext.ts:110-116) can carry user/team identifiers as arbitrary JSON, which is available to hooks and the pre-request validator for budget/rate-limit decisions.

Authentication for the local admin UI is handled by src/middlewares/adminAuth/index.ts:80. It uses a static admin_token from conf.json, supporting both Bearer token auth and cookie-based sessions (12-hour expiry, SESSION_MAX_AGE_SECONDS at line 5). The login endpoint at /public/auth (in src/start-server.ts:59) issues an HttpOnly portkey_admin_session cookie. This protects /log/stream (SSE log streaming) and the UI served at /public/.

There is no multi-tenant isolation, user directory, or API-key-based user authentication in the open-source gateway — those are Portkey SaaS features.

API keys, users, and tenant management uses a consumer-based model with the ai-quota and key-auth plugins, plus per-provider API token configuration.

Virtual keys/consumer identification: The ai-quota plugin reads the x-mse-consumer HTTP header to identify the consumer (user/tenant) and enforces quota against Redis — ai-quota/main.go:161-167. This consumer identity is propagated downstream and reused by ai-statistics for per-consumer metrics — ai-statistics/main.go:726-728. A working ai-quota key-auth example shows how API keys map to consumers: credentials are extracted from request headers and validated — plugins/wasm-go/examples/key-auth/main.go.

Upstream credential storage: Each provider config holds apiTokens []string — one or more API tokens for the upstream LLM service — ai-proxy/provider/provider.go:333. At request time, the SetApiTokenInUse method selects a token (via failover-global random or context-based selection) and stores it in the request context — ai-proxy/provider/provider.go:724-734. Token failover uses a CAS-based shared data store to track available vs. unavailable tokens per provider instance — ai-proxy/provider/failover.go:139-224.

Per-request authentication header handling: The plugin saves the original Authorization header at first hop and restores it on internal redirects (using X-HI-ORIGINAL-AUTH), distinguishing first-hop from re-entry requests via the x-higress-fallback-from header — ai-proxy/main.go:156-197. The upstream token replaces the Authorization header for the LLM call.

Admin UI/API: The ai-quota plugin exposes an admin API at /v1/chat/completions/quota with refresh, delta (increment/decrement), and query operations — ai-quota/main.go:316-331. These are authenticated by comparing the caller's consumer identity against a configured admin_consumer — ai-quota/main.go:336-339. Data is stored in Redis with key prefix chat_quota: — ai-quota/main.go:117-118.

User/team model: The system does not have a built-in user/team/tenant model in the Wasm plugins — consumer identity via the x-mse-consumer header is the primary mechanism, suitable for integration with an upstream identity provider. The Kubernetes CRD-based control plane in api/kubernetes/ manages plugin configuration through CustomResourceDefinitions — api/kubernetes/crd_contract.go:1-27.

Bifrost implements a virtual-key-based access model. Virtual keys (sk-bf-... prefix) are the primary auth mechanism: each TableVirtualKey (configstore/tables/virtualkey.go) maps to a set of TableVirtualKeyProviderConfig rows that define which providers the key can reach, along with allowed/blacklisted models, weights, and associated upstream API keys in the Keys many-to-many relationship. The Value of each TableKey (configstore/tables/key.go) stores the actual upstream credential (e.g. OpenAI API key) as a SecretVar that supports env-var references. Scope hierarchy: Each virtual key can belong to teams and customers (TableTeam, TableCustomer), forming a hierarchy for budget and rate-limit propagation. Users are identified via UserID on requests and can have their own scoped routing rules. Auth flow: The governance plugin's ResolveAccess (plugins/governance/resolver.go) resolves what a request may reach from its presented virtual key, then evaluates provider gates, model gates, and key restrictions. The HttpTransportPreHook can reject requests before they reach the LLM pipeline. Admin UI: The web UI (React frontend embedded in the Go binary) provides visual configuration of virtual keys, provider keys, teams, customers, and routing rules. OIDC/OAuth2 user provisioning (framework/oauth2/) supports OAuth 2.0 / OIDC login with background directory sync for teams and roles, writing provisioned users into the same governance store. MCP tool authorization (plugins/governance/mcpauthorization.go) extends the virtual key model to MCP tool access, with per-VMCP-tool allow/deny rules.

Authentication works through virtual API keys (AccessKey). The authenticate() function in internal/gateway/auth.go:25-52 extracts a key from the Authorization header (Bearer) or key query parameter. It hashes the plaintext with a keyHasher and looks it up in snapshot.AccessKeysByHash. The key is validated against expiration (ExpiresAtMS) and allowed peer CIDRs (AllowedPeerCIDRs), returning an AccessKeyView (internal/state/snapshot.go:190-203).

AccessKey model — each key has: ID, Name, KeyHash, KeyPrefix/KeySuffix (for display), Status (active/disabled), ExpiresAtMS, AllowedPeerCIDRs, RPMLimit, ConcurrencyLimit, PriceMultiplier, Filters (on Groups/Protocols/Models), and CostLimitRules.

Credential management — upstream API keys are stored encrypted via encryption.Service. The runtimeCredentialRegistry interface (handler.go:80-92) manages credential lifecycle: ActiveEncryptedCredentialDataIfMatch(), SetCooldownWithChange(), IncrFailure(), SetBlacklistedWithChange(), ClearFailure(). Credentials belong to Groups — each group connects to one channel (provider) with its own params, models, and proxy config. Secrets are identified via connection.Type — either "api_key" or a subscription driver.

No user/tenant model — the system does not implement user/team/tenant hierarchies. AccessKeys are the sole principal. Key-level Filters control which groups, protocols, and models a key can access (state.FilterSet, snapshot.go:110-114), providing capability-based access control without multi-tenant abstractions.

Credential encryption — execution_boundary.go:24-65 normalizes channel credentials: it validates against the channel's connection type, optionally delegates to a subscription driver (subscriptionruntime.Runtime.Driver()), and extracts api_key plus secret values (client_id, client_secret, tenant_id, etc.). Proxy configs can be per-credential, encrypted and integrity-checked via proxy.go:12-53.

Admin UI — the web frontend (web/src/main.ts) supports both a "classic" and "modern" frontend. The internal/webui/ package serves the admin dashboard with http_routes.go for API endpoints and page_routes.json/modern_page_routes.json for route definitions. The control package provides the configuration API surface for CRUD operations on keys, groups, and credentials.

API keys, users, and tenants are managed via the Kubernetes-native BackendSecurityPolicy CRD. Each AIServiceBackend has an associated BackendSecurityPolicy that defines the authentication type and stores credentials. Six auth types are supported: APIKey (Bearer token), AzureAPIKey, AnthropicAPIKey, AWSCredentials (SigV4 signing with region), AzureCredentials (access token), and GCPCredentials (access token with region/project) (internal/controller/gateway.go:913-1004). Static credentials are stored in Kubernetes Secrets referenced by the BackendSecurityPolicy's secretRef, resolved via getBSPSecretRefData() (internal/controller/gateway.go:1035-1047). The apiKeyInSecret constant defines the standard key used within secrets (internal/controller/ai_gateway_route.go:48). Cross-namespace credential references are validated through ReferenceGrant resources, enforced by the referenceGrantValidator (internal/controller/gateway.go:490-505). Per-request credential override is supported via CredentialOverride with two modes: FromRequestHeaders (injected by trusted ingress filters using x-aigw-* headers) and FromDynamicMetadata (from Envoy dynamic metadata). Default input header names per auth type include x-aigw-api-key, x-aigw-anthropic-api-key, etc. (internal/controller/gateway.go:827-853). The backendauth package implements handlers per auth type: apiKeyHandler sets the Authorization header, newAWSHandler performs SigV4 signing, newAzureHandler for Azure tokens, etc. (internal/backendauth/auth.go:16-67). Token rotation is handled by dedicated rotator controllers for AWS OIDC, Azure, and GCP tokens (internal/controller/rotators/). There is no built-in user/tenant model or admin UI — user identification is done via request headers (e.g., x-tenant-id) for downstream rate limiting.

Plano does not implement a user/tenant/API-key management system with virtual keys, user/team/tenant models, or an admin UI. These features are not present in the codebase at the reviewed commit.

API keys for upstream LLM providers are stored in each LlmProvider config entry's access_key: Option<String> field (crates/common/src/configuration.rs:765), which holds the credential used when proxying to that provider. The passthrough_auth: Option<bool> flag (line 776) controls whether the client's original Authorization header is forwarded directly to the upstream, bypassing the configured access_key. The Authorization header is obfuscated in logs via obfuscate_auth_header() in crates/common/src/pii.rs.

The tenant_header optional field on SessionCacheConfig (crates/common/src/configuration.rs:26) allows scoping session-cache keys to a tenant namespace by reading a named HTTP header, but this is for cache isolation, not authentication — there is no tenant identity provisioning or validation. No user model, team model, organization hierarchy, or virtual-key abstraction was found. The project instead delegates authentication to the client's existing infrastructure (the caller authenticates, Plano proxies). A search of the entire crates/ tree for "virtual_key", "user_model", "team", "organization", and "tenant_model" with no results.