LLM gateways

LLM gateways put many model providers behind one API. A client sends one request format, usually OpenAI's, and the gateway picks a provider, translates the call, retries or falls back on errors, and records usage. The ten projects here split along four axes. The first is form: an SDK plus proxy, a self-hosted console for sharing keys with billing, a lightweight proxy, or plugins inside an Envoy-based gateway. The second is routing: weighted or rule-based load balancing, fair scheduling over pools of keys or accounts, or intent-based model choice. The third is governance: how deep the user, team and budget model goes, and whether money is counted from a price table or only tokens. The fourth is where state lives: in one process, or shared through a database and Redis. When choosing, check which inbound formats you need. Check whether limits hold across replicas, and which features are only hooks or enterprise extras. Also check the providers' terms for pooled or subscription credentials.