The brief
A side-project-turned-real-product. Teams kept asking the same question: "how do I let my app call OpenAI / Anthropic / Azure without checking provider keys into env files everywhere?" SecureLLM is a thin LLM gateway that solves that — provider-agnostic routing, per-tenant key vaulting, usage telemetry.
What I did
- Designed and built it solo. From the data model up: tenants, projects, upstream-key vault, request log, cost ledger.
- Provider abstraction. A single
/v1/chat/completions-shape endpoint that routes to OpenAI, Anthropic, or Azure based on a per-project routing rule. Clients see one API; rotation between providers is a config change. - Tenant-scoped key vaulting. Upstream API keys live encrypted in Supabase and never leave the gateway. Clients authenticate to the gateway with their own short-lived tokens.
- Usage + cost ledger. Every request gets logged with token counts and computed cost; dashboards land out of the box.
Outcome
- Stood up the full gateway (auth, routing, logging, dashboards) end-to-end.
- Used internally to sanity-check provider migrations without touching app code.
What I learned
The interesting part of an LLM gateway isn't the proxying — it's the policy surface around it. Routing, quotas, observability, key rotation. The proxy is the easy bit.