Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ jobs:
- name: Create the R2 bucket if it is missing
if: steps.creds.outputs.ready == 'true'
run: |
if ! npx wrangler r2 bucket info llm-gateway >/dev/null 2>&1; then
npx wrangler r2 bucket create llm-gateway
if ! npx wrangler r2 bucket info sub2api >/dev/null 2>&1; then
npx wrangler r2 bucket create sub2api
fi
- if: steps.creds.outputs.ready == 'true'
run: npx wrangler deploy
133 changes: 83 additions & 50 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,88 +1,121 @@
# llm-gateway
# sub2api: a Cloudflare-based LLM gateway and proxy server

Your own subscription logins (Claude, Codex, Gemini and the others [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) supports) served as an API from your Cloudflare account. One click deploys it; then any app uses your subscriptions with a base URL and a key, from anywhere. No machine of yours has to stay on.
**Literally the [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI) (CPA) project, deployed as a Cloudflare Worker in one click.** Turn your Claude, ChatGPT Codex and Gemini subscriptions into your own private API. Point Claude Code, the Anthropic or OpenAI SDKs, or any app at your gateway URL and use your subscription from anywhere, with no server or home machine left running.

[![Deploy to Cloudflare](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/PlaygrndLabs/llm-gateway)
[![Deploy to Cloudflare](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/PlaygrndLabs/sub2api)
[![ci](https://github.com/PlaygrndLabs/sub2api/actions/workflows/ci.yml/badge.svg)](https://github.com/PlaygrndLabs/sub2api/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

```
app ──► Worker llm-gateway.<subdomain>.workers.dev ──► Container: CPA v8.0.24 ──► Claude / Codex / Gemini
│ /v1/* → key from the dashboard, then CPA │
│ /login → dashboard behind a password └─ http://r2.internal ─► Worker S3 shim ─► R2 bucket
└ everything else → 404
```

- **The real CPA**, built from a pinned upstream commit, with its matching dashboard (`management.html` v1.26.1, checksum-pinned). Not a port: every provider and API CPA supports works the same way.
- **State in R2 with no keys to create.** CPA saves its config and logins through its S3 object store. Inside the container that store points at `http://r2.internal`, which the Worker intercepts and serves from an R2 binding (`src/s3.ts`). Restarts, sleeps and redeploys keep every login.
- **Locked down by default.** API keys exist only once you make them in the dashboard; until then every API call is refused. CPA checks every `/v1/*` and `/v1beta/*` call against its key list, but on its own it lets every call through when that list is empty. So before forwarding, the Worker asks CPA with a random key: if CPA accepts it, no keys are set up and the Worker refuses the call; if CPA can't be reached, the Worker refuses too (`src/key-guard.ts`). The dashboard opens only after a password sign-in (an HMAC-signed, HttpOnly, SameSite=Strict cookie) and still asks for CPA's management key. CPA's own unauthenticated pages, such as its OAuth callbacks, return 404.
- **Cheap when idle.** One `basic` instance (1/4 vCPU, 1 GiB) that sleeps after 30 minutes without requests and wakes on the next one.
## Why

## Deploy
- **One click, no server.** Cloudflare builds and runs everything. Nothing to keep online.
- **The real CPA.** Built from a pinned upstream release with its official dashboard, so every provider and API CPA supports just works: Anthropic Messages, OpenAI-compatible and Gemini.
- **Private by default.** The dashboard sits behind a password, and every API call needs a key you create. Until then, all calls are refused.
- **Logins that stick.** Config and OAuth logins live in your own R2 bucket and survive restarts and redeploys.
- **Costs little when idle.** The container sleeps after 30 minutes and wakes on the next request.

You need a Cloudflare account on the Workers Paid plan ($5 a month; Containers require it).
## Quick start

1. Press **Deploy to Cloudflare** above. Cloudflare copies this repository to your GitHub account, creates the R2 bucket and builds the container.
2. It asks for two secrets. Generate each with `openssl rand -hex 32` (any value of at least 32 characters of `A-Z a-z 0-9 _ -` works):
- `DASHBOARD_PASSWORD`: your password for the dashboard sign-in.
- `MANAGEMENT_KEY`: CPA's management key, which the dashboard asks for after sign-in.
3. When the deploy finishes, open `https://llm-gateway.<your-subdomain>.workers.dev`, sign in, and make an API key for each app under API Keys.
You need a Cloudflare account on the [Workers Paid plan](https://developers.cloudflare.com/workers/platform/pricing/) ($5/month; Containers require it).

`MANAGEMENT_KEY` seeds CPA's config on its first start only. From then on the config in R2 is the source of truth: rotate it, and add or remove app keys, in the dashboard. `DASHBOARD_PASSWORD` stays a Worker secret and can be changed any time in the Cloudflare dashboard or with `wrangler secret put DASHBOARD_PASSWORD`.
1. Press **Deploy to Cloudflare** above.
2. Set the two secrets it asks for. Each needs at least 32 characters of `A-Z a-z 0-9 _ -`; `openssl rand -hex 32` makes one:
- `DASHBOARD_PASSWORD`: your dashboard sign-in password.
- `MANAGEMENT_KEY`: CPA's management key, asked for after sign-in.
3. Open `https://sub2api.<your-subdomain>.workers.dev`, sign in, and create an API key under **API Keys**.

## Log in to a subscription
## Connect a subscription

1. Open your gateway, sign in with the dashboard password, then enter the management key.
2. Under OAuth login, choose the provider (for Claude: Anthropic) and open the link it gives you.
3. Sign in with that provider. The browser then lands on a `localhost` address that does not load; that is expected, since each provider only returns to the address registered for its login. Copy the whole address from the address bar (it carries `code` and `state`) and paste it into the dashboard's Callback URL field. Or, in the same browser where you are signed in to the gateway, replace `http://localhost:<port>` with your gateway address and open it: the gateway hands the callback to CPA.
1. In the dashboard, open **OAuth login** and pick a provider (for Claude: Anthropic).
2. Sign in with the provider. You land on a `localhost` page that doesn't load; that's expected.
3. Copy that address into the dashboard's **Callback URL** field. Or replace `http://localhost:<port>` with your gateway URL and open it in the same browser.

CPA saves the login to R2 and refreshes its tokens itself.
CPA stores the login and refreshes its tokens on its own.

## Use it from an app
## Use it

The Anthropic Messages API, with streaming and tools, is what the Anthropic SDKs, the Claude Agent SDK and Claude Code read:
Claude Code, the Claude Agent SDK and the Anthropic SDKs:

```
ANTHROPIC_BASE_URL=https://llm-gateway.<subdomain>.workers.dev
ANTHROPIC_API_KEY=<an API key made in the dashboard>
```bash
export ANTHROPIC_BASE_URL=https://sub2api.<your-subdomain>.workers.dev
export ANTHROPIC_API_KEY=<your gateway API key>
```

Send the key as `x-api-key` or `Authorization: Bearer` (Gemini clients may use `x-goog-api-key` or `?key=`). OpenAI-compatible clients use `https://llm-gateway.<subdomain>.workers.dev/v1` with the same key; Gemini clients use `/v1beta`.
OpenAI-compatible clients (Codex CLI, OpenAI SDKs, Cursor, and others):

```bash
export OPENAI_BASE_URL=https://sub2api.<your-subdomain>.workers.dev/v1
export OPENAI_API_KEY=<your gateway API key>
```
curl https://llm-gateway.<subdomain>.workers.dev/v1/messages \

Gemini clients use `<gateway>/v1beta` with the same key.

```bash
curl https://sub2api.<your-subdomain>.workers.dev/v1/messages \
-H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" -H "content-type: application/json" \
-d '{"model":"claude-sonnet-5-5","max_tokens":64,"messages":[{"role":"user","content":"Say hi"}]}'
```

`GET /v1/models` lists the models your logged-in subscriptions offer. Give every app its own key in the dashboard, so removing one cuts off only that app. A key works as soon as you save it.
`GET /v1/models` lists what your subscriptions offer. Give each app its own key, so you can revoke one without touching the rest.

## Deploy with wrangler instead
## How it works

```
npm ci
npx wrangler deploy # needs Docker running to build the image
sh scripts/set-secrets.sh # generates the two secrets, stores them, prints them once
app ──► Worker ──► Durable Object ──► Container: CPA ──► Claude / Codex / Gemini
· API key check │
· dashboard sign-in └─► http://r2.internal ──► R2 bucket
```

To deploy from GitHub Actions, add the repository secrets `CLOUDFLARE_API_TOKEN` (Workers Scripts, Containers and Workers R2 Storage: Edit) and `CLOUDFLARE_ACCOUNT_ID`; the `deploy` workflow then runs after every green `ci` run on `main`. Without them it skips.
The Worker hands every request to one Durable Object in front of the CPA container. The Durable Object refuses API calls until a key exists and keeps the dashboard behind a password. CPA's S3 storage is served from an R2 binding, so you never create storage keys. See [SECURITY.md](SECURITY.md) for the full model.

## Develop
## Update

A Deploy-button copy redeploys whenever its `main` branch changes. To get the latest release:

```bash
git pull https://github.com/PlaygrndLabs/sub2api main
git push
```

If git refuses because the histories are unrelated, add `--allow-unrelated-histories` to the pull once. Logins and config live in R2, so updates keep them.

<details>
<summary><b>Deploy with wrangler or GitHub Actions</b></summary>

```bash
npm ci
npx wrangler login
npx wrangler r2 bucket create sub2api
npx wrangler deploy # needs Docker to build the image
sh scripts/set-secrets.sh # generates both secrets, stores them, prints them once
```

To deploy from GitHub Actions, add the repository secrets `CLOUDFLARE_API_TOKEN` (Workers Scripts, Containers, Workers R2 Storage: Edit) and `CLOUDFLARE_ACCOUNT_ID`. Every green `ci` run on `main` then deploys.

`MANAGEMENT_KEY` seeds CPA's config on first start only; after that, change it in the dashboard. `DASHBOARD_PASSWORD` can be changed any time with `wrangler secret put DASHBOARD_PASSWORD`.
</details>

<details>
<summary><b>Develop</b></summary>

```bash
npm ci
npm run verify # typecheck, lint, unit tests
sh test/container-smoke.sh # needs Docker: builds the image and runs CPA against the S3 shim
npm run verify # typecheck, lint, unit tests
sh test/container-smoke.sh # needs Docker: runs CPA against the R2 shim
```

`npm run types` regenerates `worker-configuration.d.ts` after a `wrangler.jsonc` change. To move to a newer CPA, change `CPA_COMMIT` in `container/Dockerfile` together with a dashboard release that speaks the same management API (`PANEL_VERSION` and `PANEL_SHA256`).
To upgrade CPA, change `CPA_COMMIT` in `container/Dockerfile` together with a matching dashboard release (`PANEL_VERSION`, `PANEL_SHA256`).
</details>

## Limits
## Good to know

- Requests to providers leave from Cloudflare's network, not your home connection. Check that your provider's terms allow how you use your subscription.
- The first request after the container sleeps waits a few seconds while it starts.
- One instance serves everything; CPA handles concurrent requests itself.
- Requests reach providers from Cloudflare's network. **Check that your provider's terms allow how you use your subscription.**
- The first request after the container sleeps takes a few seconds while it starts.
- Cost is the $5 Workers Paid plan plus container time while awake. See [Containers pricing](https://developers.cloudflare.com/containers/pricing/).
- Lost the management key? Set a new `MANAGEMENT_KEY` secret, delete `config/config.yaml` from the `sub2api` R2 bucket, and redeploy. CPA starts with a fresh config. Provider logins stay, but you recreate your app API keys.

Containers pricing: [developers.cloudflare.com/containers/pricing](https://developers.cloudflare.com/containers/pricing/).
Built on [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI). Not affiliated with Anthropic, OpenAI, Google or Cloudflare.

## License

MIT
[MIT](LICENSE)
23 changes: 23 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Security

## Reporting a vulnerability

Please report privately through [GitHub's private vulnerability reporting](https://github.com/PlaygrndLabs/sub2api/security/advisories/new). Don't open a public issue. We aim to reply within a week.

Bugs in CPA itself belong upstream at [router-for-me/CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI).

## Security model

Everything a deploy holds belongs to its owner: your Cloudflare account, your R2 bucket and your secrets. This repository has no keys, telemetry or shared backend.

- **API calls need a key.** On its own, CPA lets every `/v1/*` and `/v1beta/*` call through while its key list is empty. The gateway closes that gap: before forwarding, it asks CPA with a random key (`src/key-guard.ts`). If CPA accepts it, no keys are set up, so the gateway refuses the call. If CPA can't be reached, the gateway refuses too. The check runs in the single Durable Object in front of the container, so every request sees the same answer, and a dashboard change takes effect at once.
- **The dashboard needs two secrets.** First the `DASHBOARD_PASSWORD` sign-in, which sets an HMAC-signed `__Host-` cookie (HttpOnly, Secure, SameSite=Strict) that lasts 12 hours. Then CPA's own management key. Sign-in attempts are limited to 5 a minute per IP. Changing the password signs out every session.
- **Only known paths reach CPA.** The API, the dashboard, `/healthz` and the OAuth callback (for a signed-in user only) are forwarded. Everything else, and any path with encoded separators, returns 404. Browser cookies are stripped before forwarding.
- **State stays in your R2 bucket.** CPA's S3 client talks to `http://r2.internal`, which only the container can reach and which the Worker serves from an R2 binding. No R2 API tokens exist.
- **A pinned supply chain.** CPA is built from a pinned commit, and the dashboard HTML is checksum-verified (`container/Dockerfile`).

## Your part

- Use long random secrets: `openssl rand -hex 32`.
- Give each app its own API key, and delete a key the moment it may have leaked.
- Never commit `.dev.vars`. It is in `.gitignore`.
5 changes: 3 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 26 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,30 @@
{
"name": "llm-gateway",
"description": "One-shot CLIProxyAPI on Cloudflare: your subscription logins as an API from a Worker and a Container, state in R2.",
"name": "sub2api",
"description": "A Cloudflare-based LLM gateway and proxy server: the CLIProxyAPI (CPA) project deployed as a Cloudflare Worker in one click.",
"license": "MIT",
"homepage": "https://github.com/PlaygrndLabs/sub2api#readme",
"repository": {
"type": "git",
"url": "git+https://github.com/PlaygrndLabs/sub2api.git"
},
"bugs": {
"url": "https://github.com/PlaygrndLabs/sub2api/issues"
},
"keywords": [
"cliproxyapi",
"cpa",
"cloudflare-workers",
"cloudflare-containers",
"claude-code",
"llm-gateway",
"llm-proxy",
"ai-gateway",
"openai-compatible",
"anthropic",
"codex",
"gemini",
"self-hosted"
],
"private": true,
"type": "module",
"scripts": {
Expand Down
20 changes: 20 additions & 0 deletions src/cpa.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,20 @@
import { Container } from "@cloudflare/containers";
import type { Env } from "./env.ts";
import { handle } from "./gateway.ts";
import { KeyGuard } from "./key-guard.ts";
import { handleS3 } from "./s3.ts";

export const STATE_HOST = "r2.internal";
export const STATE_BUCKET = "state";

const nowSeconds = () => Math.floor(Date.now() / 1000);

// Every request runs here, in the one Durable Object in front of the container, so a single
// KeyGuard sees every dashboard change no matter which Worker isolate took the request.
export class Cpa extends Container<Env> {
override defaultPort = 8317;
override sleepAfter = "30m";
private readonly keyGuard = new KeyGuard(nowSeconds);

constructor(ctx: ConstructorParameters<typeof Container>[0], env: Env) {
super(ctx, env);
Expand All @@ -19,6 +26,19 @@ export class Cpa extends Container<Env> {
CPA_MANAGEMENT_KEY: env.MANAGEMENT_KEY,
};
}

override fetch(request: Request): Promise<Response> {
return handle(
request,
{
dashboardPassword: this.env.DASHBOARD_PASSWORD,
nowSeconds,
keyGuard: this.keyGuard,
loginLimiter: this.env.LOGIN_LIMIT,
},
(forwarded) => this.containerFetch(forwarded),
);
}
}

Cpa.outboundByHost = {
Expand Down
1 change: 1 addition & 0 deletions src/env.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import type { Cpa } from "./cpa.ts";
export interface Env {
readonly CPA: DurableObjectNamespace<Cpa>;
readonly STATE: R2Bucket;
readonly LOGIN_LIMIT: RateLimit;
readonly MANAGEMENT_KEY: string;
readonly DASHBOARD_PASSWORD: string;
}
12 changes: 11 additions & 1 deletion src/gateway.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,22 @@ import { loginPage, messagePage } from "./login-page.ts";

export type Forward = (request: Request) => Promise<Response>;

export interface LoginLimiter {
limit(options: { key: string }): Promise<{ success: boolean }>;
}

export interface GatewayConfig {
readonly dashboardPassword: string;
readonly nowSeconds: () => number;
readonly keyGuard: KeyGuard;
readonly loginLimiter: LoginLimiter;
}

const MAX_LOGIN_BODY = 4096;
export const MIN_PASSWORD_LENGTH = 16;
const NO_CLIENT_KEYS = "No API key is set up yet: add one under API Keys in the dashboard.";
const CPA_UNAVAILABLE = "The gateway is starting or unavailable. Try again shortly.";
const TOO_MANY_ATTEMPTS = "Too many sign-in attempts. Wait a minute and try again.";
const PASSWORD_NOT_SET = `Sign-in is off: set the DASHBOARD_PASSWORD secret to at least ${MIN_PASSWORD_LENGTH} characters.`;

function redirect(location: string, headers: HeadersInit = {}): Response {
Expand Down Expand Up @@ -59,8 +65,10 @@ async function api(request: Request, url: URL, config: GatewayConfig, forward: F
}

async function dashboard(request: Request, config: GatewayConfig, forward: Forward): Promise<Response> {
const writes = request.method !== "GET" && request.method !== "HEAD";
if (writes) config.keyGuard.forget();
const response = await forward(withoutCookies(request));
if (request.method !== "GET" && request.method !== "HEAD") config.keyGuard.forget();
if (writes) config.keyGuard.forget();
return response;
}

Expand Down Expand Up @@ -89,6 +97,8 @@ async function login(request: Request, config: GatewayConfig): Promise<Response>
if (config.dashboardPassword.length < MIN_PASSWORD_LENGTH) return html(loginPage(PASSWORD_NOT_SET), 503);
const length = Number(request.headers.get("content-length") ?? "0");
if (!Number.isFinite(length) || length > MAX_LOGIN_BODY) return html(loginPage("Request too large."), 413);
const client = request.headers.get("cf-connecting-ip") ?? "unknown";
if (!(await config.loginLimiter.limit({ key: client })).success) return html(loginPage(TOO_MANY_ATTEMPTS), 429);
const form = await request.formData().catch(() => null);
const password = form?.get("password");
if (typeof password !== "string" || !(await passwordMatches(password, config.dashboardPassword))) {
Expand Down
Loading
Loading