AI Model Gateway

AI Model Gateway exposes OpenAI-compatible model APIs behind Open4X authentication, billing, usage logs, cost limits, and Connected Accounts.

Supported Endpoints

Required scope:

ai:invoke

POST /v1/apps/ai/:alias/chat
POST /v1/apps/ai/:alias/chat/completions
POST /v1/apps/ai/:alias/v1/chat/completions
POST /v1/apps/ai/:alias/chat/stream
POST /v1/apps/ai/:alias/embeddings
POST /v1/apps/ai/:alias/v1/embeddings

The /chat/completions and nested /v1/* routes are OpenAI-compatible aliases for tools that build paths from a provider baseUrl.

BYOK Provider Model

The current MVP supports BYOK. Users bring their own OpenAI-compatible provider key, either directly on the AI service instance or through a Connected Account.

Supported Connected Account provider types:

openai-compatible
openrouter
deepseek
custom-api-key

Service Configuration

Direct API key mode stores an encrypted provider key:

{
  "provider": "openai-compatible",
  "mode": "byok",
  "base_url": "https://api.openai.com/v1",
  "default_model": "gpt-4o-mini",
  "api_key_encrypted": "enc:v1:...",
  "max_tokens_per_request": 2048,
  "max_cost_per_request": 0.05,
  "timeout_ms": 30000,
  "fallback_aliases": ["backup"],
  "routing_policy": "manual",
  "log_requests": false
}

Connected Account mode stores a reference instead:

{
  "provider": "openai-compatible",
  "mode": "byok",
  "connected_account_id": "conn_...",
  "base_url": "https://openrouter.ai/api/v1",
  "default_model": "anthropic/claude-sonnet",
  "max_tokens_per_request": 4096,
  "max_cost_per_request": 0.10,
  "timeout_ms": 30000,
  "fallback_aliases": ["backup"]
}

timeout_ms defaults to 30 seconds and is bounded to 250–120000 ms. It covers the upstream response headers and the idle time between streaming chunks. fallback_aliases accepts up to three enabled AI service aliases owned by the same user. The gateway only falls back for network errors, HTTP 408, 429, or 5xx responses; authentication and request validation failures are not retried. When a fallback is selected, the response includes X-OpenEdge-Fallback-Alias.

routing_policy is manual by default. Set it to complexity_first only after ordering the configured fallbacks so the strongest validated model is first. Requests with general math/geometry, debugging, architecture, compliance, medical, legal, or financial signals try fallbacks before the primary alias; this is an opt-in routing heuristic, not a correctness guarantee. The request text is inspected only in memory and is not stored by the routing layer.

The Console AI Playground can record metadata-only evaluation results. The leaderboard includes pass rate, average latency, average token usage, and average final cost; prompts, expectations, answers, and warning text remain in the browser and are not stored.

Chat Example

curl -X POST https://api.open4x.com/v1/apps/ai/default/chat \
  -H "X-API-Key: sk_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      { "role": "user", "content": "Summarize this text in three bullets." }
    ],
    "max_tokens": 1000,
    "max_cost": 0.05
  }'

The response body follows the upstream OpenAI-compatible format. Billing metadata is returned in headers:

X-OpenEdge-Estimated-Cost: 0.0005
X-OpenEdge-Final-Cost: 0.0005
X-OpenEdge-Total-Tokens: 400

Streaming

curl -N -X POST https://api.open4x.com/v1/apps/ai/default/chat/stream \
  -H "X-API-Key: sk_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{ "role": "user", "content": "Write a short release note." }],
    "stream": true
  }'

Open4X forwards Server-Sent Events from the provider and settles usage when the stream completes. If the upstream provider does not return usage, settlement falls back to the pre-authorized estimate. The initial stream response exposes the estimated cost; the final cost is not appended to an already established SSE response header, so verify final settlement in billing or usage records.

OpenAI-compatible clients can also stream through the standard chat completions route by sending "stream": true:

curl -N -X POST https://api.open4x.com/v1/apps/ai/default/v1/chat/completions \
  -H "Authorization: Bearer sk_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{ "role": "user", "content": "Write a short release note." }],
    "stream": true
  }'

OpenClaw

Use an API key with the ai:invoke scope, then add a custom OpenAI-compatible provider in OpenClaw:

{
  "models": {
    "mode": "merge",
    "providers": {
      "openedge": {
        "baseUrl": "https://api.open4x.com/v1/apps/ai/qwen122/v1",
        "apiKey": "sk_xxx",
        "api": "openai-completions",
        "models": [
          {
            "id": "mlx-community/Qwen3.5-122B-A10B-4bit",
            "name": "Qwen3.5 122B",
            "reasoning": true,
            "input": ["text"],
            "contextWindow": 131072,
            "maxTokens": 32768
          }
        ]
      }
    }
  },
  "agents": {
    "defaults": {
      "models": {
        "openedge/mlx-community/Qwen3.5-122B-A10B-4bit": {
          "alias": "qwen122"
        }
      }
    }
  }
}

OpenClaw sends provider keys as Authorization: Bearer <apiKey>; Open4X accepts that form as well as X-API-Key.

Embeddings

curl -X POST https://api.open4x.com/v1/apps/ai/default/embeddings \
  -H "X-API-Key: sk_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "text-embedding-3-small",
    "input": ["first document", "second document"],
    "max_cost": 0.02
  }'

Billing

BYOK billing charges for gateway usage:

final_cost = request_fee + total_tokens / 1,000,000 * token_gateway_fee

The gateway checks balance before the call, records estimated and final cost, and refunds over-reserved balance when final usage is lower than the estimate.

Security

• Provider API keys are encrypted with CONFIG_ENCRYPTION_KEY.
• The console never returns raw provider keys.
• base_url must be HTTPS and must pass SSRF protection.
• Request logs store provider, model, token usage, status, cost, and latency by default, not prompts or full model responses.