Skip to content

Connect another OpenAI-compatible endpoint

Goal

Route the primary Colossus model through a hosted gateway or private endpoint that implements the OpenAI-compatible contracts required by the Colossus Chat Completions adapter.

Prerequisites

  • An HTTPS endpoint and exact API prefix, commonly ending in /v1.
  • An exact model ID and verified model-catalog, Chat Completions, tool-call, and streaming behavior.
  • An environment-backed credential when authentication is required.
  • A PEM CA bundle when the endpoint certificate chains to a private authority.
  • A schema version 2 configuration generated by colossus config init.

“OpenAI-compatible” is not a guarantee that every server works. Colossus requires the specific catalog and generation response shapes used by its adapter and validates them strictly.

Steps

1. Inject the credential and trust roots

Set COLOSSUS_MODEL_TOKEN through the Colossus process environment or a secret manager. Keep only env:COLOSSUS_MODEL_TOKEN in YAML. Place public CA certificates—not private keys—in the workspace-owned PEM file used below. Omit network.caBundlePath when public trust roots already validate the endpoint.

2. Configure the compatible route

Run colossus -w . config effective, edit the reported resolution.configPath, and keep its required storage block and any intended custom settings. Apply this validated overlay, replacing the example host, model ID, limits, and CA path with exact deployment values.

schemaVersion: 2
network:
  caBundlePath: .colossus/certs/provider-ca-bundle.pem
providers:
  profiles:
    compatible-provider:
      kind: open_ai_compatible
      baseUrl: https://models.example.com/v1
      credentialReference: env:COLOSSUS_MODEL_TOKEN
      chatCompletionsOutputTokenParameter: max_completion_tokens
models:
  profiles:
    compatible:
      providerProfile: compatible-provider
      model: example-model
      contextWindowTokens: 128000
      maxOutputTokens: 16000
      capabilities:
        toolCalls: true
        streaming: true
  roles:
    primary: compatible
sandbox:
  networkDestinations:
    - https://models.example.com

Colossus appends /models and /chat/completions to baseUrl. Include the API prefix but not either operation path. Under an isolating boundary, the sandbox destination is the exact origin only; acknowledged full access needs no duplicate grant and an origin entry does not narrow ambient authority. When timeoutMs is omitted, catalog and generation requests use 5 minutes for remote hosts and 15 minutes for localhost, IPv4 loopback, or IPv6 loopback. Set a positive timeoutMs only when the connection needs an explicit override.

The example selects the modern max_completion_tokens Chat Completions field. Use max_tokens for a legacy endpoint, or omit only when the endpoint rejects both token limit fields. Omitting chatCompletionsOutputTokenParameter defaults to max_tokens so existing compatible-provider profiles keep their current wire contract. This provider setting only changes how the model profile's single maxOutputTokens value is named on the request; Colossus never sends both fields or retries with a different one.

Set toolCalls and streaming from observed support for the exact server and model. Current provider profiles use bearer credentials and do not expose arbitrary custom request headers; do not assume a service requiring additional headers is compatible.

3. Inspect routing and readiness

colossus -w . models route primary
colossus -w . provider models compatible-provider
colossus -w . provider doctor compatible-provider
colossus -w . models doctor compatible

4. Send one bounded model turn

colossus -w . run \
  "Reply with exactly: connected"

Expected result

The route names compatible, the catalog and generation diagnostics succeed with the configured trust and timeout, and the smoke test returns connected.

Verification

Run colossus -w . config effective and confirm the provider uses the intended exact declared origin or ambient network authority. Confirm config show contains only the credential reference and CA bundle path, never the token or certificate private keys.

Failure path

  • The credential is unavailable: confirm COLOSSUS_MODEL_TOKEN is present in the Colossus process environment, then rerun provider doctor compatible-provider.
  • The endpoint returns 404: verify the API prefix in baseUrl; omit explicit /models and /chat/completions suffixes.
  • TLS validation fails: correct network.caBundlePath and the PEM certificate chain, then rerun provider doctor compatible-provider.
  • The provider catalog works but generation fails: verify the exact model ID, timeout, response format, chatCompletionsOutputTokenParameter, tool-call support, and streaming support. Correct the profile and capability fields, then rerun models doctor compatible.
  • The origin is denied under isolation: grant the exact scheme, host, and effective port without the API path.

Next step

Review exact fields and compatibility constraints in Providers and models, then apply deployment policy from Providers and routing.