OpenAI-compatible providers

Inspect AI talks to any OpenAI-compatible endpoint. This page covers the model string format and the common providers.

The model string

Inspect uses a provider-qualified model string:

openai-api/<provider-key>/<model-name>

openai-api is Inspect's name for the OpenAI-compatible provider driver.
<provider-key> is an arbitrary label you choose; it disambiguates keys when you have several providers.
<model-name> is the model id the provider's API expects.

You pass this to inspect eval --model ....

Z.ai example

A Z.ai configuration looks like:

ZAI_API_KEY=...
OPENAI_BASE_URL=https://api.z.ai/api/paas/v4
INSPECT_EVAL_MODEL=openai-api/zai/glm-4.5
LANGCHAIN_AGENT_MODEL=glm-4.5

A common pattern is to map whatever keys you keep in .env to the ones the OpenAI client and Inspect expect (ZAI_API_KEY → OPENAI_API_KEY, which the OpenAI client reads, and OPENAI_BASE_URL → ZAI_BASE_URL, which Inspect reads for the zai provider key):

if [[ -n "${ZAI_API_KEY:-}" && -z "${OPENAI_API_KEY:-}" ]]; then
  export OPENAI_API_KEY="$ZAI_API_KEY"
fi
if [[ -n "${OPENAI_BASE_URL:-}" ]]; then
  export ZAI_BASE_URL="$OPENAI_BASE_URL"
fi

So the zai in openai-api/zai/glm-4.5 is resolved by Inspect to the ZAI_BASE_URL + OPENAI_API_KEY.

OpenAI itself

OPENAI_API_KEY=sk-...
INSPECT_EVAL_MODEL=openai-api/openai/gpt-4o
LANGCHAIN_AGENT_MODEL=gpt-4o

No custom base URL needed.

Other OpenAI-compatible endpoints

Any endpoint that implements the /chat/completions shape works: Together, Anyscale, OpenRouter, vLLM, Ollama's OpenAI shim, etc. Set:

OPENAI_API_KEY (or the provider's key).
A base URL pointing at the compatible endpoint.
The INSPECT_EVAL_MODEL string with a provider key you name consistently with the env vars Inspect needs (e.g. for a provider key acme, set ACME_BASE_URL and ACME_API_KEY).

Inspect provider docs

For the full provider matrix (Anthropic, Google, Bedrock, Azure, etc., which use their own drivers rather than openai-api), see Inspect providers.

Verifying

A quick connectivity check before a full eval:

inspect eval evals/evals.py --model "$INSPECT_EVAL_MODEL" --limit 1

If the first sample completes, the provider config is correct.