Evaris
log insign up

Getting Started

Quickstart

End-to-end: sign up, mint a token, run a sample Inspect eval, publish it, see it in the runs UI. Allow ~15 minutes.

Prerequisites

Complete Install. You need:

  • Python 3.10+ with inspect-ai installed.
  • Node 20+ with the evaris CLI (npm install evaris).
  • An Evaris account.

1. Sign up and bootstrap

Sign up with email + OTP. On first login the web UI bootstraps your workspace and default project automatically.

2. Mint a project API token

In the UI, create a project API token for the default project. Copy the token value immediately: it is shown only once. See Authentication for why.

Export it for the CLI:

export EVARIS_API_URL=https://api.evarisai.com
export EVARIS_API_TOKEN=<paste-token-here>

You will also need the project id (its public id, shown in the UI). The example below uses support-agent; substitute your own.

3. Run a sample Inspect eval

If you don't have an eval handy, follow Inspect AI integration to author one: a small task in evals/evals.py with a samples.jsonl next to it.

Configure an OpenAI-compatible provider in your environment (see OpenAI-compatible providers). For Z.ai:

ZAI_API_KEY=...
OPENAI_BASE_URL=https://api.z.ai/api/paas/v4
INSPECT_EVAL_MODEL=openai-api/zai/glm-4.5

Run the eval (produces a .eval log under logs/):

inspect eval evals/evals.py --model "$INSPECT_EVAL_MODEL" --log-dir logs --limit 5

4. Publish the log to Evaris

evaris publish logs --project <your-project-id>

On success the CLI prints JSON:

{
  "run_id": "...",
  "status": "succeeded",
  "artifacts": 1,
  "warnings": []
}

The log directory must contain exactly one .eval/.json(.gz) file. Publishing fails with a clear error if there are zero or more than one.

5. See the run in the UI

Open the runs page in the web UI. Your run should appear with status succeeded and the attached inspect_eval_log artifact. If the backend emitted warnings (e.g. an unsupported inspect version), the status will be ingested_with_warnings and the warnings will be listed.

Troubleshooting

  • 401 Authentication required: token missing, wrong, or scoped to a different project. Re-check EVARIS_API_TOKEN and --project.
  • 404 Project not found: the project id does not match the token's project.
  • No Inspect AI log files found: the log dir has no .eval/.json file, or you pointed at the wrong directory.
  • Expected one Inspect AI log file, found N: clean the directory or point at a specific file instead of a directory.
  • Status stays created: publish failed mid-way; the SDK auto-deletes the orphaned run. Re-run.

Next