Evaris
log insign up

Getting Started

Overview

Evaris is an evaluation platform for AI agents. You run an evaluation, today an Inspect AI eval, and Evaris stores the raw artifact, derives structured records, and renders the run in a web UI.

The loop

agent under test ─▶ inspect eval ─▶ .eval artifact ─▶ evaris publish ─▶ backend ingests ─▶ runs UI
  1. You author an eval against your agent (Python, Inspect AI).
  2. inspect eval produces an .eval log file.
  3. The evaris CLI (or publishInspectLogs from the SDK) uploads the log to the Evaris backend as an artifact attached to a run.
  4. The backend ingests the artifact, parses samples, and materializes records.
  5. The runs page in the web UI shows status, warnings, and the artifact.

Core concepts

  • Run: one execution of an eval. Has a status (created, ingesting, succeeded, failed, ingested_with_warnings), a project, tags, and an optional name. Created by createRun, completed by completeRun.
  • Artifact: the immutable raw output of a run. Today the only artifact type is inspect_eval_log. Stored once; parsed many times.
  • Run manifest: the signed summary the publisher sends when completing a run. Contains the artifact manifest, a checksum, source version, and timing. Schema versioned (2026-06-11 is current).
  • Project: the unit of ownership and isolation for runs. A tenant has one or more projects; every run belongs to exactly one project.
  • Project API token: a Bearer token scoped to a project. Used by the SDK and CLI to publish runs. Created once in the web UI; the full value is shown only at creation time.

Execution modes

A run's execution_mode is one of:

  • external: the eval runs on your own infrastructure and you publish the artifact. This is what the SDK and CLI support today.
  • managed: the backend runs the eval. Reserved for future use; the SDK does not yet drive managed execution.

What the SDK does (and does not)

The TypeScript SDK (evaris on npm) is a publishing client. It:

  • creates a run,
  • requests an artifact upload URL and uploads the bytes,
  • builds the run manifest with a checksum,
  • completes the run (which enqueues ingestion),
  • and on failure, abandons the half-created run so nothing orphans.

It does not author evals, call models, or run Inspect. That side is Python.

Next steps