Evaris
log insign up

Reference

Data model

The shapes the SDK and API exchange. These mirror the package's generated models and the OpenAPI schemas. When those change, this page is updated to match.

Run

A single eval execution.

type Run = {
  run_id: string;
  project_id: string;
  execution_mode: "external" | "managed";
  status: "created" | "ingesting" | "succeeded" | "failed" | "ingested_with_warnings";
  name?: string;
  tags: Record<string, string>;
  source?: InspectSource;
  warnings: RunWarning[];
  artifact?: Artifact;
  created_at: string;
  completed_at?: string;
};

Run status lifecycle

created ──completeRun──▶ ingesting ──▶ succeeded
                                  ├──▶ ingested_with_warnings
                                  └──▶ failed
  • created: run exists, no artifact yet. Deletable. This is the only state deleteRun accepts.
  • ingesting: completeRun accepted the manifest; the backend is parsing the artifact and materializing records.
  • succeeded: ingestion finished cleanly.
  • ingested_with_warnings: ingestion finished but the backend emitted RunWarnings (e.g. an unsupported inspect version). The artifact is still stored; samples may be partially materialized.
  • failed: ingestion threw, or the manifest declared status: "failed".

Artifact

The immutable raw output attached to a run.

type Artifact = {
  type: "inspect_eval_log";   // only type today
  uri: string;                // storage URI (R2 in prod, filesystem in tests)
  content_type: string;       // application/json | application/octet-stream
  size_bytes: number;
  checksum_sha256: string;    // hex SHA-256 of the bytes
  created_at: string;
  producer: string;           // e.g. "evaris-inspect-publish"
};

ArtifactUpload

The pre-signed upload target returned by createArtifactUpload.

type ArtifactUpload = {
  artifact_id: string;
  run_id: string;
  uri: string;
  upload_url: string;         // PUT your bytes here
  method: "PUT";
  headers: Record<string, string>;   // include these on the PUT
  expires_at: string;
};

ArtifactManifest

type ArtifactManifest = {
  schema_version: "2026-06-11";
  run_id: string;
  artifact: Artifact;
};

RunManifest

The signed summary sent on completeRun. See Publish eval logs for the full prose.

type RunManifest = {
  schema_version: "2026-06-11";
  run_id: string;
  project_id: string;
  execution_mode: "external" | "managed";
  producer: { name: string; version: string };
  source: InspectSource;
  status: "succeeded" | "failed";
  started_at: string;
  completed_at: string;
  artifact_manifest: ArtifactManifest;
  summary: { artifact_count: number; artifact_bytes: number };
  checksums: { artifact_manifest_sha256: string };
};

The checksums.artifact_manifest_sha256 is the SHA-256 of the serialized artifact_manifest JSON; the backend verifies it on completeRun.

InspectSource

Records what produced the artifact.

type InspectSource = {
  name: "inspect_ai";        // only source today
  version: string;           // detected inspect-ai version (or "unknown")
  log_format?: string;
  python_version?: string;
};

RunWarning

type RunWarning = {
  code: "unsupported_inspect_version" | "invalid_inspect_version";
  message: string;
};

Emitted by the backend during ingestion. Surfaced on the run's warnings array and in the UI.

Project API token

type ProjectApiToken = {
  id: string;
  project_id: string;
  name?: string;
  token: string;             // full token: returned ONCE at creation
  token_prefix: string;      // what's stored after creation
  created_at: string;
};

Bootstrap response

type BootstrapSetupResponse = {
  tenant: SetupTenant;
  project: SetupProject;       // the default project
  projects: SetupProject[];    // all projects the user can access
};

type SetupTenant = {
  id: string;
  slug: string;
  name: string;
  role: "owner" | "admin" | "member";
};

type SetupProject = {
  id: string;
  tenant_id: string;
  public_id: string;           // this is the --project value for the CLI
  name: string;
};

Schema versioning

ArtifactManifest and RunManifest both carry a schema_version. Current: 2026-06-11. The publisher and backend must agree on the version; a mismatch shows up as a 400 from completeRun. When the contract evolves, a new version is added (not mutated in place) and the publisher is updated to emit it.