UKGovernmentBEIS/inspect_ai

Inspect provides three tiers of scoring: per-sample scorers, per-task metrics, and epoch reducers.

Built-in scorers (src/inspect_ai/scorer/) each return a Score object with a value (str/int/float/bool/list/dict). They include:

• match — string matching at begin/end/any/exact locations, with case-insensitive and numeric options
• includes — substring containment check
• exact — normalized exact-match for QA
• f1 — SQuAD-style F1 token overlap
• choice — multiple-choice letter grading with unshuffle support
• math — symbolic math via SymPy/LaTeX parsing with sandboxed expression validation, timeout, and complexity limits
• answer — extracts ANSWER:-prefixed answers by letter/word/line pattern
• model_graded_qa / model_graded_fact — LLM-as-a-judge using configurable templates and grader models (covered under llm-judge)
• perplexity — scores via prompt logprobs NLL
• cascade — chains scorers cheapest-first, short-circuiting when threshold met
• multi_scorer — runs multiple scorers in parallel and reduces via majority/mode/mean
• precomputed_scores — loads externally computed scores from JSON/JSONL by sample ID

Metrics (src/inspect_ai/scorer/_metrics/) aggregate per-sample scores into eval-level values:

• accuracy — proportion correct, with CORRECT/INCORRECT/PARTIAL/NOANSWER sentinels and a pluggable ValueToFloat converter
• mean / std / var — arithmetic mean, sample standard deviation, variance
• stderr / bootstrap_stderr / ci / ci_wilson — standard error (plain or clustered by sample metadata), confidence intervals via t-distribution or percentile bootstrap, Wilson score for binary proportions
• frequency / categorical — categorical score distribution (counts or proportions), with StrEnum integration and zero-fill for unobserved categories
• aggregate — extracts one key from dict-valued scores and runs another metric on it
• grouped — partitions scores by metadata key and applies a metric per group
• krippendorff_alpha — inter-rater agreement across multiple judges

Custom scorers are created with the @scorer decorator. The Scorer protocol requires an async callable (state: TaskState, target: Target) -> Score | None. Custom metrics use @metric and accept list[SampleScore] -> Value. Metrics can declare a scores mode: "auto" (reduced), "reduced", or "unreduced" (per-epoch).

Score reducers (src/inspect_ai/scorer/_reducer/) aggregate multi-epoch samples: mean_score, median_score, mode_score, max_score, majority_score, pass_at, pass_k, at_least, collect_score.

LLM-as-a-judge is implemented via model_graded_qa and model_graded_fact scorers (src/inspect_ai/scorer/_model.py).

Judge prompts and rubrics. Both scorers accept a template with {question}, {answer}, {criterion}, and {instructions} variables plus any sample metadata keys. The default instructions ask the grader to emit a GRADE: C / GRADE: I (or GRADE: P for partial credit) verdict after step-by-step reasoning. The model_scoring_prompt() function formats the template, neutralizes structural delimiters ([BEGIN DATA]/[END DATA]) in dataset-controlled inputs to prevent prompt injection, and preserves media attachments.

Structured output via regex. The grade is extracted by regex with a leading greedy .* (DOTALL) that ensures the last GRADE: X wins — crucial for injection robustness. A permissive variant captures any word when the default instructions are used, then validates against the grades actually offered. Off-menu verdicts are treated as parse failures (grader_failed), not incorrect answers.

Multi-sample / consensus. When model is a list of models (or model_role binds to a list), each model grades independently via multi_scorer() which runs them concurrently with tg_collect. By default (reducer="majority"), a grade needs >50% of graders to agree. reducer="mode" picks the most common grade with tie-breaking by model order.

Judge model choice. Controlled via the model parameter (takes precedence) or model_role (default "grader", resolved from eval's model_roles). Falls back to the model under evaluation when no judge model is specified.

Calibration / bias controls. No explicit bias-calibration is provided — the approach relies on prompt engineering and majority-vote reducers. Anti-injection measures include structural delimiter neutralization and grade-pattern validation.

Dataset definition. A Dataset is a Sequence[Sample] (src/inspect_ai/dataset/_dataset.py:144-221). Each Sample has input, target, choices, id, metadata, sandbox, files, setup, and checkpoint fields. The input can be a string or a list of ChatMessage objects.

File formats and sources (src/inspect_ai/dataset/_sources/):

• hf_dataset — loads from Hugging Face datasets with retry logic for transient errors, field mapping via FieldSpec, and support for splits/shuffling
• json_dataset — JSON array of sample dicts
• csv_dataset — CSV file with configurable field mapping
• file_dataset — generic file (auto-detects JSON/CSV/JSONL by extension)
• example_dataset — inline sample generation via a function

Synthetic data generation is done in user code by writing Python functions that return MemoryDataset(list[Sample]) or by implementing a SampleSource protocol. There is no built-in synthetic data DSL beyond the Sample constructor.

Versioning. Inspect does not version datasets natively — versioning is the data provider's responsibility (e.g. Hugging Face dataset revisions pinned by revision= in hf_dataset()). The eval_set system provides log-based versioning through eval set IDs and manifest files.

Benchmark task registry. Tasks are registered with @task decorator and can be discovered via inspect list or list_tasks(). The Task object binds a Dataset, a Solver/Plan, and optional Scorers, metrics, and config.

Golden sets. There is no built-in "golden set" concept — tasks specify their own dataset at construction time. The SampleSource protocol allows dynamic sample feeding. The precomputed_scores scorer attaches externally computed labels to existing logs by sample ID.

Runners. Evals are launched via eval() / eval_async() (src/inspect_ai/_eval/eval.py). The top-level orchestrator eval_set() (src/inspect_ai/_eval/evalset.py) manages multiple tasks with shared configuration, sample dispatch, retry logic, and logging. Inside a task, task_run() sets up sandbox lifecycle and prepares task options. Samples are fanned out by SampleScheduler, a live fanout loop that accepts new samples mid-run.

Parallelism. parallel= controls concurrent samples, running in an anyio TaskGroup. Model-level parallelism uses adaptive connection pools (DEFAULT_MAX_CONNECTIONS / DEFAULT_MAX_CONNECTIONS_BATCH). max_tasks controls parallel task execution within eval sets.

Caching. Model outputs are cached with configurable TTL. The inspect cache CLI command (src/inspect_ai/_cli/cache.py) provides clear, prune, list, and size operations.

CI integration. Standard Python library approach — pip install inspect-ai && python your_eval.py. The CLI supports --detach for long-running evals, ACP for cross-machine dispatch, and JSON output modes.

Result storage. Logs use a compact binary .eval format (or JSON) via the Recorder hierarchy (src/inspect_ai/log/_recorders/). FileRecorder writes to local/remote filesystems via fsspec. BufferSampleStore provides crash-recovery through a SQLite buffer.

Comparison / regression. Eval results are structured as EvalResults with EvalScore objects per scorer and EvalMetric dicts per metric (src/inspect_ai/log/_log.py:799-904). The inspect view command starts a web dashboard for browsing logs and comparing runs. There is no built-in regression test framework — users compare metrics programmatically from returned EvalLog objects.

SDK instrumentation is provided through the events module and the hooks system, not through OpenTelemetry.

Event system (src/inspect_ai/event/). During sample execution, a timeline of typed events is recorded: ModelEvent, ToolEvent, ScoreEvent, SandboxEvent, StateEvent, StepEvent, SubtaskEvent, InputEvent, ApprovalEvent, ErrorEvent, StoreEvent, SpanEvent, and more. These are organized into an EventTree modeling the hierarchical sample run. The Timeline builds a linearized, filterable view.

Hooks (src/inspect_ai/hooks/). Lifecycle hooks fire at eval_set_start/end, run_start/end, task_start/end, sample_start/init/attempt_start/attempt_end/end/scoring, and before_model_generate. Hooks are passed to eval().

Trace logging (src/inspect_ai/_util/trace.py). A custom TRACE log level captures HTTP requests, model calls, and long-running actions. The inspect trace CLI (src/inspect_ai/_cli/trace.py) lists, dumps, filters traces by action type, and surfaces anomalies.

No OpenTelemetry. There is no opentelemetry dependency or exporter. The tracing system is file-based JSONL.

Online vs offline evals. "Online" evals run via eval() and write to log files in real-time. "Offline" analysis reads logged EvalLog objects via read_eval_log(), recomputes metrics via recompute_metrics(), or edits scores via edit_score().

Feedback and annotation. The precomputed_scores scorer attaches externally computed scores by sample ID. edit_score() modifies logged scores with provenance tracking. There is no built-in annotation UI.

Inspect provides infrastructure useful for red-teaming but has no built-in adversarial probes, attack plugins, jailbreak harnesses, or dedicated red-teaming tooling.

What exists:

• Scanner support (src/inspect_ai/_eval/task/scan.py). An eval_set can attach ScannerConfig objects that run per-sample analysis via the optional inspect_scout package. Designed for post-hoc scanning of completed transcripts, not adversarial generation.
• Review system (src/inspect_ai/review/). The Reviewer protocol intercepts tool call results mid-execution and can continue, terminate, or escalate.
• Sandbox environments (src/inspect_ai/util/_sandbox/). Docker-based sandboxes with diagnostics and egress controls provide isolation for running untrusted model outputs.
• Model-level safety settings. Several model provider modules expose safety/abuse-detection parameters (Anthropic, OpenAI, Google, Grok) but as passthroughs, not a managed red-teaming feature.

What is absent:

• No built-in adversarial attack library (no prompt injection generators, no jailbreak test suites, no fuzzing tools)
• No dedicated red-team evaluation harness or report format
• No vulnerability reporting workflow or CVE tracking
• No built-in harmfulness classifiers or refusal detectors
• No automated red-teaming loop that generates increasingly adversarial inputs

How users do red-teaming today: By writing custom @scorer functions that test for specific failure modes, using the Tool system to build adversarial tool environments, and leveraging the Plan/Solver system to construct multi-turn probe sequences. The framework is flexible enough for ad-hoc red-teaming through these primitives, but provides no turnkey solution.