Claude Fable 5
via claude-code-cli 2.1.214model + harness
Run of 18 Jul 2026
4 of 4 result checks passed
1 render or interaction check not run
Evaluation not published
Archived pilot · Historical run
Präzisions-Edit (Multi-Step)
Instruction followingbuilt by Phi-4-mini-instruct via ollama 0.32.15 model only
- Automated checks
- 3 of 4 result checks passed1 render or interaction check not run
- Judge
- Evaluation not published
- Started
- Duration
- 15 s
Artifact
Rendered by agi.jetzt from the model-produced artifactRendered offline in an isolated renderer from the stored file (sha256 984721bc7ab2…). The file itself is not served.
Same brief, other models
Scroll sideways. Each run had its own runtime and limits — compare the artifacts, not a score.
Claude Haiku 4.5
via claude-code-cli 2.1.212model + harness
Run of 17 Jul 2026
4 of 4 result checks passed
1 render or interaction check not run
Evaluation not published
GPT-5.6 Luna
via codex-cli 0.144.5model + harness
Run of 18 Jul 2026
3 of 4 result checks passed
1 render or interaction check not run
Evaluation not published
GPT-5.6 Sol
via codex-cli 0.144.5model + harness
Run of 17 Jul 2026
4 of 4 result checks passed
1 render or interaction check not run
Evaluation not published
Qwen3.8-27B
via ollama 0.32.15model only
Run of 3 Sept 2026
4 of 4 result checks passed
1 render or interaction check not run
Evaluation not published
Automated checks
Automated checks of the stored HTML file, without running it in a browser. Details are the runner’s German originals. The diff checks compare the DOM before and after the edit instruction (DOM diff against the task’s oracle).
| Check | Kind | Result |
|---|---|---|
| element_a_changedElement A exakt geändert | DOM diff | Passed |
| element_b_removedvorher 1, nachher 0 | DOM diff | Passed |
| element_c_dom_identicalElement C DOM-identisch | DOM diff | Passed |
| no_unrequested_changesunbeauftragte DOM-Änderung gefunden | DOM diff | Failed |
| turn2_renders_without_errorsSandbox not available — rendering skipped | Rendering | not run |
1 render or interaction check was not run. Recorded reason: “Sandbox not available — rendering skipped”.
Evaluation
- Deterministic score
- 75 / 100 (3 of 4 passed)Share of the result checks of this run that passed — not a verdict on the artifact as a whole.
- Judge
- not published for this run
WhyThe v2 judge record of this run names the tested model as its own judge and lists no criteria. Its score is therefore not shown — nor the combined v2 score, which blends that judge in.
Limits of this run
- 1 render or interaction check was not run. Whether the artifact works in a browser was not checked automatically.
- No judge evaluation is published for this run. There is therefore no quality verdict — only the automated checks listed here.
- Not fully reproducible; the recorded reason is listed under “Environment”.
Task
Brief (turn 1)
Baue eine responsive Programmseite für die fiktive Konferenz „Werkstatt Digital 2026“ als einzelne HTML-Datei mit Inline-CSS und ohne externe Assets. Die Seite braucht einen Hero, die Textzeile <p id="venue-text">Ort: Speicher 7, Bremen</p>, ein gut sichtbares <aside id="early-bird">Frühbucherpreis bis 31. Juli: 179 €</aside>, drei Sprechenden-Karten und einen Zeitplan als <section id="schedule"> mit vier Programmpunkten. Verwende ein warmes Off-White, dunkles Blau und Koralle, klare Typografie und eine mobile Einspaltenansicht. Die genannten IDs müssen exakt gesetzt werden.Edit instruction (turn 2)
- replace text nur den Textinhalt von <p id="venue-text">Ort: Speicher 7, Bremen</p> → „Ort: Dock 4, Hamburg“
- remove das gesamte Element <aside id="early-bird"> einschließlich seines Inhalts
- leave unchanged den vollständigen DOM-Teilbaum <section id="schedule"> einschließlich aller Programmpunkte
From the task seed (German original); the actual prompts add output-format instructions and other wording.
Environment
- Runtime
- ollama 0.32.15
- Execution
- model onlyThe runner reached the model directly — no agent harness, no tools.
- Profile
- std-v1 · T=0.2 · seed 4242 · 2 turns · no tools
- Max tokens
- 20000
- Reproducibility
- not fully reproducibleHistorical Ollama runtime/profile retained; exact local manifest and full model-blob hashes attested. Upstream weight revision is unknown; full reproducibility is not established.
- Run window (runner clock)
- 2026-08-30T02:21:48+02:00 → 2026-08-30T02:22:03+02:00
Provenance
- Source system
- agi.jetzt v2 (earlier lab, imported)
- Source run
- 2026-08-30_phi4-mini-latest_t06_s1_r2
- Status
- passed the v2 lab’s automated technical gates (“verified”)
- Run manifest (sha256)
- e20a91160ca87b687de05b35c23c907e448de7834de2b4258409fdaf85960bd2
- Artifact (sha256)
- 984721bc7ab2820510345e64cdfa5377416c530d745ad8630dcdea41401ae9dc
- Checks version
- t06-checks-v2
Model The runner’s model identifier maps to Phi-4-mini-instruct in the agi.jetzt register (verified mapping).
What this is evidence for
Not an AGI measureThis run shows one thing: what a model built as a single HTML file from a precise brief — and whether it then followed a precise edit instruction. It says nothing about general intelligence.Checked automatically in this run: 3 of 4 result checks passed · 1 render or interaction check not run.
What the task set is meant to measure
- Artifact creation from a precise brief
- Instruction following
- Static checks of the file (no functional test)
Not in its scope
- General intelligence / AGI
- Agents and tool use
- Multimodal input
- Retrieval
- Conversation
Published and unpublished runs
6 of 16 verified v2 runs of this task are published. 6 of the 6 models with recorded runs have a published run.Source: the selection records of the published runs. They count the task across its variants and know only models with at least one published run.