Claude Fable 5
via claude-code-cli 2.1.214model + harness
Run of 18 Jul 2026
Check results withdrawn (4 Oct 2026)
1 format check passed — no check of the result · 2 render and interaction checks not run
Evaluation not published
Archived pilot · Historical run
Datenviz-Dashboard mit Insights
Data visualizationbuilt by Qwen3.8-27B via ollama 0.32.15 model only
- Automated checks
- Check results withdrawn (4 Oct 2026)1 format check passed — no check of the result2 render and interaction checks not run
- Judge
- Evaluation not published
- Started
- Duration
- 635 s
Check results withdrawn (4 Oct 2026)These check results do not match the screenshots of the same runs. We are therefore withdrawing the results until the check has been repeated in the browser. Affected: three_distinct_visualizations, five_aggregate_matches, three_numeric_insights.
Artifact
Rendered by agi.jetzt from the model-produced artifactRendered offline in an isolated renderer from the stored file (sha256 3d491e1e5407…). The file itself is not served.
Same brief, other models
Scroll sideways. Each run had its own runtime and limits — compare the artifacts, not a score.
Claude Haiku 4.5
via claude-code-cli 2.1.212model + harness
Run of 17 Jul 2026
Check results withdrawn (4 Oct 2026)
1 format check passed — no check of the result · 2 render and interaction checks not run
Evaluation not published
GPT-5.6 Luna
via codex-cli 0.144.5model + harness
Run of 18 Jul 2026
Check results withdrawn (4 Oct 2026)
1 format check passed — no check of the result · 2 render and interaction checks not run
Evaluation not published
GPT-5.6 Sol
via codex-cli 0.144.5model + harness
Run of 17 Jul 2026
Check results withdrawn (4 Oct 2026)
1 format check passed — no check of the result · 2 render and interaction checks not run
Evaluation not published
Automated checks
Automated checks of the stored HTML file, without running it in a browser. Details are the runner’s German originals. Format and presence checks say nothing about whether the result fulfils the brief.
| Check | Kind | Result |
|---|---|---|
| single_html_filegenau eine HTML5-Datei | Format | Passed |
| three_distinct_visualizationsResult withdrawn on 4 Oct 2026; no value is shown. | Content | withdrawn |
| five_aggregate_matchesResult withdrawn on 4 Oct 2026; no value is shown. | Numbers | withdrawn |
| three_numeric_insightsResult withdrawn on 4 Oct 2026; no value is shown. | Content | withdrawn |
| renders_without_errorsSandbox not available — rendering skipped | Rendering | not run |
| no_external_librariesSandbox not available — rendering skipped | Rendering | not run |
2 render and interaction checks were not run. Recorded reason: “Sandbox not available — rendering skipped”.
Evaluation
- Deterministic score
- not stated — check results withdrawn
- Judge
- not published for this run
WhyThe v2 judge record of this run names the tested model as its own judge and lists no criteria. Its score is therefore not shown — nor the combined v2 score, which blends that judge in.
Limits of this run
- 2 render and interaction checks were not run. Whether the artifact works in a browser was not checked automatically.
- No executed check with a shown result evaluates what was built.
- No judge evaluation is published for this run. There is therefore no quality verdict — only the automated checks listed here.
- Not fully reproducible; the recorded reason is listed under “Environment”.
Task
Brief (turn 1)
Unten steht ein CSV-Datensatz (ca. 40 Zeilen). Baue daraus eine einzelne HTML-Datei als beeindruckendes, interaktives Dashboard.
Anforderungen:
- Mindestens 3 Visualisierungen, selbst gerendert als SVG oder Canvas — keine Chart-Libraries, keine externen Assets, keine externen Requests.
- Interaktivität mit Vanilla-JS: beim Hovern über Datenpunkte/Balken/Segmente erscheinen präzise Tooltips mit dem konkreten Wert; Achsen und Legenden sind lesbar beschriftet.
- Eine Sektion „Kernaussagen" mit 3 datengestützten Insights; jeder Insight nennt mindestens eine konkrete Zahl aus den Daten.
- Alle gezeigten Zahlen müssen rechnerisch exakt aus dem Datensatz folgen.
- Gestaltung auf Studio-Niveau: klare Hierarchie, stimmige Farbwelt, ruhige Typografie, sinnvoll gewählte Diagrammtypen für die jeweilige Kennzahl. Responsive, kein horizontaler Overflow.
Datensatz:
[CSV, 36 Datenzeilen: Monats-KPIs eines B2B-SaaS (Juli 2023 bis Juni 2026): MRR, Neukunden, Kuendigungen, Support-Tickets.]From the task seed (German original); the actual prompts add output-format instructions and other wording.
Environment
- Runtime
- ollama 0.32.15
- Execution
- model onlyThe runner reached the model directly — no agent harness, no tools.
- Profile
- std-v1 · T=0.2 · seed 4242 · 1 turn · no tools
- Max tokens
- 16000
- Reproducibility
- not fully reproducibleHistorical Ollama runtime/profile retained; exact local manifest and full model-blob hashes attested. Upstream weight revision is unknown; full reproducibility is not established.
- Run window (runner clock)
- 2026-09-03T02:23:53+02:00 → 2026-09-03T02:34:28+02:00
Provenance
- Source system
- agi.jetzt v2 (earlier lab, imported)
- Source run
- 2026-09-03_qwen3.8-27b_t02_s1_r1
- Status
- passed the v2 lab’s automated technical gates (“verified”)
- Run manifest (sha256)
- 5d3782b69a389998bd80053927503c6fab7f215e700d01f67434d3648194eb4f
- Artifact (sha256)
- 3d491e1e54076f4ab4cc0c588145f722592de80a77e3599737c03db6e5775fa0
- Checks version
- t02-checks-v2
Model The runner’s model identifier maps to Qwen3.8-27B in the agi.jetzt register (verified mapping).
What this is evidence for
Not an AGI measureThis run shows one thing: what a model built as a single HTML file from a precise brief. It says nothing about general intelligence.Checked automatically in this run: Check results withdrawn (4 Oct 2026) · 1 format check passed — no check of the result · 2 render and interaction checks not run.
What the task set is meant to measure
- Artifact creation from a precise brief
- Instruction following
- Static checks of the file (no functional test)
Not in its scope
- General intelligence / AGI
- Agents and tool use
- Multimodal input
- Retrieval
- Conversation
Published and unpublished runs
5 of 16 verified v2 runs of this task are published. 5 of the 6 models with recorded runs have a published run.Not published: Phi-4-mini-instruct (1 run; check results withdrawn).Source: the selection records of the published runs. They count the task across its variants and know only models with at least one published run.