The Lab Notes

Small experiments on AI models and the tools that run them — findings in the notes, most with a companion investigation holding the method and raw data, published as a starting point for further research.

12 notes, most recent first

Two viral prompt wrappers, tested: no confirmatory gain over a plain control

August 29, 2026|11 min read

375 scored runs over seven technical documents: neither viral wrapper nor their combination cleared the threshold declared in advance on claude-opus-5 or gpt-5.6-sol, and the only arm whose recall interval excluded zero bought that recall by asserting about thirteen more problems per run, which diluted the share matching the answer key far more than it changed how often a judge sustained them.

Method and raw data: Two Viral Prompt Wrappers: Method, Arms and the Full Grid

Everyone Found the Answer

August 15, 2026|3 min read

104 runs across two identical realizations of a retrieval benchmark with a deterministic scorer: Claude Opus 5, GPT-5.6 Sol, Grok 4.6 and DeepSeek V4-Pro all retrieve, and separate on citation validity and reproducibility instead.

Method and raw data: Four Models on a Retrieval Contract: Method, Both Realizations, and What the Instrument Cannot Say

The pilot got both signs backwards, and explained them anyway

August 13, 2026|5 min read

A 30 question pilot reported Anglish at plus 9.4 points and Latinate at plus 6.6, then attached causes and user advice. The 250 question study returned minus 2.5 and minus 3.7.

Thirty Fresh Writers and an Empty Log

August 13, 2026|5 min read

An experiment whose entire premise is a controlled information boundary shipped a finished manuscript and left the one instrument that would measure the boundary with no records in it.

Which Model Actually Ran

August 11, 2026|3 min read

The opus CLI alias resolved to the previous model on the day Opus 5 shipped, so a benchmark that trusts the alias silently tests the wrong thing. Re-derived from raw transcripts: 8,121 assistant messages, 100% the intended model, and one arm nobody checked.

Method and raw data: Model Identity Verification: Methodology and Raw Data

What Replicated, and What Did Not

August 11, 2026|3 min read

Thirteen cells, 468 runs, 43 minutes and $3.03: the Hermes optimisation batch reproduces at close to published magnitudes on weak models, attenuates rather than vanishes on a strong one, does nothing for DeepSeek, and carries a runaway-generation tail.

Method and raw data: Harness Replication: Method, Thirteen Cells and the Artefacts Caught

There Is No Setting

August 11, 2026|3 min read

A rerun of a suspiciously expensive DeepSeek V4 Flash arm reproduced its token volume to within 0.4%, with 99.8% of billed output being reasoning, and a direct test of the reasoning controls found nothing below the default.

Method and raw data: The Reasoning Floor: Methodology, Cell Data and Control Probe

The Eighty Percent That Did Not Transfer

August 11, 2026|3 min read

Measuring the always-loaded surface of a working agent config stack against a vendor's 80% trim: 22,400 tokens before the first word, 31% of defensible cuts, and a taxonomy for which text a stronger model makes redundant.

Method and raw data: Prompt Stack Trim: Method, Taxonomy and the Ranked Proposal

Local Speech to Text Reaches Parity

August 11, 2026|3 min read

Voxtral Mini 3B and Parakeet TDT 0.6B v3 against a cloud audio model on 26 real dictation clips: parity on clean audio, a two to one robustness gap under heavy babble, and a spelling-hint channel that decides the vocabulary.

Method and raw data: Speech to Text Bake-off: Methodology, Reference Design and Full Tables

Cheap Tokens, Expensive Answers

August 11, 2026|3 min read

Twelve seeded positions, five arms, $0.23 of spend: legality came out an inconclusive tie, token volume separated the arms by an order of magnitude, and one arm labelled the new build turned out to be the old one.

Method and raw data: DeepSeek V4 Flash against GPT-5.6 Luna: Methodology, Results and Retraction

A Cheap Model Against a Regex

August 11, 2026|3 min read

GPT-5.6 Luna and GPT-5.3 Spark at effort low, given a plain-language prompt, against hand-written keyword and regex classifiers on four real classification sites, independently adjudicated, with a split verdict and a batching correction.

Method and raw data: Semantic against Keyword: Method, Adjudication and Per-site Verdicts

What a Cached Token Actually Costs

August 6, 2026|4 min read

A single-window experiment measuring how heavily cached reads are weighted against the Claude Code subscription session meter: r = 0.008, 95% CI [-0.004, 0.021], with the 0.1x and 1.0x hypotheses both refuted.

Method and raw data: Cache-Read Coefficient: Methodology and Raw Data