Practice map
Eight offline-first research labs
Each lab produces an inspectable artifact and includes frozen inputs, a reproducible core route, validation, rubric, stop conditions, and an equivalent low-compute path. Live services are never required for core completion.
- Packages
- L01–L08
- Core environment
- Python 3.10+ / static files
- External accounts
- None required
Dependency graph
Artifacts accumulate into the capstone
L01L02
→L03L04
→L05L06
→L07
→L08
The graph is instructional rather than a software dependency lock: L01/L02 establish scope and sampling; L03/L04 establish system and claim diagnostics; L05/L06 establish intervention and measurement; L07 gates safety/governance; L08 releases the complete evidence chain.
Lab map
Questions, files, and pass gates
| Lab | Research question | Weeks / effort | Expected files | Pass gate |
|---|---|---|---|---|
| L01Surface & source audit | What can a second analyst reproduce from one answer-surface observation? | W01–W024 hours | system-card.json · observations.csv · source-inventory.csv+ 2 specified files | A second analyst can reconstruct the observation context and locate every visible source; no hidden-stage claim is presented as observation.Brief, rubric, package → |
| L02Query-set engineering | Does the query bank represent a declared universe rather than a desired result? | W065 hours | queries.csv · codebook.md · sampling-rationale.md+ 2 specified files | Every query has stratum/provenance; duplicates and desired-answer leakage are resolved; second-language extensions remain separately analyzable.Brief, rubric, package → |
| L03Retrieval & reranking sandbox | Which stage changes when retrieval quality changes? | W03–W047 hours | runs/*.trec · metrics.csv · ablation.csv+ 3 specified files | Metrics reproduce within declared tolerance and candidate-generation effects are never described as generation or citation effects.Brief, rubric, package → |
| L04Claim–Evidence–Source graph | Which claims are supported, contradicted, merely cited, or source-distinctively absorbed? | W02 & W056 hours | claims.csv · evidence.csv · edges.csv+ 3 specified files | Every public claim has a resolvable evidence edge or explicit unsupported status, and agreement/adjudication is reported.Brief, rubric, package → |
| L05Controlled content intervention | Can one structural or evidentiary factor change while the factual message remains constant? | W09–W106 hours | control/ · treatment/ · intervention-card.yaml+ 3 specified files | One interpretable factor changes; factual equivalence and reversibility pass independent review; no deceptive attribution is introduced.Brief, rubric, package → |
| L06Cross-surface & repeated-run measurement | Is an observed difference larger than repeated-sampling variation and time drift? | W07 & W138 hours | panel.csv · data-dictionary.md · events.jsonl+ 3 specified files | Stage metrics remain separate, denominators/missingness are explicit, and the conclusion is supported by repeated observations rather than a screenshot.Brief, rubric, package → |
| L07Drift, adversarial & governance review | What can fail, who can be harmed, and what blocks release? | W14–W156 hours | threat-model.svg · abuse-cases.csv · control-tests/+ 3 specified files | High-risk tactics cannot pass release; legal/standards status is accurately labeled; residual risk is explicit.Brief, rubric, package → |
| L08Capstone evidence release | Can an independent reviewer trace, reproduce, challenge, and revise every headline claim? | W1618–30 hours across the course | README.md · research-brief.md · claim-ledger.csv+ 7 specified files | No broken claim-to-artifact edge; a reviewer reproduces the primary result or verifies the documented non-redistributable dependency boundary.Brief, rubric, package → |
Environment contract
One predictable local workflow
- Inspect.Read README, data manifest, license/ethics note, expected outputs, and stop conditions before executing anything.
- Validate inputs.Run the package smoke test and checksum/schema validation. Do not repair a mismatch silently.
- Run or audit.Use the executable route when available or the supplied-output route when compute/dependency limits apply.
- Review and release.Run checks, inspect differences, write limitations, record deviations, and version the artifact bundle.