Generated from the same controlled README shipped in the offline package. Source SHA-256: 6ccab276a00592737b7bece10a6ce7c34c117b72706c983c5eb9b71c0a1c21df.
Linked course week: W16
Expected effort: 18–30 hours across the course
Core path: self-contained synthetic dossier, offline, deterministic, Python standard library only
Research question
Can an independent reviewer trace, reproduce, challenge, and revise every headline claim?
Learning objectives
You will learn to:
- connect every public claim to data, code, result, figure, limitation, and provenance artifacts;
- reproduce a primary result from a frozen input and compare it to a declared expected result;
- verify checksums, authorization, license, privacy, redistribution, and figure accessibility;
- preserve protocol deviations and negative results in the release rather than only the successful headline;
- distinguish software/package readiness from scientific, safety, legal, editorial, and publication approval; and
- produce a reviewer-oriented release that can be challenged and revised.
Bundled dossier
data/dossier/ is a complete synthetic release fixture. Its primary result is a transparent 12-event difference in proportions. A secondary citation-correctness result moves in the adverse direction and remains in the claim ledger and negative-results record. Neither value is evidence about a real intervention or answer engine.
The bundle contains:
- release README and research brief;
- pre-analysis plan and frozen observations;
- analysis script, expected result, environment lock, and data dictionary;
- claim ledger and artifact manifest;
- SVG figure with source mapping, caption, title/description, and alt text;
- protocol deviations, limitations, ethics/privacy/license note, negative-results record, and response note;
- release checklist/manifest and SHA-256 checksum inventory.
Run
python3 data/dossier/analysis.py \
--input data/dossier/observations.csv \
--output /tmp/geo-l08-primary-result.json
python3 scripts/validate_release.py \
--dossier data/dossier \
--analysis-result /tmp/geo-l08-primary-result.json \
--output /tmp/geo-l08
release_manifest.json declares the dossier ID, required paths, primary claim IDs, human gates, and analysis contract.artifact_manifest.csv follows ../shared/schemas/release-artifact.schema.json; every artifact has stable ID, path, hash, role, redistribution status, license, authorization, and personal-data flag.claim_ledger.csv records evidence status, semicolon-delimited artifact edges, claim ceiling, and public-headline status.checksums.sha256 covers every required file except itself.- The supplied analysis result must byte-semantically match
expected_result.json after JSON parsing.
Output contract
Table from the controlled L08 lab handout| Artifact | Contract |
|---|
release_audit.json | contract status, exact counts, reproducibility comparison, errors, and bounded decision |
claim_traceability.csv | one row per claim–artifact edge with path, role, and integrity result |
computed_checksums.sha256 | independently recomputed hashes for required dossier paths |
reviewer_instructions.md | minimal reproduction and challenge sequence plus remaining human gates |
run_manifest.json | hashes of the release inputs and generated result |
READY_FOR_HUMAN_REVIEW means only that the synthetic package passed the automated dossier contract. It does not mean accepted, replicated, safe, lawful, accessible in all environments, publishable, or ready for public deployment.
Full assignment procedure
- Freeze an approved question and pre-analysis plan before outcome inspection.
- Complete upstream labs and assign stable IDs to data, code, results, figures, limitations, and decisions.
- Link every headline claim to sufficient artifacts; remove or weaken claims with broken edges.
- Reproduce the primary result in a clean environment and retain the exact command/output.
- Audit missingness, deviations, negative results, alternative analyses, and figure-data consistency.
- Verify privacy, authorization, license, redistribution, accessibility, and source/version refresh.
- Generate checksums last and validate the release from a clean copy.
- Defend for 8 minutes, answer questions for 7 minutes, then submit a response/revision within 48 hours.
Low-compute and no-network route
The sample result is computed from 12 CSV rows in milliseconds. Validation uses JSON/CSV, hashing, and path checks only. No notebook server, model, package install, API, browser, account, or network access is required.
Negative results and interpretation
- A broken claim-to-artifact edge blocks the claim even when other checks pass.
- A checksum mismatch is an integrity failure, not a harmless formatting change.
- A non-redistributable dependency requires lawful retrieval instructions and may prevent full reproduction.
- An adverse secondary metric remains part of the release.
- Reproducing arithmetic does not independently validate data generation, construct validity, or generalization.
- An elegant primary figure cannot override privacy, license, safety, authorization, or accessibility failure.
Ethics and release rule
Do not include personal data, confidential records, unauthorized screenshots, copyrighted snapshots without permission, deceptive attribution, or live harmful tests. Any unresolved authorization, privacy, licensing, safety, accessibility, or evidence-lineage issue blocks external release regardless of result strength.
Acceptance
python3 data/dossier/analysis.py --input data/dossier/observations.csv --output /tmp/geo-l08-primary-result.json
python3 scripts/validate_release.py --dossier data/dossier --analysis-result /tmp/geo-l08-primary-result.json --output /tmp/geo-l08
python3 -c 'import json; a=json.load(open("/tmp/geo-l08/release_audit.json")); assert a["status"] == "PASS" and a["decision"] == "READY_FOR_HUMAN_REVIEW"'
See RUBRIC.md.