Generated from the same controlled README shipped in the offline package. Source SHA-256: de073f80ce5fcbefdb4f7dbffa53cb8eeca918fcfec69a80c7bdb29688750641.
Linked course weeks: W09–W10
Expected effort for the full assignment: 6 hours
Core path: local, offline, deterministic, Python standard library only
Research question
Can one structural or evidentiary factor change while the factual message remains constant?
Learning objectives
You will learn to:
- express one intervention as a versioned card with authorization, a hypothesized stage, confounds, stop rules, and rollback;
- lock atomic claims and source identities before creating treatment content;
- distinguish factual equivalence from visual or structural equivalence;
- validate basic semantic HTML and accessibility invariants on both arms;
- produce a reversible, hash-addressed before/after package; and
- avoid turning a local mechanism hypothesis into a production visibility claim.
Intervention in the fixture
The control and treatment contain the same three locked claims and the same three source references. The sole intended factor is evidence-layout proximity: the treatment stylesheet renders the unchanged claim/source blocks as bordered evidence cards. The HTML differs only in the stylesheet filename. This is a teaching intervention on a local mirror, not an experiment on a search engine.
data/control.html and data/treatment.html: English semantic HTML with lang, title, one main, one h1, stable data-claim-id attributes, and stable data-source-id links.data/control.css and data/treatment.css: the declared presentation arms.data/locked_claims.json: canonical claim text and source IDs.data/intervention_card.json: follows ../shared/schemas/intervention-card.schema.json and is restricted to local_fixture or authorized owned staging.data/factual_equivalence.csv: predeclared review checks. Every core criterion must be pass, and the reviewer must differ from the experimenter.
Run
python3 scripts/audit_intervention.py \
--control data/control.html \
--treatment data/treatment.html \
--claims data/locked_claims.json \
--card data/intervention_card.json \
--checklist data/factual_equivalence.csv \
--output /tmp/geo-l05
Output contract
Table from the controlled L05 lab handout| Artifact | Contract |
|---|
intervention_audit.json | contract status, equivalence/accessibility checks, changed-factor count, and claim ceiling |
normalized_claims.json | claims and source IDs extracted independently from both pages |
before_after.diff | unified HTML and CSS diff; never a causal estimate |
equivalence_review.csv | frozen reviewer checks copied into the release package |
rollback_manifest.json | exact treatment/control hashes and reversible replacement instruction |
run_manifest.json | hashes of every input and declared parameter |
A successful software audit establishes exact equality of the locked text/source set and declared structural scope. It does not prove that a user, crawler, retriever, answer engine, or ranking system reacted to the change.
Full assignment procedure
- Select one structural or evidentiary factor and name the hypothesized stage.
- Freeze atomic claims, source identities, control, expected signal, non-claims, and stop rules before treatment construction.
- Build both arms only on a local mirror or explicitly authorized owned environment.
- Validate HTML semantics, keyboard/focus behavior, contrast, link purpose, and responsive reading order.
- Have an independent reviewer compare every factual claim and attribution.
- Run this auditor, inspect the complete diff, and eliminate undeclared differences.
- Execute rollback and verify the restored hash before any optional collection.
- Report null, adverse, and unintended accessibility effects alongside the target result.
Low-compute and no-network route
The fixture is static HTML/CSS. The auditor uses html.parser, hashing, CSV/JSON, and unified diff from the standard library. No browser, model, crawler, API, external package, or live endpoint is required.
Negative results and interpretation
- No measured change is a valid null result; do not add factors after seeing it.
- A treatment that changes a claim, number, source identity, or attribution fails factual equivalence even if the message “feels similar.”
- An accessibility regression invalidates release regardless of a target metric.
- A local effect cannot establish cross-engine, cross-query, longitudinal, business, or causal generality.
- An extra undeclared DOM/CSS change is a confound, not polish.
Ethics and stop conditions
No cloaking, deceptive attribution, hidden user-specific variants, live poisoning, unauthorized traffic, or fabricated evidence is permitted. Stop on authorization ambiguity, factual drift, broken source identity, accessibility regression, privacy exposure, or rollback failure.
Acceptance
python3 scripts/audit_intervention.py --control data/control.html --treatment data/treatment.html --claims data/locked_claims.json --card data/intervention_card.json --checklist data/factual_equivalence.csv --output /tmp/geo-l05
python3 -c 'import json; r=json.load(open("/tmp/geo-l05/intervention_audit.json")); assert r["status"] == "PASS" and r["changed_factor_count"] == 1'
See RUBRIC.md.