# W16 Lecture Notes — Capstone synthesis and defense

Essential question: **What would make the result worth trusting?**  
Phase: **Validate**  
Capstone commitment: **18–30 hours total**  
Linked laboratory: **L08, Capstone evidence release**

## 1. Trust is a traversable release, not a polished conclusion

A capstone is worth trusting when a skeptical reviewer can identify what was asked, what was measured, which evidence was authorized, how observations became results, how results became claims, which limitations remain, and what changed after review. None of these conditions requires the reviewer to agree with the conclusion. They require the author to expose the path by which the conclusion was bounded.

The minimal path is a traceability graph:

> question → estimand → source or data manifest → code → observations → result → figure or table → claim → limitation → decision → release identity

The arrows are evidence obligations. If the claim changes from “the treatment arm had a larger descriptive rate in this fixture” to “the intervention improves production GEO visibility,” the graph must change. The latter needs a production context, a visibility outcome, a defensible comparison, and a causal identification argument. An attractive bar chart cannot create those missing nodes.

Trust also has a governance dimension. Authorized collection, privacy, license, safety, and accessibility are hard gates. A mathematically correct result produced from unauthorized or unlawfully redistributed material is not releasable. A perfectly licensed dataset does not make an unsupported causal claim valid. Evidence integrity and governance must both pass; neither compensates for the other.

L08 makes this visible with a deliberately small synthetic dossier. Its automated `PASS` routes the dossier to `READY_FOR_HUMAN_REVIEW`. That is not a scientific verdict. It says required files exist, checksums match, declared edges resolve, and frozen arithmetic reproduces. Local PASS is not human review, independent reproduction, accessibility conformance, a pilot, grading calibration, external review, or a recording.

## 2. Freeze the question, estimand, claim, audience, and decision

Before packaging, write five sentences.

1. **Question:** What comparison or description is being asked?
2. **Estimand:** What exact quantity answers that question for which units, condition, and time?
3. **Claim:** What sentence will the release permit a reader to repeat?
4. **Audience:** Who may interpret or act on it?
5. **Decision:** What will the team do if the result is positive, null, adverse, or non-reproducible?

For the L08 teaching fixture, the primary estimand is the difference in the fraction of events with `outcome=1` between six treatment and six control events. The bounded headline is `CAP-001`: treatment has four outcomes in six events, control has two in six, and the descriptive difference is (4/6-2/6=1/3). The audience is a reviewer learning release mechanics. The decision is only to route a structurally valid dossier to human review.

That sentence deliberately omits “causes,” “improves production,” “GEO visibility,” and “generalizes.” Those terms would require evidence the fixture does not contain. `CAP-003` preserves the rejected sentence “the intervention improves production GEO visibility.” Keeping the rejected claim prevents a later editor from unknowingly reviving it.

The freeze is versioned, not eternal. Evidence may force a revision, but every change must be documented. A post-defense revision can strengthen, retain, weaken, or remove a claim. It cannot silently redefine the estimand. If the author genuinely changes the question, the release notes identify a new analysis rather than presenting the change as a typo.

## 3. Build a claim-to-artifact traceability matrix

Use stable identifiers because prose labels drift. In L08, `CAP-001` links to `DATA-001`, `CODE-001`, `RESULT-001`, `FIG-001`, and `LIMIT-001`. `CAP-002` links to `DATA-001`, `CODE-001`, `RESULT-001`, `NEG-001`, and `LIMIT-001`. `CAP-003` links to `NEG-001` and `LIMIT-001` because it is rejected rather than supported. Across the three rows, there are twelve declared edges.

A useful matrix has claims as rows and at least these columns:

| Column | Reviewer question |
|---|---|
| Source/data | Which authorized observations bear on this claim? |
| Code | What deterministic transformation was applied? |
| Result | Where is the exact machine-readable value? |
| Figure/table | What human-facing representation carries it? |
| Limitation | Which scope ceiling travels with the claim? |
| Reviewer finding | Who challenged which edge? |
| Revision | Was the claim retained, weakened, or removed? |
| Checksum/release | Which immutable bytes and version contain it? |

Do not mark every cell “yes.” A rejected claim may correctly lack a supportive result or figure. The validation rule is semantic: every claim needs the artifacts required by its status. A public supported headline needs data, code, result, representation, and limitation. A rejected claim needs a durable rejection rationale and limitation route. “Not applicable” must explain why.

The matrix is a view, not the sole source of truth. Artifact manifests record paths, media types, licenses, privacy class, redistributability, and hashes. The claim ledger records wording, status, publicity, evidence IDs, and ceiling. The release validator joins these structures and fails if a reference is missing or ambiguous.

## 4. Reproduce the L08 primary result without inflating it

The L08 `observations.csv` contains twelve synthetic rows. Control events E1–E6 have two positive outcomes; treatment events E7–E12 have four. The standard-library analysis groups rows, counts successes and totals, computes rates, and subtracts control from treatment.

| Arm | Positive outcomes | Total | Rate |
|---|---:|---:|---:|
| Control | 2 | 6 | 0.333333 |
| Treatment | 4 | 6 | 0.666667 |
| Difference | — | — | 0.333333 |

The output matches `expected_result.json`. That match has three valuable meanings. First, the bytes used by the validator produce the declared arithmetic. Second, another clean process can regenerate the same output from the bundled inputs. Third, the analysis is simple enough to inspect manually.

It does not mean that randomization was adequate, observations represent a target population, measurement was valid, uncertainty is small, or a causal effect exists. There is no inferential interval in this fixture and no production sampling frame. The claim ceiling therefore reads “descriptive arithmetic for a deterministic synthetic fixture only.”

The figure is one representation of the result, not an independent evidentiary source. `FIG-001` must declare `DATA-001` and `RESULT-001` as sources, carry a caption, and expose SVG title and description. If the figure displays a number that differs from the machine result, the release fails even if the underlying JSON is correct. A figure and its text-table equivalent should encode the same quantities.

## 5. Retain adverse, null, negative, and rejected results

The secondary citation-correctness measure runs in the opposite direction. Control has five correct events in six, while treatment has four in six. The descriptive treatment-minus-control difference is (-1/6), approximately (-0.166667). L08 records this as `CAP-002` and links `NEG-001` rather than hiding it below the primary figure.

Why call it a negative result? Not because it is statistically significant; no such inference is provided. It is negative relative to a direction a naive promotional narrative might prefer. Its presence constrains interpretation of `CAP-001`. An author cannot portray the fixture as uniformly favorable when a declared secondary measure is adverse.

Null findings deserve the same preservation. A null can expose an insensitive measure, insufficient precision, a genuinely small difference, or a failed intervention. The release should state which interpretations are compatible with the design. A failed reproduction belongs in the history as well: environment, command, observed error, resolution if any, and whether the primary claim remains reproducible.

`CAP-003` is equally important. It is a rejected claim, not a deleted drafting accident. It demonstrates that a traceability ledger includes ideas the evidence did not earn. In a defense, the most trustworthy answer can be “we removed that sentence because no artifact could support the production and causal scope.”

## 6. Preserve exclusions, deviations, and failed reproductions

A clean release is not one with no problems. It is one in which problems are visible, classified, and linked to their consequences. Maintain three separate records:

- **Exclusions:** units, queries, sources, or outcomes omitted under a declared rule, with counts and reasons.
- **Protocol deviations:** departures from the pre-analysis plan, with date, trigger, decision maker, affected estimand, and consequence.
- **Reproduction failures:** attempts that did not regenerate the declared output, with environment and resolution status.

L08’s deviation log states that no post-plan row, outcome, or analysis change occurred. The release validator was added later as a packaging control and did not alter the estimand. That is a small deviation, but documenting it teaches a crucial distinction: adding an integrity check is not the same as changing the analysis, yet both belong in provenance.

Use append-only entries or explicit version histories. Do not overwrite “failed on Python X due to locale handling” with “fixed.” Preserve both the failure and the fix. A reviewer needs to see whether the failure reveals fragility. A deviation can require a sensitivity analysis, a weaker claim, a new version, or a blocked release. The decision rule should be written before the defense so social pressure cannot redefine materiality.

## 7. Separate redistributable reproduction from bounded verification

The L08 bundle is fully redistributable: synthetic course-owned inputs, code, manifests, documentation, and an SVG are included under the recorded license route. A reviewer can run the analysis and release validator without network access. This is the cleanest instructional route.

Real capstones often depend on material that cannot be placed in a public bundle: licensed databases, protected personal data, confidential records, controlled platform exports, or copyrighted corpora. “Cannot redistribute” is not permission to become vague. Create a non-redistributable dependency card with:

- stable dependency ID and role;
- issuer, title, version, date, and exact locator;
- lawful acquisition or access procedure;
- authorization and license conditions;
- privacy and security classification;
- local checksum and checksum algorithm;
- authorized reviewer or review role;
- verification command or comparison protocol;
- what can and cannot be independently regenerated; and
- expiry, revocation, or refresh condition.

Never place restricted bytes, tokens, account state, or credentials in the public release. If an authorized reviewer lawfully acquires the exact version, matches the recorded checksum, and reruns the analysis, independent reproduction may be possible. If the reviewer can inspect only supplied outputs, label the outcome “verifiable output review” or “partial verification.” If neither route is available, mark the claim unreproduced and decide whether it must be weakened or blocked.

The dependency boundary is itself reviewable. A checksum identifies bytes but does not prove lawful access, accuracy, completeness, or fitness for the estimand. Retrieval instructions can become stale. The release notes therefore identify the last successful authorized retrieval and the person or role responsible for refreshing it.

## 8. Treat governance and accessibility as hard release gates

Before public release, answer five gate questions.

**Authorization:** Was every collection, account, system, participant interaction, and reviewer access allowed for the declared purpose? **Privacy:** Are personal and sensitive data minimized, protected, documented, and released only under a lawful, reviewed basis? **License:** May the team use, transform, quote, display, and redistribute each artifact as proposed? **Safety:** Could disclosure, code, instructions, or findings create disproportionate harm, abuse, or operational risk? **Accessibility:** Can people using relevant assistive technologies perceive, navigate, and understand the material outputs?

These are conjunctive gates. A weighted score must not allow high license confidence to offset a privacy failure. Use states such as `pass`, `blocked`, `not_applicable_with_reason`, and `pending_named_review`. A pending material gate means no public release.

Automated checks can support, not replace, human decisions. A script can detect missing alt text, invalid heading structure, a path outside the release, or a license field left blank. It cannot determine whether an alt description communicates the analytical point, whether a privacy risk is acceptable, whether authorization covers a new use, or whether disclosure is safe in context.

The L08 audit therefore carries five human gates: independent scientific review; privacy authorization and license review; human figure and document accessibility review; the eight-minute defense, seven-minute questions, and forty-eight-hour response; and external publication or deployment approval. They remain pending after automated PASS.

## 9. Use checksums, environments, and release notes correctly

A SHA-256 checksum answers a narrow question: do these bytes match the bytes that were hashed? It is an identity control, not a truth score. In L08, eighteen required artifacts are verified against `checksums.sha256`; the checksum file is not self-hashed in the same list. The validator also records its input identities and produces `computed_checksums.sha256` for review.

The release needs a version, immutable identity, environment declaration, command sequence, and output inventory. The environment file should be minimal and exact enough to recreate execution. Network requirements, locale, random seeds, and external services must be declared. A deterministic fixture should not silently fetch changing content.

After any material revision, regenerate analysis outputs, figures, trace exports, checksums, run manifest, and release notes. Release notes answer:

- what changed and why;
- which claims and artifacts were affected;
- whether the estimand or data changed;
- which reviewer finding triggered the change;
- whether results changed;
- which gates were rerun; and
- what remains pending.

Never edit a file after hashing and still call it the same release. A stale checksum is useful evidence that identity has broken. Do not “fix” it by updating only the checksum; determine whether dependent results, figures, limitations, and claims also need regeneration.

Add a **release-delta audit** before assigning the new version. Begin with a byte-level inventory, then interpret changes through the traceability graph. A changed CSV is material even when the headline number stays the same, because exclusions, secondary outcomes, or future re-analysis may differ. A changed caption may be material when it modifies scope. A formatting-only change can still affect accessibility, pagination, reading order, or checksum identity. The author records the reason rather than guessing materiality from file size.

A compact delta table contains old hash, new hash, change class, triggering reviewer finding, affected claim IDs, required reruns, and reviewer status. “No numeric change” is not the same as “no evidentiary change.” For example, elevating `CAP-002` beside the primary result changes how a reader interprets the release even though `RESULT-001` remains identical. The research brief, figure caption or adjacent text alternative, claim ledger publicity field, response note, release notes, and all associated checksums may need updates.

Keep three identities distinct. The **analysis run identity** says which input and code generated a result. The **evidence release identity** says which complete set of files, gates, and claims was issued. The **review decision identity** says who reviewed which release under what authority and date. Copying an approval from version 0.1 to version 0.2 without checking affected artifacts is invalid. When a change touches a hard gate, the responsible human reviewer must reconsider that gate rather than letting a script inherit the prior state.

## 10. Design the eight-minute presentation and seven-minute defense

The presentation is an index into the evidence release, not a substitute for it. A disciplined eight-minute plan is:

1. **One minute:** question, estimand, audience, and decision.
2. **Two minutes:** design, sampling or query frame, and authorized evidence.
3. **Two minutes:** primary result, uncertainty, and one adverse/null/negative result.
4. **Two minutes:** traceability route, dependency boundary, and hard gates.
5. **One minute:** bounded conclusion, release status, and strongest limitation.

The seven-minute defense should target the weakest edge. Productive questions include: Which observation could most change the conclusion? Why is the claim ceiling not broader? What did you exclude? Which deviation was material? Can a reviewer reproduce without your working directory? What happens if the controlled dependency is unavailable? Which accessibility check requires a human? What evidence would make you remove the headline?

An honest answer can defer to an artifact: “`CAP-002` routes to `NEG-001`; the adverse secondary difference is preserved there.” It can also admit a gap: “The authorized reviewer has not yet obtained the dependency, so independent reproduction is pending and the release is blocked.” Defense quality is demonstrated by accurate routing and proportionate revision, not rhetorical certainty.

The current package contains no live defense result. The timings are instructional planning budgets. There is no pilot, grading calibration, external review, or recording from which to infer that eight plus seven minutes is sufficient for every project.

## 11. Revise within forty-eight hours and answer reviewers visibly

Within forty-eight hours of the defense, produce a response-to-reviewers table. Each row contains a reviewer finding ID, challenged claim or artifact IDs, the reviewer’s concern, the author’s decision, the change, the evidence route, and the regenerated release version. Decisions are:

- **Accept:** the evidence supports the concern; revise as requested or more strongly.
- **Partly accept:** the concern is valid but the proposed remedy exceeds the evidence; explain the bounded remedy.
- **Decline:** preserve the challenge and provide a specific evidentiary reason, not preference or authority.
- **Cannot resolve:** block or weaken the affected claim and name the missing evidence.

Suppose a reviewer challenges the phrase “improves production GEO visibility.” The correct response is accept and remove it, because `CAP-003` has no production or causal artifact. Replace it with the `CAP-001` descriptive sentence. If the reviewer notes the adverse citation-correctness result was visually secondary, accept and add a co-equal text row linked to `CAP-002` and `NEG-001`. If a human accessibility reviewer is still pending, do not write “accessible”; retain the gate.

Revision is evidence-led. It may make the story less dramatic and the release more trustworthy. Preserve the original version, reviewer findings, and change history. Regenerate all downstream artifacts and checksums. The response note should allow a third party to compare what was challenged, what changed, and why.

## 12. Interpret L08 PASS and make the final bounded decision

In a clean temporary run, L08 reports twelve events and two metrics, followed by: three claims, sixteen artifacts, analysis reproduced, and decision `READY_FOR_HUMAN_REVIEW`. Its audit records one public headline, twelve claim–artifact edges, one figure, nineteen required files, eighteen verified checksums, and no machine errors. It emits five review artifacts: `release_audit.json`, `claim_traceability.csv`, `computed_checksums.sha256`, `reviewer_instructions.md`, and `run_manifest.json`.

This is strong local integrity evidence. It shows that the declared fixture, analysis, manifests, and checks agree at the time of validation. It also exposes exactly what it does not show. The claim ceiling says automated dossier integrity only—not independent replication, scientific acceptance, legal approval, accessibility conformance, publication, or deployment readiness.

The capstone decision therefore has two layers:

1. **Automated integrity decision:** fail, or route to human review.
2. **Human release decision:** block, revise and re-review, approve a bounded teaching release, or make a separate publication/deployment determination.

For L08, only the first route is complete. `CAP-001` is ready to be questioned within its descriptive ceiling. `CAP-002` remains visible as an adverse secondary result. `CAP-003` remains rejected. The five human gates remain open. No actual eight-minute presentation, seven-minute defense, or forty-eight-hour revision is claimed.

The final test is simple to state and demanding to satisfy: Can a skeptical, authorized reviewer traverse every public claim to the correct artifacts, reproduce or precisely delimit what cannot be reproduced, see negative results and deviations, understand every gate, and observe evidence-led revision? If not, the release is not ready. If yes, it is ready for judgment—not guaranteed agreement, acceptance, conformity, safety, accessibility, publication, or production effect.