# W08 Worked Case — From Two Valid Fixtures to One Identification Audit

## Case status

This case uses three separate synthetic objects. L05 is a deterministic intervention-package audit. L06 is a deterministic repeated-measurement panel. The W08 manifest contains a small pair-blocked outcome table for arithmetic practice. Each object is local and reproducible with the Python standard library. None contains observations from a named product, user population, or live intervention.

The case demonstrates a crucial negative result: a valid treatment package and a valid descriptive panel do not become a causal experiment merely because they appear in the same course. Assignment, uptake, units, and outcomes must be joined within one preregistered design. `PAPER-10` and `PAPER-23` inform the design questions; `PAPER-32` remains audit-only. `PLAT-04` illustrates an official measurement interface but contributes no treatment assignment or counterfactual evidence.

## 1. The proposed claim and the required estimand

The defective claim is: “Putting evidence closer to claims improved generative citation by 6.25%.” It hides the treatment version, unit, outcome denominator, comparison, time, system, and assignment. It also uses percent ambiguously and joins unrelated fixtures.

A complete synthetic estimand for the W08 micro-case is:

> Among the eight frozen query-intent clusters in the manifest, at the declared synthetic state, estimate the equal-cluster average effect of assignment to version `T1` rather than `C0` on the post-period binary outcome rate, using pair-blocked assignment and a difference-in-differences estimator with the baseline period as a precision/comparability reference.

The target is finite and synthetic. Each cluster contains twenty binary opportunities per period. Assignment occurs by cluster; the forty rows that would exist inside one cluster are not forty assignment units. The treatment concerns a complete version. The outcome is an unspecified synthetic stage event to prevent accidental interpretation as citation, revenue, or a live engine response.

The observed estimator is not automatically equal to the finite-sample causal estimand. Identification additionally requires implemented pair assignment, treatment uptake, an untreated parallel counterfactual trend, stable measurement, no cross-cluster interference, correct missingness handling, and no unlogged state change.

Potential outcomes clarify the target: each cluster has a post-period outcome under treatment and another under control, but only the assigned one is observed. Pair blocking and randomization within the frozen pairs would support an assignment-based comparison; the notation alone would not. If rollout instead used staggered adoption, the target and assumptions would change to cohort- and timing-specific contrasts.

## 2. Reproduce L05 and state its ceiling

The L05 command audits the checked-in control and treatment HTML/CSS, locked claims, intervention card, and independent equivalence checklist. Its deterministic output is:

`L05 PASS: 3 locked claims, 1 changed factor, 0 error(s)`

The audit record reports three source identities, six equivalence checks, and one factor named `evidence_layout_proximity`. The control and treatment have identical claim text and source sets. The HTML differs only by the stylesheet reference, and the stylesheets differ only in the declared evidence-block rule. Rollback hashes are preserved.

This establishes treatment-version integrity for a local presentation fixture. The intervention card hypothesizes a human-comprehension stage and explicitly says that no crawler, retrieval, ranking, citation, traffic, or business effect is measured. No output response is collected. Therefore L05 cannot supply the outcome in our defective claim.

## 3. Reproduce L06 and state its ceiling

The L06 command analyzes its frozen panel and prints:

`L06 PASS: 360 events, 20 queries, 2 surfaces, 3 blocks`

The panel contains three repetitions per query–surface–block cell. There are 354 complete events and six noncomplete events. Each of six surface–time cells has sixty designed events; complete counts are 58, 59, or 60. Missing outcomes remain empty and visible rather than being coded zero.

L06 reports mention, citation, entailment, absorption, and referral separately. Its Wilson intervals are descriptive and not corrected for repeated-query dependence. It records time-block changes, including a synthetic last-minus-first citation rate of `-0.028736` for `SYN-A` and `-0.024839` for `SYN-B`. These are fixture patterns, not platform drift estimates.

No content treatment is assigned in L06. `SYN-A` and `SYN-B` are teaching surfaces, not treatment arms. Therefore L06 cannot supply a treatment comparison for L05. Joining them after seeing outputs would be outcome-driven design fabrication.

## 4. Draw the broken identification graph

The invalid graph has two disconnected components. In the first, an L05 assignment label leads to a local style version and stops at a structural-audit outcome. In the second, query, synthetic surface, time block, and repetition lead to L06 stage outcomes. There is no edge from the L05 version to any L06 event, no shared unit registry, no assignment field in the panel, and no uptake record.

Adding an arrow in a slide does not create the missing data-generating relationship. A valid joined design would need treatment and control versions assigned to eligible units, event rows keyed to those assignments, timing that respects propagation, a frozen measurement rule, and a control exposed to the same time state. Until then, the strongest conclusion is: L05 is intervention-ready and L06 is measurement-ready as separate synthetic exercises.

## 5. Reproduce the pair-blocked micro-case

The W08 manifest freezes four matched pairs. In each pair, one cluster is assigned treatment and one control. Every cluster has twenty opportunities at baseline and twenty post-period opportunities.

| Pair | Cluster | Assignment | Baseline successes | Post successes | Baseline rate | Post rate | Change |
|---:|---|---|---:|---:|---:|---:|---:|
| 1 | Q-01 | treatment | 8 | 11 | 0.40 | 0.55 | 0.15 |
| 1 | Q-02 | control | 8 | 9 | 0.40 | 0.45 | 0.05 |
| 2 | Q-03 | treatment | 10 | 12 | 0.50 | 0.60 | 0.10 |
| 2 | Q-04 | control | 10 | 11 | 0.50 | 0.55 | 0.05 |
| 3 | Q-05 | treatment | 6 | 9 | 0.30 | 0.45 | 0.15 |
| 3 | Q-06 | control | 6 | 7 | 0.30 | 0.35 | 0.05 |
| 4 | Q-07 | treatment | 12 | 13 | 0.60 | 0.65 | 0.05 |
| 4 | Q-08 | control | 12 | 13 | 0.60 | 0.65 | 0.05 |

Treated cluster changes average to `(0.15 + 0.10 + 0.15 + 0.05) / 4 = 0.1125`. Control cluster changes average to `0.0500`. The difference-in-differences contrast is `0.1125 - 0.0500 = 0.0625`.

This is 0.0625 outcome units, or 6.25 percentage points on the binary-rate scale. It is not a 6.25 percent relative increase. The validator calculates the rates from integer counts, preserves equal cluster weighting, and rejects a changed assignment or total.

## 6. Audit the DAG and identification assumptions

The synthetic DAG includes assignment `D`, uptake `U`, outcome `Y`, pair block `B`, baseline level `X`, time `T`, synthetic state `S`, measurement rule `M`, missingness `R`, and neighboring assignments `D_-i`. Blocking on `B` occurs before assignment. The primary analysis uses assignment, not post-treatment uptake, so it targets an assignment effect.

The central open paths and threats are:

| Threat | Path or failure | Design response | Residual assumption |
|---|---|---|---|
| baseline prognosis | `X → Y` | match/block and randomize within pairs | assignment implemented |
| common time change | `T/S → Y` | concurrent controls and DiD | untreated trends comparable |
| uptake failure | `D → U → Y` absent or variable | log uptake; intent-to-treat primary | assignment affects access to version |
| measurement change | `T → M → Y_observed` | freeze parser and labels | measurement equivalence |
| selection | `D → R ← Y` | preserve all planned cells; sensitivity | missingness policy adequate |
| interference | `D_i → Y_j` | cluster boundary and spillover audit | cross-pair spillover negligible |

The table makes clear that difference-in-differences subtraction does not verify its assumptions. Multiple pre-periods, placebo dates, negative controls, and state logs would strengthen a real design.

This is also a validity separation. Internal validity concerns whether the synthetic contrast can be attributed within these eight clusters under the stated assumptions. External validity concerns transport to other queries, interventions, systems, or dates. Even perfect synthetic assignment would not establish a live-platform effect.

## 7. Power, clustering, and sequential rules

The micro-case has eight assignment units, not 320 independent binary rows. Twenty opportunities within a cluster determine a cluster rate but share intent and assignment. With only four treated and four control clusters, the case is arithmetic instruction, not a well-powered effect study.

A real plan would specify a minimum worthwhile effect, baseline rate, cluster variance, within-cluster correlation, number and size of clusters, missingness, assignment ratio, primary test, guardrails, and simulations. More repeated calls refine within-cluster rates but cannot replace diverse independently assigned intents.

The plan also needs a fixed horizon or a valid sequential rule. Looking after each pair and stopping when the contrast is favorable would change false-positive behavior. Safety and factual-fidelity stops remain active, but efficacy monitoring needs its own predeclared boundary. The micro-case does not provide a p-value or confidence interval because four pairs cannot support the desired external decision.

## 8. Preregistration and deviation log

Before outcomes, freeze version hashes, authorization, unit registry, pair formation, within-pair assignment, estimand, outcome definition, twenty-opportunity cell size, baseline and post windows, missingness, retries, primary DiD estimator, equal-cluster weights, guardrails, stopping, and release rule.

The append-only deviation schema contains `deviation_id`, planned rule, observed departure, detected time, outcome-visibility state, affected clusters, reason, corrective action, analysis consequence, approver, and final status. A propagation delay may trigger a delayed post window. A parser change may require reprocessing every arm from raw artifacts. Control contamination may invalidate a pair. The original preregistration remains unchanged.

## 9. Source-route audit

`PAPER-10` supports examining content-centric units, query clusters, and distinct influence dimensions within its benchmark. It does not make the W08 micro-case a validated source-influence estimator.

`PAPER-23` supports examining an end-to-end reconstructed pipeline and stage-specific outcomes within its environment. It does not establish the hidden behavior of a current commercial platform or transport an effect.

`PAPER-32` is audit-only. Its strategy-learning claims prompt questions about test selection, repeated adaptation, attribution accuracy, run uncertainty, and interference. No reported headline effect is used as a W08 course finding.

`PLAT-04` can define named dashboard counts for its preview and checked date. It provides no assignment mechanism and does not turn an interface trend into the effect of a source edit.

## 10. Bounded conclusion and learner deliverable

A defensible conclusion is:

> L05 reproducibly validates a one-factor, fact-locked local treatment package, and L06 reproducibly validates a 360-event descriptive synthetic panel; they are not one intervention study. In the separate eight-cluster W08 teaching table, the equal-cluster difference-in-differences contrast is 0.0625 outcome units. The arithmetic supports a design exercise only. Causal interpretation remains conditional on assignment integrity, uptake, untreated parallel trends, stable measurement, no interference, and the frozen synthetic state; no live platform or population effect is estimated.

Submit the estimand card, disconnected L05/L06 graph, cluster arithmetic, DAG path table, power limitation, preregistration excerpt, one deviation entry, and the bounded conclusion. Self-check: Did the estimate point to an estimand? Did uncertainty follow the assignment unit? Did a DAG arrow replace evidence? Did a platform count become an outcome without construct validation? Did any causal or transport sentence exceed the synthetic design? Revise until every answer is no.