# W03 Slide Script — Discovery, Indexing, and Retrieval ## Production conventions - **Format:** 16:9; deep purple, warm ivory, graphite, and restrained gold. Color is never the only carrier of state. - **Typography:** minimum 30 pt body and 44 pt title. Formulas receive both symbolic and plain-language readings. - **Visual authorship:** every visual below is an original instructional diagram. Do not paste a search-product interface, paper figure, or repository screenshot. - **Boundary mark:** an open solid outline means inspectable; a hatched outline means externally unobserved; a dashed arrow means a hypothesis rather than an observed transition. - **Accessible production:** retain the exact alt text in speaker materials, keep table equivalents, and avoid animation-dependent meaning. ## Slide 01 — Candidate access before optimization **On-screen text** > W03 · Discovery, indexing, and retrieval > Essential question: Can the system obtain the source at all? Footer: “Eligible ≠ indexed ≠ retrieved ≠ used.” **Visual specification** An ivory canvas with a source card at left and an answer card at right. Between them sit five gates. Only the first gate is open; the remaining gates are unlabeled and hatched. A gold question mark floats above the path. The cards resemble no commercial interface. **Speaker notes** Begin with a page that is publicly reachable. Ask whether reachability proves it can appear in one generated answer. Introduce the week as a study of candidate access, not a collection of ranking tactics. State that the five gates are an analytical scaffold, not a proprietary platform diagram. **Teaching check** Learners write two distinct events that the word “visible” might hide. **Alt text** “A source and an answer are separated by five gates; only the first is visibly open, so downstream access remains unknown.” ## Slide 02 — Outcomes and evidence artifacts **On-screen text** By the end, you can: 1. Separate discover, fetch, represent, index, retrieve, and select. 2. Build an inverted-index trace. 3. Calculate a declared BM25 comparison. 4. Report Recall@k with its denominator. 5. Bound open-sandbox and closed-platform claims. **Visual specification** Five stacked outcome cards, each paired with a small artifact icon: stage map, postings list, score sheet, metric card, and boundary memo. Numbers and labels, not icons, establish the order. **Speaker notes** Explain that each outcome leaves inspectable learner evidence. The goal is not memorizing a formula in isolation. The formula becomes useful when it is attached to an immutable corpus, tokenizer, index snapshot, qrels, cutoff, and claim boundary. **Teaching check** Readiness poll: who can define a qrel and who needs the bridge route? **Alt text** “Five numbered outcomes align with five outputs: a stage map, postings list, score sheet, metric card, and boundary memo.” ## Slide 03 — One funnel, many denominators **On-screen text** | Boundary | Example count | |---|---:| | Declared URLs | 1,000 | | Discovered | 760 | | Requested | 700 | | Usable responses | 650 | | Parsed representations | 610 | | Indexed | 570 | | Retrieved for one query | 100 | | Selected passages | 8 | Caption: “Every ratio needs a question.” **Visual specification** A stepped funnel with the same eight counts. Every step has a bracket labeled with its possible conditioning set. The final two steps switch from corpus-level rectangles to a query-labeled branch, making the denominator change visible. **Speaker notes** Walk through why 650/700 is a fetch-success quantity while 570/610 is an admission quantity. The 100 retrieved candidates belong to one query, so 100/570 is not a general exposure rate. A passage selector may operate on passages rather than documents. Emphasize that counting is not enough; units and conditioning events matter. **Teaching check** Ask learners to reject one seductive but invalid ratio from the table. **Alt text** “Counts narrow from one thousand declared URLs to eight selected passages, with a marked change from corpus-level counts to query-conditioned counts.” ## Slide 04 — The stage vocabulary **On-screen text** Discover → Fetch → Parse/represent → Index → Retrieve → Select Below the line: Generate · Attribute · Act Rule: “A later event is never implied by an earlier label.” **Visual specification** Six purple rectangular stage cards form the upper row. Three gold rounded cards form a separated lower row for downstream events. Between upper cards are conditional arrows with a small “may” label. No arrow connects select to downstream without a dashed boundary. **Speaker notes** Define each word briefly. The upper row is W03’s focus. W04 examines fixed-candidate ordering and context selection more deeply; W05 examines generation and attribution. The decomposition helps locate tests but is not claimed as the internal structure of every system. **Teaching check** Give “the page was cited” and ask which earlier stages must be hypothesized versus directly observed. **Alt text** “Six conditional acquisition and retrieval stages appear above three separated downstream events, emphasizing that later events require additional evidence.” ## Slide 05 — Discovery is an address, not an outcome **On-screen text** Possible discovery signals: - lawful links and feeds - submitted maps - curated source lists - prior authorized captures Not established: fetch · index · rank · citation **Visual specification** Four source-signal tiles point to an address ledger. Beyond the ledger are four closed gates labeled Fetch, Index, Rank, Cite. The first arrows are solid; every later arrow is dashed. **Speaker notes** Describe discovery as learning a source identity that could be considered. A sitemap entry can be inspected as a publisher-side signal, but it does not demonstrate that a third party discovered or processed the entry. Keep authorization separate from technical reachability. **Teaching check** Complete the sentence: “A sitemap entry supports…, but cannot support….” **Alt text** “Links, maps, lists, and captures enter an address ledger; fetch, index, rank, and citation remain behind separate closed gates.” ## Slide 06 — Fetch and representation change the object **On-screen text** Fetch record: identity · time · request · response · rights Representation record: parser · fields · passage unit · hash · exclusions “HTTP 200” is not a complete representation trace. **Visual specification** A two-panel transformation. Left: a byte-document envelope with request metadata. Right: three extracted cards labeled title, body passage, and structured field. A discarded navigation strip is shown in a dotted waste tray with a recorded reason. **Speaker notes** Explain content negotiation, redirects, locale, rendering, segmentation, boilerplate removal, and truncation as possible causes of representation differences. Do not imply which choices a closed platform makes. In an owned system, record the exact transformation and identity. **Teaching check** Name one way the same URL could yield two lawful, materially different representations. **Alt text** “Fetched bytes with request metadata are transformed into title, passage, and structured-field records while an excluded navigation strip retains a reason.” ## Slide 07 — Eligible, indexed, retrievable **On-screen text** **Eligible:** satisfies declared admission conditions **Indexed:** present in one named snapshot **Retrievable:** can enter a candidate list under one query/configuration All three are versioned statements. **Visual specification** Three nested but non-identical sets. The eligible set overlaps the indexed snapshot; one indexed card is filtered from a query-specific retrievable subset. Each set has a version tag and clock icon with textual date placeholders. **Speaker notes** Avoid drawing perfect containment: policy changes, stale snapshots, or separate index partitions can break naive nesting. “Retrievable” needs a query, cutoff, filters, and configuration. A document can be indexed yet receive no score for a given query. **Teaching check** Rewrite “our content is indexed” as a statement with an index identity and observation method. **Alt text** “Three versioned sets distinguish eligibility, one index snapshot, and a query-specific retrievable subset; their boundaries do not perfectly coincide.” ## Slide 08 — Build the inverted index **On-screen text** Documents: - A: “retrieval trace retrieval” - B: “citation trace” - C: “retrieval boundary” Postings: - `retrieval → (A,2), (C,1)` - `trace → (A,1), (B,1)` **Visual specification** Three document strips on the left and a token-to-postings cabinet on the right. Fine lines connect each highlighted token to its posting. Document IDs and frequencies are printed next to every connector. **Speaker notes** Use the micro-corpus to explain document frequency versus term frequency. The posting identifies candidate access and may store positions or fields. The trace is easy because tokenization is declared. A different tokenizer can create a different index from the same visible text. **Teaching check** Learners create the postings entry for `boundary` and state its document frequency. **Alt text** “Token occurrences in three short documents connect to postings lists; retrieval occurs twice in A and once in C, while trace occurs once in A and B.” ## Slide 09 — Tokenization is a modeling decision **On-screen text** `AI-search`, `AI search`, `ＡＩ検索`, `生成式搜索` Questions: - normalization? - punctuation and case? - word or character units? - stemming or segmentation? - fields and truncation? **Visual specification** Four input strings pass through three differently labeled tokenizer boxes, yielding visibly different token tiles. The outputs are arranged as parallel rows; none is marked “correct” without a language/task label. **Speaker notes** Explain that the L03 tokenizer is English-oriented and intentionally simple. It cannot justify multilingual effectiveness. For a multilingual study, tokenizer identity and locale-specific evaluation are part of the protocol, not a footnote. **Teaching check** Ask which comparison becomes invalid if two methods use different document segmentation. **Alt text** “Four multilingual and punctuated expressions produce different token sequences under three tokenization policies, none universally preferred.” ## Slide 10 — BM25 as a declared convention **On-screen text** \[ BM25(q,d)=\sum_{t\in q}IDF(t) \frac{f(t,d)(k_1+1)}{f(t,d)+k_1(1-b+b|d|/avgdl)} \] L03: positive IDF · `k1=1.2` · `b=0.75` **Visual specification** The equation sits above three labeled brackets: rarity, frequency saturation, and length normalization. A manifest card at the side lists tokenizer, index snapshot, parameters, cutoff, and tie policy. **Speaker notes** Read the equation in words. For each query term, BM25 multiplies a corpus-rarity component by a saturated within-document frequency component adjusted for document length, then sums. Call it a convention because implementations vary. **Teaching check** Which terms in the equation change if the corpus snapshot changes but the document text does not? **Alt text** “The BM25 equation is annotated with rarity, frequency saturation, and length normalization, beside a required implementation manifest.” ## Slide 11 — IDF is corpus rarity, not importance **On-screen text** \[ IDF(t)=\log\left(1+\frac{N-n_t+0.5}{n_t+0.5}\right) \] As `n_t` rises with `N` fixed, this IDF falls. Not measured: truth · authority · social value **Visual specification** A simple original curve with horizontal axis `n_t / N` and vertical axis “positive IDF.” Three annotated points say rare in this corpus, mid-frequency, and common in this corpus. A separate crossed-out badge says “world importance.” **Speaker notes** Clarify that document frequency counts documents containing the token, not repetitions. The value depends on the frozen corpus and tokenization. A rare typo can have high IDF; a common authoritative term can have low IDF. **Teaching check** Why might an uncommon promotional phrase receive high lexical weight yet remain poor evidence? **Alt text** “A decreasing curve links higher corpus document frequency to lower positive IDF; a separate marker rejects interpreting rarity as world importance.” ## Slide 12 — Saturation and length normalization **On-screen text** Term frequency: diminishing returns Document length: declared normalization `k1` controls saturation · `b` controls length effect **Visual specification** Two small plots. Left: a concave term-frequency contribution curve with first and fourth occurrence marked. Right: two documents of different length with the same one occurrence; a scale icon indicates that the score can differ under `b>0`. **Speaker notes** The fourth repetition normally adds less than the first. This prevents unlimited linear reward for repetition, but it is not an anti-spam guarantee. Length normalization compares frequency in relation to document length under the corpus average. It does not judge writing quality. **Teaching check** What happens to length normalization when `b=0`? **Alt text** “A concave curve shows diminishing term-frequency contribution; a second panel compares the same term count in short and long documents under length normalization.” ## Slide 13 — Reproduce one score before interpreting it **On-screen text** `N=3`, `avgdl=100`, `n_t=2`, `k1=1.2`, `b=0.75` | Document | length | tf | term score | |---|---:|---:|---:| | A | 100 | 3 | 0.738 | | B | 50 | 1 | 0.591 | Conclusion: A ranks above B **under this setup**. **Visual specification** A calculation ledger with input cells, one IDF cell `≈0.470`, and two transparent arithmetic paths ending at the table. No winner trophy; use a small “conditional order” label. **Speaker notes** Work the numbers slowly. Then ask what did not enter the formula: factuality, provenance, entitlement to a claim, citation interface, or user action. This separation is the purpose of the baseline. **Teaching check** Learners circle every assumption that must travel with the two scores. **Alt text** “A transparent ledger calculates positive IDF around 0.470 and term scores 0.738 for A and 0.591 for B under stated parameters.” ## Slide 14 — Qrels are protocol records **On-screen text** `topic_id · doc_id · relevance · assessor protocol` L03 T-03: - D-005 → 3 - D-006 → 2 - D-010 → 1 Unlisted pairs: gain 0 **inside this closed fixture**. **Visual specification** A qrel ledger with the three rows, a policy stamp, and an outer dashed box labeled “tiny CC0 teaching corpus.” Outside the box, an unjudged card carries a question mark rather than zero. **Speaker notes** Explain that labels depend on a unit and assessment rule. L03 can simplify unlisted pairs because the policy and small closed corpus are explicit. In larger pools, unjudged often means not assessed. Never change labels after seeing which method benefits. **Teaching check** What evidence would be needed to change an unjudged pair into a relevance judgment? **Alt text** “Three graded T-03 qrels sit inside a closed-fixture boundary; an external unjudged record remains a question mark.” ## Slide 15 — Candidate Recall@k **On-screen text** \[ Recall@k=\frac{|top\text{-}k\cap relevant|}{|relevant|} \] T-03 BM25 top 3: D-005, D-006, D-008 Relevant: D-005, D-006, D-010 Recall@3 = 2/3 **Visual specification** Two aligned sets with document chips. D-005 and D-006 overlap; D-008 is only in retrieved; D-010 is only in relevant. The denominator brace encloses all three relevant chips. **Speaker notes** Name D-010 as a candidate miss at cutoff three. D-008 is nonrelevant under the fixture labels, not proven universally useless. Explain why the cutoff and qrel policy are inseparable from the metric. **Teaching check** If D-010 is rank four, what are Recall@3 and Recall@5? **Alt text** “The retrieved and relevant sets overlap on two of three relevant documents, so T-03 BM25 Recall at three is two thirds.” ## Slide 16 — A reranker cannot recover an absent candidate **On-screen text** Candidate set at depth 3: `{D-005, D-006, D-008}` Reorder freely → maximum Recall@3 remains 2/3. Candidate generation sets the downstream ceiling. **Visual specification** A transparent box contains three cards that can rotate among three rank slots. D-010 remains visibly outside the box. A lock icon sits on the box membership, while rank arrows move inside. **Speaker notes** Distinguish membership from order. A fixed-set reranking experiment may change NDCG or reciprocal rank, but it cannot add D-010. To test candidate generation, change the first stage or candidate depth under a frozen task. To test reranking, prove set invariance. **Teaching check** Which variable must be held fixed to isolate ordering effects? **Alt text** “Three candidates can change order inside a locked box, while relevant D-010 remains outside and cannot be recovered by reranking.” ## Slide 17 — Dense similarity has an identity **On-screen text** Dense route identity: model/checkpoint · pooling · truncation · normalization · language · index build Similarity ≠ entailment ≠ source authority **Visual specification** A query and three passages map into vector arrows, followed by a score column. Above the mapping is an identity card with six required fields. A fence separates similarity from two later labels: entailment and authority. **Speaker notes** Dense retrieval can help with paraphrase, but every result belongs to a model and build. The geometry alone does not establish claim support. Preview the special L03 constraint: its second run has no model identity because scores were authored for teaching. **Teaching check** Why is “semantic match” still too broad for a claim-support judgment? **Alt text** “Query and passage vectors produce similarity scores under a six-field identity card; entailment and authority remain behind a separate boundary.” ## Slide 18 — Hybrid requires a fusion rule **On-screen text** L03: per-topic min–max normalization + weighted interpolation `hybrid_alpha = 0.55` Do not add heterogeneous raw scores without a reason. **Visual specification** Two vertical score thermometers labeled lexical and supplied opaque. Each passes through its own normalization box, then into a balance labeled alpha. A final ranked list emerges with a full parameter tag. **Speaker notes** Explain that raw scales may be incomparable. Normalization makes one explicit choice, not the only choice. Rank fusion and learned calibration are alternatives. A fusion result can improve coverage while worsening order on one topic. **Teaching check** What additional comparison should accompany a favorable macro hybrid score? **Alt text** “Lexical and opaque score columns are normalized separately, weighted by alpha 0.55, and combined into one parameter-labeled ranking.” ## Slide 19 — Know the L03 fixture ceiling **On-screen text** L03 input: - 10 CC0 synthetic documents - 3 authored topics - 7 graded qrels - 1 synthetic opaque score file - `model_identity: none` Use: pipeline and metric exercise only. **Visual specification** Five inventory cards arranged inside a bold box labeled “offline deterministic fixture.” Outside the box, faded labels—commercial engine, live crawl, user outcome—are crossed by a boundary line. **Speaker notes** Read the metadata language. Never rename the opaque scores after a known model. The fixture supports reproducibility, stage, and metric reasoning. Its scale is intentionally too small and artificial for retriever-family claims. **Teaching check** Correct the sentence: “The dense model beats BM25 on production retrieval.” **Alt text** “A boxed inventory lists ten synthetic documents, three topics, seven qrels, and one authored opaque run with no model identity; production claims remain outside.” ## Slide 20 — T-03: locate the candidate miss **On-screen text** | Method | Top 3 | Recall@3 | |---|---|---:| | BM25 | 005, 006, 008 | 0.667 | | Supplied opaque | 005, 010, 006 | 1.000 | | Hybrid | 005, 006, 010 | 1.000 | Observation: D-010 crosses the cutoff in the supplied routes. **Visual specification** Three horizontal ranked tracks with the cutoff after slot three. Relevant cards have a `qrel>0` text tag. D-010 travels from outside BM25’s cutoff to slots two or three in the other tracks; no causal arrow is drawn between methods. **Speaker notes** Make the result precise. The difference exists in one tiny frozen fixture. The opaque route’s values are authored. The observation concerns membership at cutoff three, not why a real model or platform would behave similarly. **Teaching check** Which single document accounts for the Recall@3 difference on T-03? **Alt text** “Three T-03 rankings show relevant D-010 outside BM25’s top three but inside the supplied opaque and hybrid top three, raising fixture recall from two thirds to one.” ## Slide 21 — T-02: recall can hide ordering risk **On-screen text** BM25 and hybrid top 3: D-009, D-003, D-004 Recall@3 = 1.000 NDCG@3 ≈ 0.665 D-009: unsupported promotional record, qrel gain 0. **Visual specification** A three-slot podium places D-009 first with a gray `gain 0` band; D-003 and D-004 carry gains 3 and 2 below. Beside it, a full recall gauge and a lower ordering gauge demonstrate different questions. **Speaker notes** All labeled-relevant documents are present, so candidate recall is complete. Yet a gain-zero record comes first, lowering the graded rank metric. This is not a candidate miss. It is an ordering problem under the fixture judgments and motivates W04. **Teaching check** What changes could improve NDCG without changing Recall@3? **Alt text** “A gain-zero promotional document ranks above two relevant documents; recall is complete while the graded ordering score remains lower.” ## Slide 22 — Claim ladder: what the run establishes **On-screen text** **Observed in L03:** inputs · parameters · lists · qrels · metrics · hashes **Not evaluated:** crawl · production index · context · generation · citation · action Closed-platform mechanism: unknown. **Visual specification** A two-story evidence house. The solid lower story contains inspectable L03 artifacts. The hatched upper story contains downstream and production labels. There is no staircase between stories; a dashed research-question arrow sits outside. **Speaker notes** Stress that a boundary is productive: it shows what experiment comes next. L03 can reproduce list and metric behavior. A closed response observation can measure a visible event under conditions. Neither grants the missing internal trace. **Teaching check** Learners add one “not evaluated” item that is frequently conflated with retrieval. **Alt text** “Inspectable L03 artifacts occupy a solid lower story; crawling, production indexing, context, generation, citation, and action occupy a separate hatched story.” ## Slide 23 — Two equivalent learning routes **On-screen text** **Synchronous:** 90-min planned seminar + 75-min core studio **Asynchronous:** notes + worked case + offline run + transcript + boundary memo Status: authoring budgets; no timed pilot. **Visual specification** Two parallel paths converge on five identical outcome badges: stage map, BM25 trace, Recall card, topic diagnosis, bounded conclusion. The paths differ in activity sequence but not in assessed outputs. **Speaker notes** Explain that equivalence means the scientific propositions and assessed evidence are available without live media. The transcript is not an existing recording. Real learner time must be measured in a pilot before claims about pacing or workload are made. **Teaching check** Ask learners to choose a route and name the first artifact they will produce. **Alt text** “Synchronous and asynchronous paths converge on the same five assessed artifacts; a status label says the timing has not been piloted.” ## Slide 24 — Exit ticket: two bounded sentences **On-screen text** Sentence 1: one L03 result with corpus, cutoff, and denominator. Sentence 2: the strongest downstream or closed-platform claim it does **not** establish. If “visibility” appears without an event, revise. **Visual specification** Two large writing fields. The first has three small required-field tabs: scope, metric, result. The second has two tabs: unknown stage, forbidden extrapolation. A final arrow points to W04: fixed candidates → ranking and context selection. **Speaker notes** Close by collecting two examples. A satisfactory first sentence may report T-03 BM25 Recall@3 as 2/3 in the synthetic corpus. The second should reject a claim about commercial indexing, citation, or user outcomes. Preview W04’s ordering question. **Teaching check** Submit both sentences before leaving; peer-check the denominator and boundary. **Alt text** “Two writing fields require a scoped metric result and a separate forbidden extrapolation, followed by a bridge to W04 ranking and context selection.”