Multilingual, Multimodal, and Agentic Surfaces

1. Two discontinuities: representation and action

W11 begins where a text-only answer audit becomes insufficient. The first discontinuity is representation. A visual asset can be available as pixels, alternative text, a nearby caption, OCR text, page metadata, an embedding, a retrieved crop, or a generated description. These objects are related, but they are not interchangeable. The second discontinuity is action. A model can recommend an operation in prose, propose a structured tool call, receive authorization, attempt the call, receive a tool response, verify an external outcome, and perhaps roll it back. Those are different events with different evidence.

A useful audit therefore starts with two ledgers. The representation ledger has one row per transformation and records input identity, transformer identity, version, locale, parameters, output identity, hash, and review status. The action ledger has one row per state transition and records the actor, requested capability, permission decision, target, expected side effects, attempt status, returned value, independent verification, log location, stop rule, and rollback status.

Do not draw a single arrow labeled “understood image” or “agent acted.” Such labels hide the very events an evaluator needs. The aim is not to claim that every production system follows our diagram. The diagram is an inspection grammar: it forces a claim to identify which observable representation or action state supports it.

The W05 distinctions remain active. Source presence does not establish absorption; absorption does not establish entailment; a correct claim can be misattributed; a cited claim can be incorrect. With images, “source” must name the representation. With actions, “answer” must not be used as shorthand for a side effect.

2. Asset identity before model output

An audit unit should be defined before any transformation. Give the asset a stable identifier, record its origin, acquisition time, byte hash, media type, dimensions, declared license, and whether it contains personal, confidential, or regulated information. For a course-owned synthetic chart, the origin can be an authored fixture and the license can be stated directly. For an external asset, a URL is not enough: mutable content can change at the same address, and the right to access does not imply the right to reproduce.

Separate the asset from its page context. A page may contain a headline, caption, surrounding paragraphs, structured data, and accessibility fields. A model-facing system may use any subset or transform. If the generated answer matches a caption, that observation does not prove the pixels were processed. If it matches OCR text, that does not prove the nearby explanation was used. The audit should preserve the candidate paths instead of choosing a hidden mechanism from surface similarity.

The first stop conditions are rights and safety conditions. Stop if the asset identity cannot be resolved, the license does not cover the proposed classroom use, the content exposes a person, or the provenance record is missing. A synthetic label is not a loophole: the asset must actually be fictional or generated under a recorded course process. Do not rename a real screenshot “synthetic.”

3. Cross-modal transformation ledger

Treat every derived representation as a new artifact. A minimal ledger can contain these stages:

The hashes in this ledger do not prove semantic fidelity. They prove only that two recorded byte sequences are or are not identical. Semantic comparison needs explicit checks: numeric agreement, entity identity, polarity, uncertainty, units, time, and scope. A chart with values 12 and 8 can yield valid alt text that mentions both, invalid OCR that drops the 8, a caption that mentions only the direction, and a fluent generated description that invents causality. Fluency is not a fidelity metric.

Record transformer configuration at the level needed to explain reruns. “OCR” is not an identity. At minimum record product or library name, version, language pack, preprocessing, crop, and date. If the model or service cannot be identified, say unknown; do not substitute a familiar product name. If an embedding vector cannot be meaningfully inspected, retain the input/output identities and retrieval behavior while clearly stating the interpretability ceiling.

4. Evidence-bearing alternative text and long descriptions

Alternative text is part of the evidence surface, not decorative metadata. Good short alt identifies the kind of visual and conveys its task-relevant conclusion. It should be concise enough for navigation but specific enough that a learner who cannot see the image can participate in the same teaching check. “Chart” is inadequate. “Bar chart comparing Group A and Group B” identifies form but not evidence. “Bar chart: 12 verified cases in Group A and 8 in Group B; four records remain unresolved” carries the relevant values and uncertainty.

Complex visuals need a long description or table alternative. The long description should state the reading order, encodings, values, annotations, and limitations. It should distinguish observed values from interpretation. It should not turn visual proximity into causality or replace uncertainty with a confident narrative. For an action diagram, describe states and allowed transitions, including the non-transition from a recommendation to execution.

Authored alt and generated descriptions require provenance of their own. Record who or what wrote them, which asset version was observed, language and locale, review status, and change history. Translation of alt text is not a clerical afterthought: a translated label can shift an entity, unit, or culturally specific concept. If bilingual review is pending, the record should say so and block pooling.

L07 makes this operational. Its AB-05 case concerns loss of accessible claim-to-source association. The deterministic script yields an escaped high-residual case and blocks release. The lesson is not that one automated accessibility field guarantees equivalence. It is that accessibility is a release condition with evidence, ownership, and rollback—not a cosmetic task at the end.

5. OCR, caption, and description are different witnesses

OCR answers a narrow question: what text-like marks did a configured recognizer extract from pixels or a region? It may omit small labels, reorder columns, confuse scripts, or normalize punctuation and numbers. Always keep the OCR output separate from authored text. When a recognizer drops the value 8, the audit should preserve that failure. Silently correcting the output destroys evidence about the transformation.

A caption is normally authored in page context. It may summarize the visual, explain why it matters, or identify a source. It is not necessarily a literal transcription. PAPER-25 is useful as a caption-in-context multimodal proxy, but its role is bounded. A caption response does not establish direct image consumption, a universal caption rule, or transfer to another system. The evaluated configuration, task, version, and publication record determine the ceiling.

A generated description is yet another witness. It can combine visible evidence, OCR tokens, retrieved context, or prior knowledge. A description that repeats a number is not proof of where that number came from. Use ablations or controlled representation availability when the system is inspectable: pixels only, alt only, caption only, and combinations. Even then, report the tested configuration rather than a universal architecture. In a closed surface, record observations and plausible paths without asserting hidden internals.

6. Embeddings, retrieval, and description without identity collapse

An embedding is a derived numerical representation used for comparison or retrieval. It is not a citation, quotation, or accessible explanation. Its identity includes the encoder, version, preprocessing, input region, dimensionality, normalization, and index snapshot. When those are unknown, causal claims about why an item was retrieved are not available.

Cross-modal retrieval adds at least three evaluable boundaries: whether the relevant asset or region is represented in the index, whether it enters the candidate set for a particular query, and whether the downstream system receives the relevant representation. A correct generated description may still rest on the wrong asset. A retrieved correct asset may still be described incorrectly. A faithful description may still be attributed to the wrong page. Keep identity, retrieval, entailment, and attribution as separate columns.

PAPER-02 provides a bounded threat study involving coordinated image and text changes in evaluated vision-language ranking conditions. It motivates adversarial review of cross-modal signals. It does not establish that every vision-language system is vulnerable, reveal a commercial ranker, or authorize reproducing an attack. W11 converts the threat class into defensive questions: do transformations disagree, can one representation dominate without corroboration, are suspicious changes logged, and does release stop when evidence is inaccessible or unverifiable?

7. Cross-modal support, conflict, and attribution

For each claim, build a matrix whose rows are representations and whose columns are identity, value, unit, polarity, uncertainty, time, and source association. Label a cell supports, contradicts, incomplete, not present, or not auditable. Avoid a single “consistent” score that hides which field failed.

Suppose pixels and the source table show 12 versus 8 with four unresolved records. English alt carries all three facts. A caption says only that A exceeds B. OCR returns “12 versus” and loses the second value. A generated description says “A performed 50% better because of the intervention.” The caption is incomplete but not contradictory. OCR is incomplete. The generated description includes a correct arithmetic contrast if the denominator is B, but the phrase “performed better” may change the construct, and “because” invents causality. Answer correctness therefore requires claim-level decomposition.

Attribution also crosses modalities. A description may be correct about the chart but cite the enclosing page incorrectly. An alt string may have been authored for a prior version. Record asset hash and representation hash together. If a citation points to a page whose current asset differs from the audited hash, the association is unresolved.

8. Provenance integrity is not truth, rights, or ranking

S04, the C2PA Content Credentials Technical Specification 2.4 candidate in the controlled reference universe, is used narrowly. It describes cryptographically bound, tamper-evident provenance manifests and validation behaviors. A successful validation can support an integrity statement about the manifest and bound asset under the specified conditions. It does not tell an evaluator whether the content is true or good.

Keep six judgments separate:

1. Integrity: did the bound content or manifest change outside the validated chain?
2. Truth: do the represented claims correspond to the world?
3. Authorship: who actually created the expressive content?
4. Rights: does the current use have permission or a valid license?
5. Accessibility: can affected users obtain equivalent task-relevant evidence?
6. Ranking: did a given representation affect retrieval or ordering in a specified system?

S04 directly addresses only a portion of the first judgment. A perfectly intact false image remains false. An intact asset can still infringe rights. A signed asset can be inaccessible. A validated manifest does not reveal a hidden ranking mechanism. This separation should appear in every learner conclusion.

9. Multilingual entity and cultural validity

Translation is not a lossless string substitution. Begin with an entity ledger: canonical course ID, original-language name, native script, aliases, transliterations, translated labels, homonyms, geography, time validity, and reviewer. A transliteration preserves sound approximately; a translation may preserve meaning approximately; neither automatically resolves identity.

Measurement can fail even when sentences are grammatically correct. A query can invoke a culturally specific category, politeness norm, regulatory term, or product taxonomy. Response length, citation style, refusal behavior, and source availability can vary by language and locale. A comparison should therefore identify the construct being measured and ask whether the task has the same meaning and opportunity set across cells.

The PAPER-01 route is especially important as a process lesson. Its reported publication and collection timing conflict with the 2026-08-24 course freeze. W11 quarantines it: learners may audit the time-provenance inconsistency, but they may not import its substantive cultural findings as course facts. Quarantine is not a claim that every result is false. It is a claim that the current evidence package cannot pass admission.

Cultural validity requires affected-language review, not just back-translation. A bilingual reviewer should check entity resolution, connotation, task equivalence, omitted context, and potential harms. Record disagreements. Do not average them away.

10. The locale, model, and pooling record

Every observation cell should record provider or system label, surface, observable version, model identity if exposed, query text, language, locale, geography, account state, tool/search state, time, repetition number, and sampling route. For transformations, add OCR/caption/translation model and version. For retrieval, add index snapshot if available. Unknowns stay explicit.

Pooling is a decision, not a default. Before combining languages or locales, state:

• the common estimand or descriptive target;
• evidence that the construct and task are comparable;
• entity alignment and translation review status;
• identical or intentionally modeled sampling frames;
• model, surface, account, geography, time, and tool-state comparability;
• scoring-rule equivalence and missingness policy; and
• a plan for heterogeneity rather than an assumption of invariance.

If locale and language change together, the contrast is confounded. If a service version changes between runs, the time contrast is confounded. A pooled mean can be numerically valid yet scientifically uninterpretable. W11’s fictional fixture therefore records English and zh-CN rows separately and sets pooling to blocked pending bilingual and measurement-equivalence review.

11. Recommendation is not action

An agentic surface requires an event model. Use seven states:

1. Recommendation: unstructured or structured text suggests an operation.
2. Proposal: a candidate tool name, arguments, target, and expected effect are formed.
3. Authorization: a policy or human grants a scoped capability for a defined target and time.
4. Attempt: the system invokes the tool under that capability.
5. Return: a value, error, or acknowledgement comes back.
6. Verification: an independent observation checks the intended and unintended outcomes.
7. Rollback: a compensating operation is attempted and then verified.

No state implies the next. A proposal with permission=denied is not an action. A tool response saying “success” is not independent outcome verification. A rollback token is not proof that rollback will work. The action ledger preserves these differences.

PAPER-39 provides a bounded recommendation-agent risk and defense benchmark. Its reported outcomes belong to the specified suite, agents, and attack variants. The source does not establish universal rates, guarantee a defense, or authorize live adversarial operations. In W11, it motivates permission boundaries, observation of side effects, and failure containment.

12. Permissions, side effects, logs, and rollback

Define permission before evaluation. Use least privilege: the smallest capability, target set, duration, data scope, and operation class needed. Separate read from write; draft from publish; create from delete; and local synthetic buffers from external accounts. A human approval should bind exact arguments, not merely a broad intent such as “handle this.”

Preview side effects: primary effect, secondary writes, notifications, cost, privacy exposure, rate-limit consumption, and downstream triggers. Require idempotency or a deduplication key where repetition is possible. Create an append-only event log containing request, decision, attempt, return, verification, and rollback events with timestamps and identities. Sensitive values should be redacted without destroying auditability.

Rollback must match the action. Deleting a sent message may not retract notifications or copies. Restoring a record may not restore ordering or external caches. State the rollback ceiling and residual effects. Stop if permission is missing, the target differs, the expected side-effect set changes, logs fail, verification is unavailable, or rollback cannot contain a material risk.

The W11 action fixture is disconnected. One event is only a recommendation. One proposal is denied and not executed. One authorized event is an offline simulation whose target begins synthetic:// and whose log and rollback are local fixture records. There is no live platform claim.

13. Source roles, industrial audits, and quarantines

Source admission is part of the method. PAPER-02, PAPER-25, and PAPER-39 are core because they illuminate bounded multimodal or agent-risk mechanisms relevant to this week. Each still carries an evidence ceiling. PAPER-05 is audit-only: it is useful for examining an industrial vision-language and agent architecture, denominators, randomization claims, attribution, and missing uncertainty. Its reported traffic or active-user outcomes are not reproduced here and cannot be converted into causal business-growth claims.

PAPER-01 is quarantined because time provenance is unresolved relative to the course cutoff. S04 is a standards candidate, not a law, certification, or proof of implementation. These roles prevent a reference list from becoming an undifferentiated authority stack.

When writing conclusions, attach every external claim to a public identifier and role. Say “PAPER-25 motivates a caption proxy in its evaluated context,” not “images are ranked through captions.” Say “S04 can validate specified provenance bindings,” not “S04 proves authenticity.” Say “the L07 fixture blocks release under its rule,” not “the system is unsafe everywhere.”

14. Bounded synthetic audit and release reasoning

The worked case combines the ledgers. Its fictional chart contains 12 verified cases in Group A, 8 in Group B, and four unresolved records. The English alt and long description preserve those facts. The OCR row deliberately omits 8. The Chinese alt is a course-authored translation marked pending bilingual review. Pooling remains blocked. The exercise asks learners to detect and preserve these states, not to improve a real asset.

The action trace separates recommendation, denied proposal, and authorized offline simulation. The offline return is verified only against a synthetic buffer record. It creates no external side effect. Even this modest fixture retains a rollback token and a stop rule, because safe habits should not appear only after a production connection exists.

L07 then supplies the governance consequence. A deterministic run reports six cases, three blocked, two detected, one escaped, audit status PASS, and release decision BLOCKED. AB-05 is the blocker. The correct conclusion is: the frozen course fixture was processed consistently; an escaped high-residual accessibility case meets the declared stop rule; external release is blocked; and no general safety, compliance, legal, or platform-effect claim follows.

The final W11 standard is deliberately demanding. A multimodal claim must name its representation and transformation path. A multilingual comparison must carry identity, locale, model, and pooling records. An agentic claim must name the action state and evidence for authorization, side effects, verification, logging, and rollback. Provenance integrity must remain separate from truth, rights, accessibility, and ranking. When any required evidence is absent, the disciplined result is not a guess about hidden machinery. It is an explicit unknown, quarantine, or stop.

Discussion prompts and exit test

1. When would caption agreement be insufficient to claim pixel use?
2. What task-relevant evidence belongs in short alt, and what belongs in a long description or table?
3. Which facts can a valid S04 manifest support, and which remain separate audits?
4. What would make a bilingual comparison poolable, and what is still missing in the fixture?
5. Which log event distinguishes a recommendation from an attempted action?
6. Why can L07 legitimately report audit PASS and release BLOCKED at the same time?

For the exit test, write one multimodal conclusion, one multilingual conclusion, and one action conclusion. Each must name its unit, observable evidence, unknowns, and stop condition. Include all six controlled routes—PAPER-02, PAPER-25, PAPER-39, PAPER-05, PAPER-01, and S04—without exceeding their assigned roles.