# The `modality` gates

> A proposed addition to A2UI: three fields that tell a renderer **whether to speak**, never what to say.

## The problem

A model reading a component tree can work out *what to say*. It can see there are four charging stations with names and prices, and phrase that better than any template you would write. Nothing needs to declare that.

What it cannot work out is **what happens if it gets this wrong**.

- The tree says `Image`. Is describing it fine ("a photo of your receipt") or a disaster ("the contract you're about to sign")?
- The tree says `Button: Submit`. Is that submitting a search, or wiring €4,000?
- The tree says `Text: "•••• 4417"`. Fine on a screen. Fine to say aloud with passengers in the car?

None of these are inferable from structure. They are facts about consequences, and consequences are declared, not guessed — the same reason a permissions manifest is written by a developer rather than derived from the code.

## The three fields

```json
{
  "id": "payButton",
  "component": "PaymentConfirm",
  "accessibility": { "label": "Pay for this charge" },
  "modality": {
    "requiresVisual": false,
    "stakes": "readback",
    "spokenSensitive": false
  }
}
```

| Field | Type | Default | Meaning |
|---|---|---|---|
| `requiresVisual` | boolean | `false` | Speech cannot carry this. The renderer must escalate to a display or decline the turn. It must **never** substitute a description. |
| `stakes` | `none` \| `confirm` \| `readback` | `none` | How careful a commit has to be. `confirm` asks yes or no; `readback` restates the specifics and waits for an explicit yes. |
| `spokenSensitive` | boolean | `false` | Renders normally, is never spoken. Account numbers, medical details, addresses. |

They are **gates, not scripts.** None of them contains words. That is why there are only three.

## Where they live

Beside `AccessibilityAttributes` on `ComponentCommon`.

That is the precedent, and the argument is the same one A2UI already accepted: metadata that isn't visual, that every component can carry, that a renderer needs in order to behave correctly. A screen reader needed `label` and `live`; a voice renderer needs to know what it must not say. **A voice renderer is a screen reader that also knows the stakes.**

`live` is already doing announce-on-change, which is voice-adjacent work — so the protocol has one foot here already.

### Declared per component *type*, not per surface

Set `modality` in the **catalog**, on the component definition. Mark `PaymentConfirm` as `readback` once and every surface that ever uses it inherits the gate. Agents emit nothing extra; nothing is repeated per turn; a designer cannot forget it.

An instance may override, mirroring how explicit accessibility attributes override inferred defaults.

## Degrading with zero annotations

A catalog that has never heard of `modality` must still render correctly. Every field defaults to the permissive-but-quiet value, and the renderer falls back to inferring from structure — exactly the "Infer Default Semantics" rule the spec already makes normative for accessibility.

Annotations improve the output. They are never a precondition. Make them mandatory and nobody writes them.

## Adoption, in two steps

**Step 1 — needs nobody's approval.** Catalogs define their own component properties. So `modality` can go straight into your own catalog today, with **zero protocol change**. That is exactly what [`catalog/charging.catalog.json`](../catalog/charging.catalog.json) in this repo does, and why the demo runs.

**Step 2 — standardise.** Propose it for `ComponentCommon` so a voice renderer can rely on it existing rather than sniffing for a vendor-specific key. A2UI already has the machinery: extensions are defined by URI and advertised in the AgentCard, renderers negotiate through `a2uiRendererCapabilities`, and there is a live proposals process.

Build first, propose second. A spec written before anyone implements it is a spec nobody uses.

## What this does *not* add

- **No `speech` templates.** How to phrase a list is the model's job at runtime, and it is better at it than a template.
- **No verbosity settings.** How chatty to be while the car is moving is a *host* decision, not a component author's. The head unit knows its speed; the developer does not.
- **No new messages.** `modality` is a component property, so it rides on `updateComponents` like every other property.

## A hole found by running it

The gates are declared on **components**. Sensitivity is a property of **data**. Those are not the same thing, and the gap between them is exploitable — not maliciously, just by a model doing something reasonable.

Observed live, with the travel catalog. `SensitiveList` carries `spokenSensitive`, so card details are safe there. But asked to confirm a booking, the agent wrote the card into `PaymentConfirm`'s `detail` field, and `readback` duly spoke it:

> "Ryanair FR7412 — Lisbon to Dublin, eighty-nine euros ninety, including aisle seat 5C, **charged to the Visa ending 4417**. Confirm?"

Nothing was violated. `PaymentConfirm` is not `spokenSensitive`, and the renderer did exactly what the gates told it. The declaration was simply attached to the wrong thing.

**This is a defect in the proposal, not in the renderer**, and prompt instructions only paper over it — the demo now tells the agent to keep sensitive values out of spoken fields, which reduces the frequency and fixes nothing structural.

The real fix is that `spokenSensitive` needs to be expressible on a **data path**, not only a component type — closer to how a schema marks a field as PII than to how ARIA marks a node. Something like a surface-level `sensitivePaths: ["/charges/*/card"]`, checked wherever a value is about to be spoken rather than wherever a component is about to be rendered.

That is a larger change than three boolean-ish fields, and it is the main thing standing between this proposal and one worth upstreaming. Anyone evaluating the idea should start here.

## Open questions

1. **Should `stakes` be an enum or a number?** An enum reads better; a number composes better when a surface mixes components. Currently the highest wins.
2. **Is `requiresVisual` a component fact or a data fact?** Same shape as the hole above — a chart always needs eyes, but an `Image` might be decorative. Both may belong on the binding rather than the type.
3. **How does `spokenSensitive` interact with earbuds?** Private audio changes the answer, so this may be renderer-side policy rather than a hard boolean.