Recall(From the previous post)
The dual-model architecture post cited TML-Interaction-Small's benchmark numbers (FD-bench V1 latency, FD-bench V1.5 interruption-handling quality) without explaining why those numbers look the way they do. This post is the explainer — what the model actually does differently, for anyone who hasn't been following voice-AI architecture closely.
Start with what it replaces
Almost every voice assistant you've used — Siri, Alexa, ChatGPT's voice mode in its earlier forms — is built as a pipeline of separate components:
- ASR (automatic speech recognition, e.g. Whisper) converts your audio to text
- A language model reads that text and generates a text response
- TTS (text-to-speech) converts the response back to audio
- A voice activity detector (VAD) sits in front of all this, deciding when you've stopped talking so step 1 can run
Definition(Turn-based pipeline)
A system where each component waits for the previous one to finish a complete unit of work — full utterance in, full utterance out — before the next component starts. WordBridge's proposal explicitly lists "Whisper + LLM turn-based pipeline" as one of its baselines. This is what that means.
This works fine for "ask a question, get an answer." It breaks down for anything that requires the system to react while you're still talking — which is exactly WordBridge's core requirement.
The actual architectural change
TML-Interaction-Small doesn't bolt real-time behavior onto this pipeline — it removes the pipeline. The model is trained from scratch, end to end, on continuous multimodal input:
- Audio is converted to embeddings via a lightweight scheme called dMel — no separate Whisper-style ASR model
- Video is split into small patches and encoded directly
- Text is tokenized normally
All three feed into one model that's trained jointly. There's no "convert speech to text, then think, then convert back to speech" — the model operates on the raw streams the whole way through.
Intuition(Why this matters more than it sounds)
In a pipeline system, each stage adds latency and each handoff loses information (tone of voice, hesitation, timing — all the things that get flattened when audio becomes text). A model trained end-to-end on the raw streams doesn't have those handoff points. For WordBridge, this is the difference between "the text transcript shows a pause" and "the model directly perceives the pause as it happens."
200ms micro-turns: always listening, always (potentially) speaking
The core mechanism: the model processes input and output as continuous interleaved streams, broken into 200-millisecond chunks. At every chunk, the model is simultaneously:
- Consuming the last 200ms of incoming audio/video
- Producing the next 200ms of its own output (which might be silence, might be speech)
Example(What this looks like in practice)
In a turn-based system: you talk → silence → VAD decides you're done → system processes → system replies.
In TML-Interaction-Small: you're talking, and the model is already processing every 200ms slice of that as it arrives. If you pause mid-sentence, the model doesn't need a VAD to "notice" — it's been tracking the stream continuously and can react within the next 200ms chunk.
This is what makes full-duplex possible: the model can speak while you're speaking (backchanneling — "mhm," "right"), handle interruptions gracefully, and — the capability most relevant to WordBridge — proactively interject based on visual or temporal cues, without anyone prompting it.
Why it's a 276B model that runs efficiently
TML-Interaction-Small is a 276B-parameter mixture-of-experts (MoE) model with 12B active parameters.
Definition(Mixture-of-experts (MoE))
An architecture where the model contains many specialized sub-networks ("experts"), but only a small subset is activated for any given input. Total parameter count (276B) determines how much the model has learned overall; active parameter count (12B) determines how much compute each forward pass actually costs. This is how the model can have GPT-4-class scale while running at the speed the 200ms micro-turn loop requires.
On the inference side, the team built "streaming sessions" — each 200ms chunk is sent as its own request, with results appended into persistent GPU memory rather than reprocessing context from scratch each time. This was significant enough that it was upstreamed into SGLang (an open-source LLM serving framework), meaning it's not a proprietary trick — it's now available infrastructure.
Native time-awareness — relevant to the Background Model
One benchmark result stands out: TimeSpeak, which tests temporal awareness. TML-Interaction-Small scores 64.7% versus 4.3% for GPT-Realtime-2 in minimal-reasoning mode.
Note(Why this connects to WordBridge's contextual anchors)
A contextual anchor like "you were asking about your medication schedule, that was about ten minutes ago" requires the model to actually track elapsed time within a conversation — not just sequence ("this came after that"), but duration. Pipeline-based systems generally don't have this natively; timestamps have to be bolted on as metadata. TML-Interaction-Small's native time-awareness is close to a prerequisite for the Background Model's anchor-generation role to work the way the proposal describes it.
The trade-off: a small intelligence cost
Optimizing for interactivity isn't free. On Audio MultiChallenge (a general audio-reasoning benchmark), TML-Interaction-Small scores 43.4% versus 48.5% for GPT-Realtime-2 at its highest reasoning setting — a 5.1-point gap.
Warning(This is the H1 comparison, concretely)
H1 claims WordBridge's passive detection is "non-inferior to explicit-query baselines" — and one of the named baselines is explicit-query GPT-4o-class models. The Audio MultiChallenge gap suggests the interaction model by itself gives up a small amount of raw reasoning quality for its latency/interactivity advantage. WordBridge's bet is that the temporal advantage (H2 — getting a candidate at 500ms instead of after the full utterance) outweighs this ~5-point intelligence gap. That's a real trade-off, not a free win, and it's worth treating as its own ablation: how much of the gap does the Background Model's longer-horizon context close, if any?
A modality WordBridge's proposal doesn't currently use
TML-Interaction-Small processes video in the same loop as audio — enabling things like counting exercise repetitions or detecting posture errors in real time, which audio-only competitors (GPT-Realtime-2, Gemini Live) can't do at all.
WordBridge's current design uses audio (conversation) + wearable HR/GSR (physiological state). It doesn't use video. Given the previous post's concerns about HR/GSR reliability in real-world conditions, visual cues (posture, facial expression, gesture during circumlocution — people often gesture more when groping for a word) are a third signal channel the architecture already supports, that the current proposal leaves on the table. Whether it's worth the privacy trade-off of a camera vs. just audio + wearable is a separate question — but it's an available option, not a hypothetical one.
Safety methodology, briefly
Two details from TML's safety work are relevant to the previous post's discussion of WordBridge's own safety tiers:
- Modality-appropriate refusals — when the model needs to decline something, it does so as natural speech via TTS, not as a jarring mode-switch to text. For WordBridge, this is the same design problem as Tier 2/3 transitions: the way a suggestion gets suppressed matters as much as the suppression itself.
- Automated red-teaming for multi-turn robustness — testing not just single responses but extended interaction sequences. WordBridge's tier transitions are inherently multi-turn (Tier 1 → 2 → 3 across a conversation), so this kind of testing methodology is directly applicable to validating the tier-transition logic itself, not just individual suggestions.
Summary(Summary)
TML-Interaction-Small's core innovation is training one model end-to-end on continuous multimodal streams in 200ms micro-turns, rather than chaining ASR → LLM → TTS with a VAD front-end. This is what makes full-duplex, proactive interjection, and native time-awareness possible — all three are close to load-bearing requirements for WordBridge's design. The trade-offs worth tracking: a measurable (~5 point) intelligence gap versus explicit-query baselines on general reasoning, which WordBridge's temporal-advantage hypothesis (H2) needs to outweigh; and an unused video modality that could address some of the physiological-signal reliability concerns raised in the safety tiers post, at the cost of adding a camera to what's currently an audio-plus-wearable design.