There is a threshold in closed-loop biosignal work that does not show up in accuracy numbers. Somewhere in the low tens of milliseconds, the experience changes character. Below it, people describe the system as responsive. Above it, they describe it as something they are operating.
Cortex Bridge is our attempt to stay on the right side of that line with dry electrodes, which make everything harder.
Where the milliseconds go
Latency in these systems is never one big cost. It is a dozen small ones, and most teams only measure two of them.
- Acquisition. The ADC window itself. Fixed by your sample rate and largely non-negotiable.
- Transport. Getting samples off the headset. USB bulk transfers batch, and batching is latency.
- Filtering. Every linear-phase filter has group delay proportional to its length. A clean 60 Hz notch is not free.
- Artifact rejection. Usually the largest cost, because most methods need a window of future samples to decide whether the present one is contaminated.
- Inference. Almost always the part people optimize, and almost never the bottleneck.
- Actuation. Whatever the signal drives, plus its own buffering.
When we first instrumented the full path end to end, inference was under a tenth of the budget. Artifact rejection was over half.
The lookahead trap
Standard artifact rejection is non-causal. To decide whether sample n is corrupted by a blink or a jaw clench, the easy methods look at samples n+1 through n+k. That works beautifully offline and costs you k samples of latency online.
Every offline benchmark rewards lookahead. Every real-time system pays for it. This is why offline accuracy is such a poor predictor of how a BCI feels.
We moved to a causal rejector that runs on the current sample plus a short history, and it is measurably worse by offline metrics. Users prefer it, unambiguously, because the loop closes fast enough to feel connected to their intent.
Dry electrodes change the problem
Gel electrodes give you a stable, low-impedance contact for hours. Dry electrodes give you a contact that changes as the subject moves, sweats and shifts the headset, and the impedance change shows up as slow drift plus intermittent high-amplitude junk.
Two things helped more than better filtering:
Track impedance continuously and weight channels by it. A channel that has gone bad should stop contributing before its noise reaches the classifier, not after. We inject a small out-of-band current and estimate per-channel contact quality every 250 ms.
Treat recalibration as normal. Fixed calibration assumes a fixed sensor. Ours is not fixed. The decoder updates its baseline continuously against a slow reference, so a headset that shifts mid-session degrades gracefully instead of falling off a cliff.
for block in stream.blocks(ms=4):
q = impedance.estimate(block) # per-channel contact quality
clean = reject.causal(block, weights=q)
decoder.update_baseline(clean)
emit(decoder.step(clean))Measure the whole loop
The single most useful thing we built was not an algorithm. It was a harness that measures true end-to-end latency by injecting a known pulse at the electrode and timestamping when the actuator responds.
Every component's documented latency, summed, told us one number. The harness told us a different one, consistently larger, because buffering between stages is invisible to the stages themselves. If you only measure your own code, you will be wrong about your system.