What is Real-Time Processing?

The ability of a BCI system to acquire, decode, and act on neural signals within a time frame fast enough for the user to perceive continuous, responsive control — typically under 50 milliseconds of end-to-end latency.

What category does Real-Time Processing belong to in BCI?

Real-Time Processing is a Signal Processing concept in brain-computer interface science.

Real-Time Processing — BCI Glossary

Real-time processing in BCI refers to the requirement that the entire signal processing pipeline — from neural signal acquisition through decoding to device output — operates fast enough that the user experiences smooth, responsive control. For cursor-control BCIs, this means updating the cursor position at 50-100 Hz (every 10-20 ms). For speech BCIs, it means generating words within tens of milliseconds of the user's neural speech attempt.

Latency Budget

A typical BCI real-time processing pipeline allocates its latency budget across stages:

Signal acquisition: 0.5-2 ms (amplification, filtering, ADC conversion)
Data transmission: 1-5 ms (wireless link from implant to external processor, or wired connection)
Feature extraction: 1-5 ms (spike detection, threshold crossing counting, spectral analysis)
Decoding: 2-20 ms (Kalman filter: <1 ms; RNN: 2-5 ms; transformer: 5-20 ms)
Output rendering: 1-10 ms (cursor movement, speech synthesis, robotic arm command)

Total end-to-end latency should be under 50 ms for cursor control and under 100 ms for speech synthesis to feel natural. Latencies above 200-300 ms cause noticeable lag that degrades user performance and satisfaction.

Computational Platforms

External workstations: BrainGate's research systems use desktop computers running custom real-time software (xPC Target, custom C++ code) connected via wired percutaneous cables. High computational power, no size constraints.
Embedded processors: Neuralink's N1 performs initial signal processing (spike detection, compression) on an implanted ASIC, then transmits compressed data wirelessly to an external device for full decoding.
Edge devices: The trend toward wireless BCIs requires decoding on mobile or wearable processors (smartphones, dedicated wearable computers) rather than desktop systems.

Challenges

Real-time processing constrains decoder complexity. A deep learning decoder that achieves the highest offline accuracy may be too slow for real-time use on embedded hardware. This creates a tradeoff between decoder sophistication and latency. Model compression, quantization, and efficient architectures (distilled transformers, efficient RNNs) are active areas of research to bring state-of-the-art decoders within real-time budgets.

Closed-Loop Considerations

For closed-loop BCIs that deliver stimulation based on decoded neural states, real-time processing is even more critical. Seizure detection systems (NeuroPace RNS) must detect and respond to seizure onset within milliseconds. Adaptive DBS must adjust stimulation parameters in real time based on oscillatory biomarkers. The processing latency directly determines the system's ability to intervene before pathological neural states propagate.