Meta Develops AI System to Translate Brain Activity into Text for Paralysis Patients

Meta has unveiled an advanced artificial intelligence technology capable of decoding brain waves and converting them into text, offering new communication possibilities for millions affected by paralysis and speech impairments.

On Monday, the company introduced Brain2Qwerty v2, a system that can be likened to a primitive form of "thought reading" through algorithmic processing. Although still in early research stages, this technology could soon enable patients with conditions such as speech loss, locked-in syndrome, amyotrophic lateral sclerosis (ALS), and other neurological disorders causing paralysis to communicate merely by thinking, without requiring complex and costly brain implant surgeries.

Meta stated in its announcement, "We believe this research has the potential to make a real difference for millions of people with brain injuries that prevent them from communicating."

To accelerate scientific progress in neuroscience, Meta has made the core source code of both the new system and its predecessor freely available online, allowing researchers worldwide to access and further develop the technology.

Training and Testing the Brain2Qwerty System

Researchers collaborated with the Basque Center on Cognition, Brain and Language in San Sebastián, Spain, to train the new model. Nine healthy volunteers aged between 25 and 56 participated in the experiments, during which they were asked to type over 2,500 sentences across ten sessions.

During these sessions, their brain activity was monitored using magnetoencephalography (MEG), a technique that measures the subtle electric fields produced by neural activity. The written sentences and brain scans served as raw training data for the AI system.

Accuracy and Improvements in Brain Signal Decoding

The updated system achieved a remarkable word-level decoding accuracy of 78%, indicating that more than half of the decoded sentences contained no more than one linguistic error. This represents a significant improvement over the previous version, Brain2Qwerty v1, which attained only 48% accuracy.

Researchers also observed that the system's accuracy improved with increased training data, suggesting that applying straightforward measurement principles could lead to more capable systems in the future.

The research team commented, "If extensive training on non-invasive MEG data can eventually eliminate the need for neurosurgery, it would represent a transformative shift in patient care."

Decoding Process and AI Integration

Brain2Qwerty v2 employs pattern recognition techniques similar to those used in popular chatbot programs like Meta's ChatGPT and Llama. The decoding occurs in multiple stages:

• First, brain waves measured by AI are translated into codes representing individual characters.
• Next, a separate AI system organizes these characters into complete words.
• Finally, a large language model converts the disordered characters and words into coherent and comprehensible sentences.

This marks the first successful use of a large language model to translate noisy brain activity into structured sentences, providing a valuable framework for future brain-machine interface research.

In addition to the multi-level decoding system, Brain2Qwerty incorporates a set of autonomous AI agents tasked with self-improving the decoding process to enhance accuracy and efficiency. These agents were trained to "frequently modify the codebase to invent new and better architectures," leading to a substantial reduction in word-level error rates.