Researchers at the Weizmann Institute of Science have unveiled an AI tool that can infer what a person is looking at from a functional magnetic resonance imaging (fMRI) scan and then recreate the image with notable accuracy. The system, described as a “mind-reading” model, marks a leap from earlier blurry reconstructions.
How the decoder works
The team, led by Michal Irani, trained a dual-branch neural network on high-resolution fMRI data collected from eight volunteers who viewed roughly 9,000 pictures each. One branch predicts an image’s structural layout, while the other estimates its semantic content, for example, a bunch of bananas on a plate.
These predictions feed a diffusion model that iteratively refines a noisy pixel field into a coherent picture. To expand the training set beyond the limited scans, the researchers built a companion encoder that predicts brain activity from any given image. By looping the encoder and decoder together, they could generate synthetic scans for images never shown to participants, effectively amplifying the data pool.
According to the authors, the combined system can reconstruct a new subject’s visual experience after only one hour of fMRI recording, a stark contrast to earlier approaches that demanded 40 hours of scanning per individual.
Applications and ethical concerns
Neuroethicist Judy Illes of the University of British Columbia praised the work as “magnificent,” noting its therapeutic promise for people with severe motor impairments. If a locked-in patient can generate a reliable brain scan, the model might translate those signals into visual output, offering a new communication channel.
At the same time, neuroscientist Tommy Sprague of the University of California, Santa Barbara warned that the technology could be misused to extract mental imagery without consent. “But if there’s a way to surreptitiously extract information about what you’re thinking about, then …150 years of sci-fi can come true anytime, and that’s worrisome in a lot of ways,” he said.
Read Also: AI Claims First Scientific Discovery, Sparks Debate
Beyond clinical settings, the researchers identified brain regions that seem to respond consistently to certain categories: one region responded to images of food, while another responded to images of sports. These findings support the development of universal brain encoders that can operate on scans from new participants with minimal adjustment, potentially accelerating basic neuroscience research.
Despite the progress, Irani acknowledges occasional failures. A cake was once rendered as three sandwiches, and a dog in a bathtub appeared as a goat of similar hue. Such errors highlight the current limits of the approach, even as it outperforms previous models by a significant margin.
One practical advantage is the reduced calibration time. Whereas prior tools often required extensive imaging sessions, Irani’s decoder achieved usable results after just a single hour of fMRI data collection.
Expanding the training pipeline
Through a clever pairing of two AI components, the researchers generated synthetic brain scans for images that had never been shown inside a scanner. An encoder predicts the neural response to a visual stimulus, while a decoder translates that predicted activity back into an image. Repeating this loop many times gradually sharpens both models, allowing the system to learn from virtually unlimited picture sets.
Future directions and ethical environment
Beyond static pictures, the developers aim to extend the framework to moving visuals and sound, hoping to capture imagined scenes or even dream content. Such ambitions would push the decoder toward interpreting internally generated imagery rather than only externally presented stimuli.
