From thought to text: AI transforms silent speech into written words

 A new artificial intelligence system, a semantic decoder, can convert brain activity into a continuous text. The system could revolutionize communication for people who are unable to speak due to conditions like stroke.

This non-invasive approach uses fMRI scanner data to turn thoughts into text without the need for surgical implants. While this AI system isn't perfect, it successfully captures the essence of a person's thoughts half the time.

{toc}

Key facts:

1. The University of Texas at Austin researchers developed the AI Semantic Decoder.
2. It operates on a transformer model similar to that used by ChatGPT Open AI and Bard Google.
3. The system has the potential to be used with more portable brain imaging systems such as near-infrared functional spectroscopy (fNIRS).

A new artificial intelligence system called a semantic decoder can transform a person's brain activity - while listening to a story or imagining a story - into a continuous stream of text.

The system may help people who are conscious but unable to speak physically, such as those who are debilitated after a stroke.

Unlike other language decoding systems under development, this system does not require subjects to have surgical implants, making the process non-invasive.

Brain activity is measured using an fMRI scanner after extensive decoder training, in which a person listens to podcasts for hours on the scanner.

Later, provided the participant is ready to decipher their thoughts, their listening to a new story, or imagining telling a story, allows the machine to generate the appropriate text based on brain activity alone.

“For a non-invasive method, this is a real leap forward compared to what was done before, when single words or short sentences were usually used,” Huth said. "We get a model for decoding a continuous language over long periods of time with complex ideas."

The result is not a verbatim transcript. Instead, researchers designed it to capture the essence of what is being said or thought about, albeit imperfectly. About half the time when a decoder is trained to track a participant's brain activity, the machine produces text that closely (and sometimes exactly) matches the intended meaning of the original words.

For example, in experiments, a participant who listened to someone say "I don't have a driver's license yet" translated their thoughts as "She hasn't even started learning to drive yet." Listening to the words: “I didn’t know whether to scream, cry or run away. Instead, I said, “Leave me alone!” which was transcribed as “Started screaming and crying, and then she just said, “I told you to leave me alone.”

In addition to having the participants listen or think about the stories, the researchers asked the subjects to watch four short videos without sound while in the scanner. The semantic decoder was able to use their brain activity to accurately describe certain events from the video.

The system is currently not practical for use outside of the laboratory due to its time dependency on the fMRI machine. But the researchers believe this work could be transferred to other, more portable brain imaging systems, such as near-infrared functional spectroscopy (fNIRS).

So here's a review. I did it for a decent time, and if you liked it, then subscribe to my tg, there are a lot of interesting things coming out every day =)

https://t.me/diakob_net

You might like

Show more

You might like

Show more

Add To Cart

Buy Now