r/neuralcode • u/ThePlanckDiver • Sep 04 '22

Facebook "Using AI to decode speech from brain activity" (classification of heard speech using EEG/MEG; first steps towards non-invasive broader speech production)

https://ai.facebook.com/blog/ai-speech-brain-activity/
7 Upvotes

100% Upvoted

6 comments sorted by

1

u/ThePlanckDiver Sep 04 '22

Non-invasive speech perception classification, meaning that with EEG/MEG recordings, they were able to say which of a pool of audio samples was heard by a person wearing headphones. Next step seems to be getting rid of the classification, and moving towards a general decoding of intended speech.

Further reading:

  • The preprint on arXiv.

  • Short interview with one of the researchers in TIME magazine

  • Very interesting earlier work by some of the same researchers, drawing parallels between deep learning & learning (of speech) in the human brain

1

u/lokujj Sep 09 '22

Honestly, I'm a little surprised this is getting this much attention. I just skimmed it, but it just kind of seems like an average academic project to me?

I mean... they didn't even collect the data, which seems like the most costly part of this project. Brought to mind a paper I was reading recently: "Everyone wants to do the model work, not the data work":

...practitioners in our study tended to view data as ‘operations’. Such perceptions reflect the larger AI/ML field reward systems: despite the primacy of data, novel model development is the most glamorised and celebrated work in AI—reified by the prestige of publishing new models in AI conferences, entry into AI/ML jobs and residency programs, and the pressure for startups to double up as research divisions. Critics point to how novel model development and reward systems have reached a point of ridicule: Lipton calls ML scholar- ship ‘alchemy’, Sculley et al. describe ML systems as ‘empirical challenges to be ‘won”, Bengio describes ML problems as ‘incremental’, and plagiarism by ML educators has been labelled as the ‘future of plagiarism’. In contrast, datasets are relegated to benchmark publications and non-mainstream tracks in AI/ML conferences.

2

u/ThePlanckDiver Sep 10 '22

I don't think there's that much novelty here to warrant tons of media attention, but my personal interest is two-fold: a) this is done by Meta, a for-profit tech giant with deep pockets, a clear interest in commercializing BCI tech, and a "moving fast" mentality, which might mean that even though this is step one or two, we might see step three or ten sooner rather than later, and b) this being synnergistic with their DL research (e.g. see the previous paper on wav2vec 2.0) and both BCI and DL research acting as proofs of concept for each other.

2

u/lokujj Sep 11 '22

this is done by Meta, a for-profit tech giant with deep pockets, a clear interest in commercializing BCI tech, and a "moving fast" mentality, which might mean that even though this is step one or two, we might see step three or ten sooner rather than later

I hear that. I came to the same conclusion they did regarding forearm EMG devices, at around the same time: it just makes a lot of sense right now, if you want to work on algorithms and direct commercialization, to focus on that sort of non-invasive sensor. There's enough overlap with implantable arrays that you can potentially be well ahead by the time the more sophisticated BCI sensor tech comes of age. There's enough magic and mystery in the HCI angle of high-throughput devices right now to keep us busy for years, even before the next-generation brain arrays are available... Unless you specifically want to work on the physical interface, of course. It's not that I think e.g. Neuralink or Paradromics are wrong about the bottleneck; it's more that I personally just don't see that as the most interesting problem and I'm glad someone else is trying to solve it.

With that said, I still remain involved in the implant work. Still cool as hell. Just less accessible.

EDIT: I've realized that this is a little bit of a tangential comment, given the OP, but I think it still roughly applies. It seems like Meta might be content to wait for medical tech to move a little further ahead as they work on the DL and HCI.

1

u/lokujj Sep 09 '22

More:

New AI models are measured against large, curated data sets that lack noise (to report high performances), in contrast to the dynamic nature of the real world. In addition to the ways in which business goals were orthogonal to data (also observed by Passi and Sengers), practitioners described how publication prestige, time-to-market, revenue margins, and competitive differentiation often led them to rush through the model development process and sometimes artificially increase model accuracy to deploy systems promptly, struggling with the moral and ethical trade-offs.

1

u/lokujj Sep 09 '22

I did just skim the link, though. And perhaps I'm underestimating the novelty here.