Skip to content

Re-implementation of "Decoding speech from non-invasive brain recordings"

Notifications You must be signed in to change notification settings

arayabrain/speech-decoding

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Reimplementation of speech decoding paper by MetaAI

Paper: https://arxiv.org/pdf/2208.12266.pdf

Status

Works for Gwilliams2022 dataset and Brennan2018 dataset.

TODOs

  • Full reproducibility support. Will be useful for HP tuning.
  • Match accuracy to numbers reported in the paper.

Usage

For EEG (Brennan et al. 2022)

Run python train.py dataset=Brennan2018 rebuild_datasets=True. When rebuild_datasets=False, existing pre-processed M/EEG and pre-computing embeddings are used. This is useful if you want to run the model on exactly the same data and embeddings several times. Otherwise, the both audio embeddings are pre-computed and M/EEG data are pre-processed before training begins.

For MEG (Gwilliams et al. 2022)

Run python train.py dataset=Gwilliams2022 rebuild_datasets=True When rebuild_datasets=False, existing pre-processed M/EEG and pre-computing embeddings are used. This is useful if you want to run the model on exactly the same data and embeddings several times. It takes ~30 minutes to pre-process Gwilliams2022 and compute embeddings on 20 cores. Set rebuild_datasets=False for subsequent runs (or don't specify it, becuase by default rebuild_datasets=False). Otherwise, the both audio embeddings are pre-computed and M/EEG data are pre-processed before training begins.

Monitoring training progress with W&B

To do that, set entity and project in the wandb section of config.yaml.

Datasets

Gwilliams et al., 2022

Brennan et al., 2019

You will need S01.mat to S49.mat placed under data/Brennan2018/raw/ to run the code.

About

Re-implementation of "Decoding speech from non-invasive brain recordings"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%