Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New feature for removing heart artifacts from EEG or ESG data using a Principal Component Analysis - Optimal Basis Sets (PCA-OBS) algorithm #13037

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
2013fb4
feat: add initial source code
emma-bailey Sep 1, 2024
6d5f5b2
Minimum working example with local data
emma-bailey Sep 6, 2024
3622540
Implement testing dataset
emma-bailey Sep 9, 2024
6c42f33
Feat: Update examples
emma-bailey Oct 7, 2024
38b6b6a
Merge pull request #2 from mne-tools/main
steinnhauser Oct 23, 2024
2fe6a88
Merge pull request #1 from emma-bailey/feat/pca-obs
steinnhauser Oct 23, 2024
61f58b2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 23, 2024
070037d
fix: adjust import paths, add init file to module
steinnhm Oct 23, 2024
e1b6732
refactor: rearrange PCA_OBS arg structure, remove kwarg 'sr'
steinnhm Oct 23, 2024
ccd4298
refactor: move common variables to module init, further cleanup
steinnhm Oct 23, 2024
61a92ea
feat/initial-cleanup: Remove custom pchip as not in use
emma-bailey Oct 25, 2024
a1eb6f6
refactor/initial-cleanup: answer question
emma-bailey Oct 25, 2024
d75e927
refactor: rename files to match other modules in preprocessing, reduc…
steinnhm Nov 4, 2024
035a9f6
refactor: remove unused fit_ecgTemplate variable 'baselinerange'
steinnhm Nov 4, 2024
2debb61
refactor: make main methods private, import from module __init__, rem…
steinnhm Nov 4, 2024
97f9a15
refactor: add docstring templates, gather imports
steinnhm Nov 4, 2024
34a2e06
test: add placeholder tests, add multiple todos to address where we g…
steinnhm Nov 4, 2024
3760c02
Removing extra examples
emma-bailey Nov 11, 2024
0a55712
Moving example
emma-bailey Nov 11, 2024
e887ace
Adding Niazy reference
emma-bailey Nov 11, 2024
ee3b73e
Update example, put pca-obs in a single file
emma-bailey Nov 11, 2024
adae352
TODO
emma-bailey Nov 11, 2024
b3c1b4e
Update example
emma-bailey Nov 11, 2024
ba36dc2
Merge pull request #4 from mne-tools/main
steinnhauser Nov 15, 2024
e6aa17d
refactor: change way of calling pca obs method, add comments
steinnhm Nov 15, 2024
b150b6c
refactor: move pca obs method out of separate python module, change l…
steinnhm Nov 15, 2024
951e87c
Refactor: Docstrings, removed classes
emma-bailey Nov 15, 2024
c70db0a
refactor: remove unrequired index references, adjust variable namings…
steinnhm Nov 24, 2024
fb8c7ff
docs: minor edits in docstrings
steinnhm Nov 24, 2024
4c73a96
fix: add minor sanity checks for the function inputs
steinnhm Nov 25, 2024
8a0d73c
Merge pull request #3 from emma-bailey/refactor/initial-cleanup
steinnhauser Nov 25, 2024
e69039b
test: add initial test structure, missing validation of post-hear-art…
steinnhm Nov 25, 2024
4bd3a51
style: run pre-commit hooks on test file
steinnhm Nov 25, 2024
07458ba
docs: remove duplicated docstring
steinnhm Nov 25, 2024
b2be499
Merge branch 'mne-tools:main' into main
steinnhauser Nov 25, 2024
a5e68d7
Removed filter_coords from within the method
emma-bailey Nov 27, 2024
1938573
Adding info to filter to example
emma-bailey Nov 27, 2024
19e0802
style: run import sorter pre-commit hook
steinnhm Dec 4, 2024
49a96d0
refactor,test: migrate data shape to be 1d, add sanity checks for PCA…
steinnhm Dec 4, 2024
a6e11fc
Merge branch 'mne-tools:main' into main
steinnhauser Dec 4, 2024
925b293
example:Reshape ecg_events before pca_obs, tests:ecg_events is in sam…
emma-bailey Dec 6, 2024
a3d1123
example: update channel selection
emma-bailey Dec 6, 2024
e36e665
Merge branch 'mne-tools:main' into main
steinnhauser Dec 18, 2024
2ae84b0
refactor,test: change public qrs kwarg to be more clear about being i…
steinnhm Dec 18, 2024
b8d8d7c
Merge pull request #5 from emma-bailey/tests/general-pca-obs
steinnhauser Dec 18, 2024
d19049b
style: move fit_ecg_template function to bottom of file to improve re…
steinnhm Dec 18, 2024
3e9a812
Merge branch 'mne-tools:main' into main
steinnhauser Dec 19, 2024
826afec
test: add pytest importorskip for pandas lib in pca obs tests
steinnhm Dec 19, 2024
16937c8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 19, 2024
8f4dcdd
docs: add apply_pca_obs algorithm to doc/api/preprocessing.rst list
steinnhm Dec 19, 2024
35ab331
Merge branch 'main' of https://github.com/emma-bailey/mne-python
steinnhm Dec 19, 2024
121f8dd
Merge branch 'mne-tools:main' into main
steinnhauser Dec 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/api/preprocessing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ Projections:
read_ica_eeglab
read_fine_calibration
write_fine_calibration
apply_pca_obs

:py:mod:`mne.preprocessing.nirs`:

Expand Down
10 changes: 10 additions & 0 deletions doc/references.bib
Original file line number Diff line number Diff line change
Expand Up @@ -1335,6 +1335,16 @@ @inproceedings{NdiayeEtAl2016
year = {2016}
}

@article{NiazyEtAl2005,
author = {Niazy, R. K. and Beckmann, C.F. and Iannetti, G.D. and Brady, J. M. and Smith, S. M.},
title = {Removal of FMRI environment artifacts from EEG data using optimal basis sets},
journal = {NeuroImage},
year = {2005},
volume = {28},
pages = {720-737},
doi = {10.1016/j.neuroimage.2005.06.067.}
}

@article{NicholsHolmes2002,
author = {Nichols, Thomas E. and Holmes, Andrew P.},
doi = {10.1002/hbm.1058},
Expand Down
212 changes: 212 additions & 0 deletions examples/preprocessing/esg_rm_heart_artefact_pcaobs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
"""
.. _ex-pcaobs:

=====================================================================================
Principal Component Analysis - Optimal Basis Sets (PCA-OBS) removing cardiac artefact
=====================================================================================

This script shows an example of how to use an adaptation of PCA-OBS
:footcite:`NiazyEtAl2005`. PCA-OBS was originally designed to remove
the ballistocardiographic artefact in simultaneous EEG-fMRI. Here, it
has been adapted to remove the delay between the detected R-peak and the
ballistocardiographic artefact such that the algorithm can be applied to
remove the cardiac artefact in EEG (electroencephalogrpahy) and ESG
(electrospinography) data. We will illustrate how it works by applying the
algorithm to ESG data, where the effect of removal is most pronounced.

See: https://www.biorxiv.org/content/10.1101/2024.09.05.611423v1
for more details on the dataset and application for ESG data.

"""

# Authors: Emma Bailey <[email protected]>,
# Steinn Hauser Magnusson <[email protected]>
# License: BSD-3-Clause
# Copyright the MNE-Python contributors.

import glob

import numpy as np

###############################################################################
# Download sample subject data from OpenNeuro if you haven't already
# This will download simultaneous EEG and ESG data from a single participant after
# median nerve stimulation of the left wrist
# Set the target directory to your desired location
import openneuro as on
from matplotlib import pyplot as plt

import mne
from mne import Epochs, concatenate_raws, events_from_annotations
from mne.io import read_raw_eeglab
from mne.preprocessing import find_ecg_events, fix_stim_artifact

# add the path where you want the OpenNeuro data downloaded. Files total around 8 GB
# target_dir = "/home/steinnhm/personal/mne-data"
target_dir = "/data/pt_02569/test_data"

file_list = glob.glob(target_dir + "/sub-001/eeg/*median*.set")
if file_list:
print("Data is already downloaded")
else:
on.download(
dataset="ds004388", target_dir=target_dir, include="sub-001/*median*_eeg*"
)

###############################################################################
# Define the esg channels (arranged in two patches over the neck and lower back)
# Also include the ECG channel for artefact correction
esg_chans = [
"S35",
"S24",
"S36",
"Iz",
"S17",
"S15",
"S32",
"S22",
"S19",
"S26",
"S28",
"S9",
"S13",
"S11",
"S7",
"SC1",
"S4",
"S18",
"S8",
"S31",
"SC6",
"S12",
"S16",
"S5",
"S30",
"S20",
"S34",
"S21",
"S25",
"L1",
"S29",
"S14",
"S33",
"S3",
"L4",
"S6",
"S23",
]

# Sampling rate
fs = 1000

# Interpolation window for ESG data to remove stimulation artefact
tstart_esg = -0.007
tmax_esg = 0.007

# Define timing of heartbeat epochs
iv_baseline = [-400 / 1000, -300 / 1000]
iv_epoch = [-400 / 1000, 600 / 1000]

###############################################################################
# Read in each of the four blocks and concatenate the raw structures after performing
# some minimal preprocessing including removing the stimulation artefact, downsampling
# and filtering
block_files = glob.glob(target_dir + "/sub-001/eeg/*median*.set")
block_files = sorted(block_files)

for count, block_file in enumerate(block_files):
raw = read_raw_eeglab(
block_file, eog=(), preload=True, uint16_codec=None, verbose=None
)

# Isolate the ESG channels (including ECG for R-peak detection)
raw.pick(esg_chans + ["ECG"])

# Find trigger timings to remove the stimulation artefact
events, event_dict = events_from_annotations(raw)
trigger_name = "Median - Stimulation"

fix_stim_artifact(
raw,
events=events,
event_id=event_dict[trigger_name],
tmin=tstart_esg,
tmax=tmax_esg,
mode="linear",
stim_channel=None,
)

# Downsample the data
raw.resample(fs)

# Append blocks of the same condition
if count == 0:
raw_concat = raw
else:
concatenate_raws([raw_concat, raw])

###############################################################################
# Find ECG events and add to the raw structure as event annotations
ecg_events, ch_ecg, average_pulse = find_ecg_events(raw_concat, ch_name="ECG")
ecg_event_samples = np.asarray(
[[ecg_event[0] for ecg_event in ecg_events]]
) # Samples only

qrs_event_time = [
x / fs for x in ecg_event_samples.reshape(-1)
] # Divide by sampling rate to make times
duration = np.repeat(0.0, len(ecg_event_samples))
description = ["qrs"] * len(ecg_event_samples)

raw_concat.annotations.append(
qrs_event_time, duration, description, ch_names=[esg_chans] * len(qrs_event_time)
)

###############################################################################
# Create evoked response about the detected R-peaks before cardiac artefact correction
# Apply PCA-OBS to remove the cardiac artefact
# Create evoked response about the detected R-peaks after cardiac artefact correction
events, event_ids = events_from_annotations(raw_concat)
event_id_dict = {key: value for key, value in event_ids.items() if key == "qrs"}
epochs = Epochs(
raw_concat,
events,
event_id=event_id_dict,
tmin=iv_epoch[0],
tmax=iv_epoch[1],
baseline=tuple(iv_baseline),
)
evoked_before = epochs.average()

# Apply function - modifies the data in place. Optionally high-pass filter
# the data before applying PCA-OBS to remove low frequency drifts
mne.preprocessing.apply_pca_obs(
raw_concat, picks=esg_chans, n_jobs=5, qrs_indices=ecg_event_samples.reshape(-1)
)

epochs = Epochs(
raw_concat,
events,
event_id=event_id_dict,
tmin=iv_epoch[0],
tmax=iv_epoch[1],
baseline=tuple(iv_baseline),
)
evoked_after = epochs.average()

###############################################################################
# Comparison image
fig, axes = plt.subplots(1, 1)
axes.plot(evoked_before.times, evoked_before.get_data().T, color="black")
axes.plot(evoked_after.times, evoked_after.get_data().T, color="green")
axes.set_ylim([-0.0005, 0.001])
axes.set_ylabel("Amplitude (V)")
axes.set_xlabel("Time (s)")
axes.set_title("Before (black) vs. After (green)")
plt.tight_layout()
plt.show()

# %%
# References
# ----------
# .. footbibliography::
2 changes: 2 additions & 0 deletions mne/preprocessing/__init__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ __all__ = [
"realign_raw",
"regress_artifact",
"write_fine_calibration",
"apply_pca_obs",
]
from . import eyetracking, ieeg, nirs
from ._annotate_amplitude import annotate_amplitude
Expand Down Expand Up @@ -85,6 +86,7 @@ from .maxwell import (
maxwell_filter_prepare_emptyroom,
)
from .otp import oversampled_temporal_projection
from .pca_obs import apply_pca_obs
from .realign import realign_raw
from .ssp import compute_proj_ecg, compute_proj_eog
from .stim import fix_stim_artifact
Expand Down
Loading
Loading