Adding feature extraction #77

KarsVeldkamp · 2024-11-26T11:37:50Z

Ha Erik,

Hierbij de PR voor feature extraction. Heb op het eind nog even wat dingen aangepast zodat het echt alleen relevant is voor feature extraction (en wat preprocessing/restructuring)

… into classification-ppg

… into feature_extraction_ppg

…t files

…iles

Erikpostt · 2024-11-27T07:55:22Z

src/paradigma/heart_rate/feature_extraction.py

+    l_signal_to_noise_ratios = []
+    for segment in ppg_segments:
+        arr_signal = np.var(segment)
+        arr_noise = np.var(np.abs(segment))
+        signal_to_noise_ratio = arr_signal / arr_noise
+        l_signal_to_noise_ratios.append(signal_to_noise_ratio)
+
+    return l_signal_to_noise_ratios


Je kan ook gebruik maken van Numpy arrays hier, dan hoef je niet te loopen. Ik weet niet of dit precies zo werkt, maar ongeveer. Dan is je return ook een np.ndarray ipv een list.

Suggested change

l_signal_to_noise_ratios = []

for segment in ppg_segments:

arr_signal = np.var(segment)

arr_noise = np.var(np.abs(segment))

signal_to_noise_ratio = arr_signal / arr_noise

l_signal_to_noise_ratios.append(signal_to_noise_ratio)

return l_signal_to_noise_ratios

arr_signal = np.var(ppg_segments, axis=1)

abs_signal = np.abs(ppg_segments, axis=1)

arr_noise = np.var(abs_signal, axis=1)

signal_to_noise_ratio = arr_signal / arr_noise

return signal_to_noise_ratio

Erikpostt · 2024-11-27T07:56:49Z

src/paradigma/heart_rate/feature_extraction.py

+    Compute the autocorrelation of the PPG signal.
+
+    Args:
+        ppg_signal (np.ndarray): 2D array where each row is a segment of the PPG signal.


Dtype is duidelijk hierboven. Idem voor parameters in andere functies.

Suggested change

ppg_signal (np.ndarray): 2D array where each row is a segment of the PPG signal.

ppg_segments: 2D array where each row is a segment of the PPG signal.

Erikpostt · 2024-11-27T07:59:23Z

src/paradigma/heart_rate/feature_extraction.py

+        autocorrelations = biased_autocorrelation(segment, fs*3)
+        peaks, _ = find_peaks(autocorrelations, height=0.01)
+        peak_values = autocorrelations[peaks]
+        sorted_peaks = np.sort(peak_values)[::-1]


Klopt het dat je ze hier sorteert in descending order?

yes, want wil de hoogste piek hebben

Dan zou je eventueel ook het volgende kunnen doen (bijvoorbeeld):

l_auto_correlations.append(np.max(peak_values, initial=0)) # extract the highest peak

Maar dat laat ik aan jou voor later ;)

Erikpostt · 2024-11-27T08:00:53Z

src/paradigma/heart_rate/feature_extraction.py

+    Returns:
+        np.ndarray: Biased autocorrelation values for lags 0 to max_lag.
+    """
+    x = np.array(x) # Ensure x is a numpy array instead of a list


Ik weet niet precies wat je wil, maar je zou ook gebruik kunnen maken van np.asarray: https://stackoverflow.com/questions/14415741/what-is-the-difference-between-np-array-and-np-asarray

TLDR: np.asarray maakt geen copy, wat np.array wel doet.

eens, maar zie nu eig ook dat hij overbodig is want de input is sws al een np.array

Erikpostt · 2024-11-27T08:02:04Z

src/paradigma/heart_rate/feature_extraction.py

+
+    return l_auto_correlations
+
+def biased_autocorrelation(


Je past deze functie nu per segment toe, maar je kan wellicht ook de input een 2D array maken. Alle numpy functies werken daar iig prima op (dmv axis parameter).

Erikpostt · 2024-11-27T08:03:13Z

src/paradigma/heart_rate/feature_extraction.py

+    autocorr_values = np.zeros(max_lag + 1)
+
+    for lag in range(max_lag + 1):
+        # Compute autocorrelation for current lag
+        overlapping_points = x[:N-lag] * x[lag:]
+        autocorr_values[lag] = np.sum(overlapping_points) / N  # Divide by N (biased normalization)


Wellicht kan je hier gebruik maken van np.correlate, dat wordt dan iets als: autocorr_values = np.correlate(x, x, mode='full')[N-1:N+max_lag] / N.

zou ik even later moet checken, want ben hier even mee bezig geweest om exact dezelfde output te krijgen als in matlab

Erikpostt · 2024-11-27T08:04:47Z

src/paradigma/heart_rate/feature_extraction.py

+    """
+    Calculate relative power within the dominant frequency band in the physiological range (0.75 - 3 Hz).
+    """
+    hr_range_idx = np.where((freqs >= 0.75) & (freqs <= 3))[0]


Ik zou de frequency band voor HR ergens wegschrijven in je config file.

Erikpostt · 2024-11-27T08:05:17Z

src/paradigma/heart_rate/feature_extraction.py

+    peak_idx = np.argmax(psd[hr_range_idx])
+    peak_freq = freqs[hr_range_idx[peak_idx]]
+
+    dom_band_idx = np.where((freqs >= peak_freq - 0.2) & (freqs <= peak_freq + 0.2))[0]


Is 0.2 hier een marge die je pakt? Dan zou ik ergens de margin definiëren als variabele in de functie (of in je config)

src/paradigma/heart_rate/feature_extraction.py

Erikpostt · 2024-11-27T08:08:27Z

src/paradigma/heart_rate/feature_extraction.py

+    df_windowed[f'f_dom'] = l_dominant_frequencies
+    df_windowed[f'rel_power'] = l_relative_powers
+    df_windowed[f'spectral_entropy'] = l_spectral_entropies


Ook hier en bij andere kolomnamen soms: de f is alleen nodig als je variabelen wilt includeren in de kolomnaam.

ja eens, had dat in eerste instantie wel zo staan, vergeten aan te passen, nu gedaan

Erikpostt · 2024-11-27T08:09:01Z

src/paradigma/heart_rate/feature_extraction.py

+
+    for segment in ppg_segments:
+        # Compute power spectral density (PSD) once using Welch's method
+        freqs, psd = welch(


Dit kan ook met vectorizatie dmv numpy, maar daar kunnen we later een keer naar kijken samen.

lijkt mij goed

Erikpostt · 2024-11-27T08:16:08Z

src/paradigma/heart_rate/heart_rate_analysis_config.py

+        # self.ppg_colname: List[str] = [
+        #     DataColumns.PPG
+        # ]


Dit kan weg?

src/paradigma/heart_rate/heart_rate_analysis_config.py

Erikpostt · 2024-11-27T08:19:37Z

src/paradigma/heart_rate/heart_rate_analysis_config.py

+        self.sqa_window_overlap_s: int = 5
+        self.sqa_window_step_size_s: int = 1


Deze informatie staat ook bij feature extraction, dus ik zou deze ook bij PPGConfig zetten zodat je ze maar één keer hoeft te definiëren.

src/paradigma/heart_rate/heart_rate_analysis_config.py

Erikpostt

Goed werk! Loop even de comments en suggesties na en kijk maar wat je ermee wilt doen. Ik zie geen grote dingen die per se anders moeten, dus ik kan het mergen als jij er klaar voor bent.

KarsVeldkamp · 2024-11-27T12:41:49Z

@Erikpostt Comments verwerkt ;)

Erikpostt · 2024-12-02T09:50:50Z

src/paradigma/heart_rate/feature_extraction.py

+    if statistic == 'mean':
+        return [np.mean(np.abs(x)) for x in sensor_col]
+    elif statistic == 'var':
+        return [np.var(x, ddof=1) for x in sensor_col]  # ddof=1 for unbiased variance is used, same as matlab


"Same as Matlab" is wellicht verwarrend voor een gebruiker, of niet?

Erikpostt · 2024-12-02T09:52:16Z

src/paradigma/heart_rate/heart_rate_analysis_config.py

Waarschijnlijk heb je dit van mij overgenomen, dus ik kan je dit niet kwalijk nemen, maar wellicht wil je deze als float hebben.

Erikpostt · 2024-12-02T09:52:37Z

src/paradigma/heart_rate/heart_rate_analysis_config.py

+
+        self.freq_band_physio = [0.75, 3] # Hz
+        self.bandwidth = 0.2   # Hz


Wat is dit precies?

Erikpostt · 2024-12-02T09:53:17Z

src/paradigma/heart_rate/heart_rate_analysis_config.py

+        self.sqa_window_overlap_s: int = 5
+        self.sqa_window_step_size_s: int = 1
+        min_window_length = 10


Is deze weer in seconden?

Erikpostt · 2024-12-02T09:53:30Z

src/paradigma/heart_rate/heart_rate_analysis_config.py

+
+        # Heart rate estimation parameters
+        hr_est_length = 2


Erikpostt

Ziet er prima uit! Heb wat vragen gesteld over je code, maar dit zit de merge niet in de weg. Kleine suggesties wellicht voor de volgende keer.

KarsVeldkamp added 10 commits October 14, 2024 10:24

Folder structure updates + feature extraction

45cbfa3

Merge branch 'main' of https://github.com/biomarkersParkinson/paradigma…

c9a8619

… into classification-ppg

Merge branch 'main' of https://github.com/biomarkersParkinson/paradigma…

717d4c7

… into feature_extraction_ppg

Merge branch 'main' of https://github.com/biomarkersParkinson/paradigma…

ba4e57b

… into feature_extraction_ppg

Merge branch 'main' of https://github.com/biomarkersParkinson/paradigma…

7f00725

… into feature_extraction_ppg

Merge branch 'main' of https://github.com/biomarkersParkinson/paradigma…

e192f69

… into feature_extraction_ppg

adding feature extraction for PPG

67049bc

delete files based on folder renaming and restructuring

adf84bd

change preprocessing for equal outputs and to relative time for ppg

fe638eb

adding feature extraction in heart_rate_analysis and updating relevan…

0b18e47

…t files

KarsVeldkamp requested a review from Erikpostt November 26, 2024 11:37

KarsVeldkamp assigned Erikpostt and KarsVeldkamp Nov 26, 2024

KarsVeldkamp added 3 commits November 26, 2024 12:44

remove unnecessary signal_quality_predictions call

af896b8

update heart_rate_analysis.py with only relevant feature_extraction f…

c0845a9

…iles

change expected outputs to lists

f530a30

Erikpostt reviewed Nov 27, 2024

View reviewed changes

src/paradigma/heart_rate/feature_extraction.py Outdated Show resolved Hide resolved

Erikpostt reviewed Nov 27, 2024

View reviewed changes

src/paradigma/heart_rate/feature_extraction.py Outdated Show resolved Hide resolved

Erikpostt reviewed Nov 27, 2024

View reviewed changes

src/paradigma/heart_rate/feature_extraction.py Outdated Show resolved Hide resolved

Erikpostt reviewed Nov 27, 2024

View reviewed changes

src/paradigma/heart_rate/feature_extraction.py Outdated Show resolved Hide resolved

Erikpostt reviewed Nov 27, 2024

View reviewed changes

src/paradigma/heart_rate/heart_rate_analysis_config.py Outdated Show resolved Hide resolved

Erikpostt reviewed Nov 27, 2024

View reviewed changes

src/paradigma/heart_rate/heart_rate_analysis_config.py Outdated Show resolved Hide resolved

Erikpostt reviewed Nov 27, 2024

View reviewed changes

src/paradigma/heart_rate/heart_rate_analysis_config.py Outdated Show resolved Hide resolved

Erikpostt reviewed Nov 27, 2024

View reviewed changes

biomarkersParkinson deleted a comment from Erikpostt Nov 27, 2024

KarsVeldkamp added 3 commits November 27, 2024 11:01

update feature extraction based on comments

c9084da

update config

39bd17c

update config feature_extraction.py

3671b37

KarsVeldkamp closed this Nov 27, 2024

KarsVeldkamp reopened this Nov 27, 2024

clear execution count

2cfa7d7

Erikpostt reviewed Dec 2, 2024

View reviewed changes

src/paradigma/heart_rate/heart_rate_analysis_config.py

# Heart rate estimation parameters

hr_est_length = 2

Copy link

Contributor

Erikpostt Dec 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

En deze?

Erikpostt approved these changes Dec 2, 2024

View reviewed changes

Merge branch 'main' into feature_extraction_ppg

1e62acd

Erikpostt merged commit 9e68513 into main Dec 2, 2024
1 check passed

KarsVeldkamp deleted the feature_extraction_ppg branch December 3, 2024 11:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding feature extraction #77

Adding feature extraction #77

KarsVeldkamp commented Nov 26, 2024 •

edited

Loading

Erikpostt Nov 27, 2024 •

edited

Loading

Erikpostt Nov 27, 2024 •

edited

Loading

Erikpostt Nov 27, 2024

KarsVeldkamp Nov 27, 2024

Erikpostt Nov 27, 2024 •

edited

Loading

Erikpostt Nov 27, 2024 •

edited

Loading

KarsVeldkamp Nov 27, 2024

Erikpostt Nov 27, 2024

Erikpostt Nov 27, 2024

KarsVeldkamp Nov 27, 2024

Erikpostt Nov 27, 2024

Erikpostt Nov 27, 2024

Erikpostt Nov 27, 2024

KarsVeldkamp Nov 27, 2024

Erikpostt Nov 27, 2024

KarsVeldkamp Nov 27, 2024

Erikpostt Nov 27, 2024

KarsVeldkamp Nov 27, 2024

Erikpostt Nov 27, 2024

Erikpostt left a comment •

edited

Loading

KarsVeldkamp commented Nov 27, 2024

Erikpostt Dec 2, 2024

Erikpostt Dec 2, 2024

Erikpostt Dec 2, 2024

Erikpostt Dec 2, 2024

Erikpostt Dec 2, 2024

Erikpostt left a comment

	ppg_signal (np.ndarray): 2D array where each row is a segment of the PPG signal.
	ppg_segments: 2D array where each row is a segment of the PPG signal.

		self.sqa_window_overlap_s: int = 5
		self.sqa_window_step_size_s: int = 1


		self.freq_band_physio = [0.75, 3] # Hz
		self.bandwidth = 0.2 # Hz

Adding feature extraction #77

Adding feature extraction #77

Conversation

KarsVeldkamp commented Nov 26, 2024 • edited Loading

Erikpostt Nov 27, 2024 • edited Loading

Choose a reason for hiding this comment

Erikpostt Nov 27, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Erikpostt Nov 27, 2024 • edited Loading

Choose a reason for hiding this comment

Erikpostt Nov 27, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Erikpostt left a comment • edited Loading

Choose a reason for hiding this comment

KarsVeldkamp commented Nov 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Erikpostt left a comment

Choose a reason for hiding this comment

KarsVeldkamp commented Nov 26, 2024 •

edited

Loading

Erikpostt Nov 27, 2024 •

edited

Loading

Erikpostt Nov 27, 2024 •

edited

Loading

Erikpostt Nov 27, 2024 •

edited

Loading

Erikpostt Nov 27, 2024 •

edited

Loading

Erikpostt left a comment •

edited

Loading