Error with feature selection.

[renzha-miun]

I am trying to extract features for machine learning using tsfresh. It worked for feature extraction. However, when I try to do feature selection, an error popped up:

Traceback (most recent call last):
File "/Users/renzha/Library/Application Support/JetBrains/PyCharmCE2021.1/scratches/pywavelet.py", line 65, in
filtered_features = select_features(X, y)
File "/Users/renzha/PycharmProjects/pythonProject2/venv/lib/python3.8/site-packages/tsfresh/feature_selection/selection.py", line 153, in select_features
assert len(X) == len(y), "X and y must contain the same number of samples."
AssertionError: X and y must contain the same number of samples.

It seems the X and y have different shapes.

df1 => (23400, 3)
y => (23400,)
X => (26, 787)

What should I do to correct the error? Thank you.

Here is the code:

df1 = pd.DataFrame(np.concatenate(HBT_data))
df1.columns = ['id','time','signal']
df1 = df1.astype({col:float for col in ['signal']}, {col:int for col in ['time']})
y = df1.iloc[:,0].values
X = extracted_features = extract_features (df1, column_id="id", column_sort='time', n_jobs=0)
print(extracted_features.shape)
from tsfresh import select_features
from tsfresh.utilities.dataframe_functions import impute
impute(X)
filtered_features = select_features(X, y)
print(filtered_features.head())

Here is the input data (id is the 26 English letters):

     id time    signal
0     j    0  1.769988
1     j    1  1.792561
2     j    2  1.811032
3     j    3  1.806369
4     j    4  1.790852
...  ..  ...       ...
2595  p   95  2.068064
2596  p   96  2.107263
2597  p   97  2.154644
2598  p   98  2.190007
2599  p   99  2.205297

[nils-braun]

Hi @renzha-miun!
tsfresh will extract one set of features (= one row in the output dataframe) per time series you give to it - which means one per unique ID. It will treat all data for the same ID as belonging together (and forming one time series).
So as you have 26 IDs, your resulting dataframe will have 26 rows.
This is optimized for the typical classification use case: you want to predict a single class for each time series (or ID). So we assume you have as many entries in the target vector as you have time series (and IDs).

If you want to do a prediction use case (which would fit to having as many target entries as time steps), you need to "roll" the data first. Have a look into our documentation