data leakage

[bbb801]

Dear Sir/Madam,

I noticed in your '01 Feature Extraction and Selection.ipynb' that it first uses feature extraction and then splits the data into train and test. Is it leading to data leakage? Can we split the data first and then train the model to do feature extraction on the test set?

[nils-braun]

Hi @bbb801 - yes, you are indeed correct. This is not the best practice and therefore, our example should not include it.
Would you like to do a PR and change this? Thank you :)

[bbb801]

However, when I read your '02 sklearn Pipeline.ipynb', it seemed statistics from training can be applied to the test set. May I know these steps still lead to data leakage? I am wondering if 'ppl.set_params(augmenter__timeseries_container=df_ts);' will compute the statistics from all data (including train and test set), and thus lead to data leakage. Thank you.

[nils-braun]

No, there is no data leakage. The example in the docs gibes some more information: https://tsfresh.readthedocs.io/en/latest/api/tsfresh.transformers.html#tsfresh.transformers.relevant_feature_augmenter.RelevantFeatureAugmenter
You fit it on the train set but you predict on the test set.

[nils-braun]

Would you still like to do a PR to fix the example notebook?

[bbb801]

Thank you for your explanation. I checked your codes and found that I might not be able to fix them. I am just a beginner. Sorry.
By the way could you tell me where does 'read_in_timeseries()' come from (https://tsfresh.readthedocs.io/en/latest/api/tsfresh.transformers.html#tsfresh.transformers.relevant_feature_augmenter.RelevantFeatureAugmenter) ? I checked all of your codes and cannot import it.

[nils-braun]

The read_in_timeseries is the function you need to write. This is where you load your own data, so we can not provide this function for you :)