Set sample split error

[hazelkang]

I am splitting the data with set_sample_splitting. I only get this error when I manually sample split and don't get it with the same data when I don't set sample split - wonder if this function is not working.

smpls = (train_idx, test_idx) # Correctly formatted tuple

Set the sample splitting for the DoubleMLPLR object

dml_plr_obj.set_sample_splitting(smpls)
dml_plr_obj.fit()

but I keep on getting this below error after fitting:
/usr/local/lib/python3.10/dist-packages/sklearn/utils/validation.py in _assert_all_finite_element_wise(X, xp, allow_nan, msg_dtype, estimator_name, input_name)
173 "#estimators-that-handle-nan-values"
174 )
--> 175 raise ValueError(msg_err)
176
177

ValueError: Input contains NaN.

[SvenKlaassen]

Thank you for reporting this.
Could you post a more concrete example? Especially, which version you are using?
In the current version development version only cross-fitting is supported such that you would require to additionally add an additional split.

[hazelkang]

Thank you for your response.
I'm using the below version for doubleml/python.

Name: DoubleML
Version: 0.7.1
Summary: Double Machine Learning in Python
Home-page: https://docs.doubleml.org/
Author: Bach, P., Chernozhukov, V., Kurz, M. S., and Spindler, M.
Author-email: 
License: UNKNOWN
Location: /usr/local/lib/python3.10/dist-packages
Requires: joblib, numpy, pandas, plotly, scikit-learn, scipy, statsmodels
Required-by:

Below is the code I'm running that I run into the value error mentioned above.
I basically want to 1) NOT use cross-fitting 2) designate sample split externally.
With the same dataset when I run the code by 1) using cross-fitting=True 2) Not designating sample split externally - This works without error. Wondering why this is so - if the version doesn't support it or I'm doing something wrong?
I do need to designate sample split externally, because I need the train, test dataset to include same level of clusters for my analysis.

    # DoubleML setup
    data = dml.DoubleMLData.from_arrays(X_combined, Y, d, use_other_treat_as_covariate=True)
    dml_plr_obj = dml.DoubleMLPLR(data, ml_g, ml_m, apply_cross_fitting=False, n_folds=2, draw_sample_splitting = False)

    # Set the sample splitting for the DoubleMLPLR object
    gss = GroupShuffleSplit(n_splits=1, test_size=0.5, random_state=35)
    id_train, id_test = next(gss.split(X_combined, y, groups = group_labels))
    smpls = (id_train, id_test)

    # Set the sample splitting for the DoubleMLPLR object
    dml_plr_obj.set_sample_splitting(smpls)
    dml_fit = dml_plr_obj.fit()

[SvenKlaassen]

I tried to replicate you problem but on 0.7.1 this works

import doubleml as dml
import numpy as np
from doubleml.datasets import make_plr_CCDDHNR2018

from sklearn.ensemble import RandomForestRegressor
from sklearn.base import clone
from sklearn.model_selection import train_test_split

print(f"DoubleML Version: {dml.__version__}")

learner = RandomForestRegressor(n_estimators=100, max_features=20, max_depth=5, min_samples_leaf=2)
ml_l = clone(learner)
ml_m = clone(learner)

np.random.seed(1234)
obj_dml_data = make_plr_CCDDHNR2018(alpha=0.5, n_obs=100)


np.random.seed(314)
dml_plr_obj_external = dml.DoubleMLPLR(
    obj_dml_data,
    ml_l,
    ml_m,
    n_folds = 2,
    apply_cross_fitting=False,
    draw_sample_splitting=False)
 
smpls = train_test_split(np.arange(obj_dml_data.n_obs), train_size=0.8)
print(f"Samples: {smpls}")
dml_plr_obj_external.set_sample_splitting(tuple(smpls));
print(dml_plr_obj_external.fit().summary)

with output

DoubleML Version: 0.7.1
Samples: [array([28, 49, 84, 98, 59, 48, 56,  6, 29, 67, 19, 14, 34, 88, 55, 39, 27,
       74, 97,  3, 17, 70, 44, 41,  2, 94,  1, 30, 26, 60, 33, 69, 75, 76,
       47, 38, 21, 81, 18, 11, 87, 46, 65, 72, 85, 89, 63, 79, 37, 57, 91,
        0, 50, 99, 36, 90, 66, 82, 54, 61,  9, 16, 45, 68, 32,  4, 93, 22,
       35, 80, 71, 96, 58,  7, 78, 42, 86, 51, 73,  8]), array([83, 24, 92, 95, 13, 52, 23, 15, 12, 20, 25, 64, 53, 40, 10, 31, 77,
       62, 43,  5])]
       coef   std err         t     P>|t|     2.5 %    97.5 %
d  0.604704  0.143514  4.213563  0.000025  0.323422  0.885986

You could also try using external predictions. With version 0.8.1 cross-fitting will also be required (and external predictions can be used to not use cross fitting).