Voting methods for feature ranking in efs

[bblodfon]

• Use fastVoteR, where 4 voting theory methods are now implemented in Rcpp
• Add embedded_ensemble_fselect()
• Refactoring/Simplified code on both ensemble feature selection functions and EnsembleFSResult()

[be-marc]

Can we say that differently? Scoring on a train set sounds wrong. Is this the outer train set which is split by the inner resampling? We score the inner resample result?

[bblodfon]

Yes, its the outer train set. The inner_resampling generates N train/test splits. The inner_measure is used to optimize/tune on the train set and you get the best subset and final model + score on that train set. We use these final models to also score the corresponding test splits (the inner resampling result you ask), with the measure. In embedded efs we only do the second (no inner_measure is needed/used).

[bblodfon]

can change the wording to specifically mention the train/test splits of the inner resampling (I also mentionthat earlier in the doc), what do you think?

[be-marc]

you get the best subset and final model + score on that train set

It is the final model with the best subset and corresponding performance estimated on the inner resampling. There is no scoring on the outer training set but scoring on the inner resampling result. This is very similar to nested resampling. Maybe stick to the words used bellow figure 4.5

https://mlr3book.mlr-org.com/chapters/chapter4/hyperparameter_optimization.html#sec-nested-resampling

[bblodfon]

Yes sorry Marc, it's as you say, when I was writing the above comment, I meant outer resampling (what we call init_resampling) as the one that generates the train/test splits. And yes, pretty much we are doing nested CV, with outer resampling the N times holdout split. I will update the doc