Measure's check_prerequisites is ignored when calling $score() on a ResampleResult

[sebffischer]

Maybe a misunderstanding from my side, but shouldn't I get a warning in the code below, as the regr.debug learner does not have the "selected_features" property?

library(mlr3)

rr = resample(tsk("mtcars"), lrn("regr.debug"), rsmp("holdout"), store_models = TRUE)
#> INFO  [17:13:24.525] [mlr3] Applying learner 'regr.debug' on task 'mtcars' (iter 1/1)
measure = msr("selected_features")
# is default anyway
measure$check_prerequisites = "warn"

rr$score(measure)
#>    task_id learner_id resampling_id iteration selected_features
#>     <char>     <char>        <char>     <int>             <num>
#> 1:  mtcars regr.debug       holdout         1                NA
#> Hidden columns: task, learner, resampling, prediction

^{Created on 2024-02-01 with reprex v2.0.2}

[be-marc]

This is also the case when running measure$score().

learner = lrn("regr.debug")
learner$train(tsk("mtcars"))
pred = learner$predict(tsk("mtcars"))
measure = msr("selected_features")
measure$score(pred, task = tsk("mtcars"), learner = learner)

[be-marc]

Looks like we don't the check the learner properties "importance", "selected_features", "oob_error" at all. We just return NA. I would vote for adding the warning directly to Mesaure$.score() instead of adding requires_selected_features and requires_oob_error.

[be-marc]

Except if there are multiple measures that need oob_error or the selected_features.