Multi-fidelity tabular benchmark compatible additions

[Neeratyoy]

Main changes/updates (wrt ML benchmarks primarily):

• Editing abstract benchmark function signature to include option for choosing fidelity space type
  • Adds an extra parameter called fidelity_choice=None which shouldn't break existing code
  • Allows for future extension of the benchmark to a different fidelity (e.g.: dataset fraction) or even multi-multi-fidelity
• Adds a ML-benchmark specific template that can be inherited to easily add more parameter spaces by defining only their ConfigurationSpace and just the model definition
• The above point naturally implies that the data preprocessing and the model pipeline for the ML models are being fixed/standardized
  • More decisions need to be taken for the final design, but future decisions under this class design would mean any change to the pipeline affects all ML parameter spaces

TODO:

• Verify parameter spaces
• Hist Gradient Boosting not tested, only as placeholder

[KEggensperger]

Could you please rebase to development?

[PhMueller]

Hey Neeratyoy,

thanks for your work!
Before changing the Interface, maybe a quick question:

Couldn't we also solve it via inheritance?

I think about having a BaseMLBenchmark() and create new Classes which overwrite the corresponding fidelity space.
Is this something you could use?
Something like that?

class BaseMLBenchmark()
    def __init__():
        do_stuff()
        
   def objective_function():
       do_compute()
       
   def get_fidelity_space(seed=None):
       raise NotImplementedError()
       
class SmallFidelityBenchmark(BaseMLBenchmark):
    def get_fidelity_space(seed=None):
        return SmallFidelitySpace()
        
class LargeFidelityBenchmark(BaseMLBenchmark):
    def get_fidelity_space(seed=None):
        return LargeFidelitySpace()

Since the other benchmarks don't use the fidelity_choice, it would be cool to talk a little bit about it.

[Neeratyoy]

it would be cool to talk a little bit about it.

Sure, I think we should.

I went with that design since I believe there are checks for the abstract class function signature checks. And to not affect containerization potentially, I thought of changing the abstract class definition where I presumed a None will keep things safe.
I did try the inheritance (which I am already doing) but went back to this design for some reason I can't recall.

As for the multiple class definitions for different fidelity spaces, sure, that is an alternate solution. However, I just felt that it makes the code more verbose. Also might make things more brittle for code using these classes. Hence, I went with the parameterization approach.

In any case, these are probably design choices more than functionality so we just need to come to an agreement wrt the scope of the package. Happy to discuss!

[Neeratyoy]

@PhMueller shall we close this?

[Neeratyoy]

Deprecated.
Subsumed in #121.