Unblock CI failures from scikit-learn 1.4.0, pandas 2.2.0 #4264

Job	Run time
Check for upstream trigger phrase	5s
Test importing with bare requirements	12m 2s
Build & Test (ubuntu-latest, python: 3.9, distributed: false)	8m 24s
Build & Test (ubuntu-latest, python: 3.10, distributed: false)	8m 40s
Build & Test (ubuntu-latest, python: 3.11, distributed: false)	8m 39s
Build & Test (ubuntu-latest, python: 3.12, distributed: false)	8m 59s
Build & Test (windows-latest, python: 3.9, distributed: false)	16m 22s
Build & Test (windows-latest, python: 3.10, distributed: false)	16m 36s
Build & Test (windows-latest, python: 3.11, distributed: false)	16m 47s
Build & Test (windows-latest, python: 3.12, distributed: false)	17m 59s
Build & Test (macos-latest, python: 3.9, distributed: false)	14m 43s
Build & Test (macos-latest, python: 3.10, distributed: false)	12m 34s
Build & Test (macos-latest, python: 3.11, distributed: false)	14m 4s
Build & Test (macos-latest, python: 3.12, distributed: false)	18m 56s
Build & Test (ubuntu-latest, python: 3.9, distributed: true)	8m 41s
Build & Test (ubuntu-latest, python: 3.11, distributed: true)	9m 34s
	3h 13m 5s

Provide feedback