Pod predictor will need interpod affinity checks

[burmanm]

After updating the Kubernetes dependencies to 1.31, we can use the interpodaffinity plugin in the scheduler predictor also. This is required to verify that the pods can really be scheduled or not. At the moment, we could predict adding new pods succeeds while in reality some of the pods could not be scheduled as existing pods could already be on the nodes with anti-affinity rules.

Checking for resources is not always enough.

┆Issue is synchronized with this Jira Task by Unito
┆Issue Number: K8CL-5