fix: port k8s perf fix

[maxrussell]

Description

Cherry picked the k8s performance fix into

Test Plan

Manually test k8s flows in release party

Checklist

• Changes have been manually QA'd
• New features have been approved by the corresponding PM
• User-facing API changes have the "User-facing API Change" label
• Release notes have been added as a separate file under docs/release-notes/
  See Release Note for details.
• Licenses have been included for new code which was copied and/or modified from any external code

[netlify]

✅ Deploy Preview for determined-ui canceled.

Name	Link
🔨 Latest commit	e76d11e
🔍 Latest deploy log	https://app.netlify.com/sites/determined-ui/deploys/673d0b1f5fc72e00089307fc

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 57.84%. Comparing base (258455d) to head (e76d11e).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10213      +/-   ##
==========================================
+ Coverage   53.88%   57.84%   +3.96%     
==========================================
  Files        1257      752     -505     
  Lines      156507   103521   -52986     
  Branches     3653     3653              
==========================================
- Hits        84335    59885   -24450     
+ Misses      72039    43503   -28536     
  Partials      133      133              

Flag	Coverage Δ
backend	43.80% <ø> (-2.27%)	⬇️
harness	67.37% <ø> (ø)
web	54.75% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 505 files with indirect coverage changes

---- 🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests
• JS Bundle Analysis - Avoid shipping oversized bundles

[stoksc]

lgtm

[maxrussell]

This was tested on earlier versions and seems to be only a problem in the automated test code. Merging so we can manually test.