fix: run checkpoint GC more aggressively to ensure tensorboards are GC'd

[stoksc]

Ticket

CM-540

Description

Unfortunately we can't be clever about skipping checkpoint GC since there still may be tensorboard files.

Test Plan

Tested manually and left a comment so this change isn't undone.

[netlify]

✅ Deploy Preview for determined-ui ready!

Name	Link
🔨 Latest commit	0088cfd
🔍 Latest deploy log	https://app.netlify.com/sites/determined-ui/deploys/66fee6f0cee7220008a26528
😎 Deploy Preview	https://deploy-preview-10017--determined-ui.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[codecov]

Codecov Report

Attention: Patch coverage is 71.42857% with 2 lines in your changes missing coverage. Please review.

Project coverage is 54.59%. Comparing base (a0cc818) to head (0088cfd).
Report is 37 commits behind head on main.

Files with missing lines	Patch %	Lines
master/internal/api_experiment.go	71.42%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #10017   +/-   ##
=======================================
  Coverage   54.59%   54.59%           
=======================================
  Files        1259     1259           
  Lines      157245   157244    -1     
  Branches     3620     3618    -2     
=======================================
+ Hits        85843    85846    +3     
+ Misses      71269    71265    -4     
  Partials      133      133           

Flag	Coverage Δ
backend	45.35% <71.42%> (+<0.01%)	⬆️
harness	72.74% <ø> (ø)
web	54.34% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
master/internal/experiment.go	30.12% <ø> (ø)
master/internal/api_experiment.go	57.06% <71.42%> (+0.23%)	⬆️

... and 4 files with indirect coverage changes

[jesse-amano-hpe]

Ugh, sorry I didn't notice this PR in my inbox. Looks great -- I should've approved two weeks ago.