fix: grid hp samping ignored empty nests #9966
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for determined-ui canceled.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #9966 +/- ##
=======================================
Coverage 54.50% 54.50%
=======================================
Files 1255 1255
Lines 156733 156736 +3
Branches 3600 3600
=======================================
+ Hits 85420 85423 +3
Misses 71180 71180
Partials 133 133
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Previously, when a hyperparameter contained an emtpy "nested"
hyperparameter:
hyperparameters:
empty: {}
It was lost during grid sampling, and was never passed to the trial.
This is because the nested hyperparameter isn't considered its own axis
for the grid search, and is only implicitly populated by being on the
path of some inner hyperparameter axis. However, when there is no inner
hyperparameter axis, we need to detect that and emit a one-point axis
that preserves the empty nested hyperparameter.
Another possible solution would be to convert the empty nested
hyperparameter as a const hyperparameter with an empty map for a value
when unmarshaling the hyperparameters yaml object. I think no user code
would be affected by which solution is used, since user code will see
the same hyperparameter sample either way.
rb-determined-ai
force-pushed
the
rb/fix-grid
branch
from
6797175 to
1b99512
Compare
maxrussell
approved these changes
Sep 23, 2024
| @@ -74,6 +74,20 @@ func TestHyperparameterGridMethod(t *testing.T) { | |||
| len(getGridAxes([]string{"x"}, expconf.Hyperparameter{RawConstHyperparameter: &constParam})[0]), | |||
| 1, | |||
| ) | |||
| // Regression test: make sure empty nested hyperparameters don't disappear during sampling. | |||
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously, when a hyperparameter contained an emtpy "nested" hyperparameter:
It was lost during grid sampling, and was never passed to the trial. This is because the nested hyperparameter isn't considered its own axis for the grid search, and is only implicitly populated by being on the path of some inner hyperparameter axis. However, when there is no inner hyperparameter axis, we need to detect that and emit a one-point axis that preserves the empty nested hyperparameter.
Another possible solution would be to convert the empty nested hyperparameter as a const hyperparameter with an empty map for a value when unmarshaling the hyperparameters yaml object. I think no user code would be affected by which solution is used, since user code will see the same hyperparameter sample either way.
Test Plan