Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve hyperparameter logging #153

Merged
merged 1 commit into from
Jan 18, 2024
Merged

Conversation

ArneBinder
Copy link
Owner

@ArneBinder ArneBinder commented Jan 18, 2024

This PR improves the hyperparameter logging when executing train.py or evaluate.py in the following ways, i.e. log_hyperparameters():

  • the hydra-run config gets resolved before logging it, that means ConfigDict entries are correctly un-nested (compare the dataset entry in the W&B run config view, for instance)
  • remove duplicated entries by not logging model._config() and taskmodule._config() at all, because the model parameters are automatically logged by calling self.save_hyperparameters() in the model and in the future each model should receive taskmodule_config as parameter, so that will get logged as well (taskmodule._config() is currently still logged in some cases for backwards compatibility)
  • the method allows for arbitrary keyword arguments that will get logged
  • the logging keys for the config, number of model parameter, and all other parameters to log are prefixed with key_prefix, which is "_" per default, to distinguish them from the model parameters.

In addition, with this PR two more parameters are logged:

  • _best_checkpoint: os.path.basename(best_ckpt_path) taken from the checkpoint callback
  • _checkpoint_dir: dirpath taken from the checkpoint callback

Note: this requires pie-modules>=0.9.0

@ArneBinder ArneBinder added the enhancement New feature or request label Jan 18, 2024
@ArneBinder ArneBinder merged commit 2130659 into main Jan 18, 2024
2 checks passed
@ArneBinder ArneBinder deleted the improve_hyperparameter_logging branch January 18, 2024 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant