Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metric from pathfinder to pre-heat NUTS #238

Open
SamuelBrand1 opened this issue May 29, 2024 · 4 comments
Open

Add metric from pathfinder to pre-heat NUTS #238

SamuelBrand1 opened this issue May 29, 2024 · 4 comments
Labels
enhancement New feature or request EpiAware

Comments

@SamuelBrand1
Copy link
Collaborator

We are by default using pathfinder.jl to pre-heat our NUTS sampling by providing an (approximate) sample from the posterior distribution as the initial point for NUTS warm-up, as per here.

The reason we didn't also include an estimate of the underlying kinetic energy matrix/inverse curvature metric was because of the API to Turing not being straightforward.

As of https://turinglang.org/docs/tutorials/docs-16-using-turing-external-samplers/ this appears to be fixed/made easy to do, and so we should have metric pre-heating.

@SamuelBrand1
Copy link
Collaborator Author

Unfortunately, AdvancedHMC.NUTS doesn't have an adtype option because its designed to interface directly with a LogDensityProblemAD.ADgradient object... which means that we can't direct port across.

I've flagged this with Turing TuringLang/Turing.jl#2248

@SamuelBrand1
Copy link
Collaborator Author

The direction of travel here:

TuringLang/Turing.jl#2248 (comment)

@SamuelBrand1
Copy link
Collaborator Author

SamuelBrand1 commented Jun 6, 2024

This can go forwards after #262 because the latest version of Turing has a kwarg to externalsampler to pass the adtype choice.

@SamuelBrand1
Copy link
Collaborator Author

Update: f2f chat with @dylanhmorris and @athowes (don't have his git handle) as part of STF meeting included discussion around when sometimes Pathfinder doesn't improve NUTS pre-heating.

Some experimentation on the associated branch to this issue is having mixed results with including the metric in pre-heat; I'm de-prioritizing investigating this for now but would like to keep this issue open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request EpiAware
Projects
None yet
Development

No branches or pull requests

1 participant