Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue/228/calpreddistr #291

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open

Issue/228/calpreddistr #291

wants to merge 31 commits into from

Conversation

torluca
Copy link
Collaborator

@torluca torluca commented Nov 28, 2022

This is a pull request to merge the implementation of the calibrated predictive distribution into the main branch.
The file that has been added is in the src/rail/evaluation folder.
I added a demo notebook on how to run DSPS (/examples/creation/calibrated_predictive_distributions_demo.ipynb).
Unit tests are still missing and taking now most of the time.
There were suggestions of modifying the code to have an estimation and an evaluation part.

… utils files from Bitrateep in condition_pit_utils
…8/calpreddistr

merging main branch into issue/228/calpreddistr branch as of 20220819
…putation of Bitrateep instead of that from class UnconditionPIT, addition of utils.py file from Bitrateep work
merge with main changes as of 2022.11.07
…ated predictive distribution method, test file to run the latter added
@torluca torluca added documentation Improvements or additions to documentation evaluation metrics of photo-z posteriors refactor Moving code around to make the world a better place labels Nov 28, 2022
@torluca torluca self-assigned this Nov 28, 2022
@codecov
Copy link

codecov bot commented Dec 5, 2022

Codecov Report

Patch coverage: 23.07% and project coverage change: -1.57 ⚠️

Comparison is base (659c4e8) 100.00% compared to head (9df8ca3) 98.43%.

❗ Current head 9df8ca3 differs from pull request most recent head 437794f. Consider uploading reports for the commit 437794f to get more accurate results

Additional details and impacted files
@@             Coverage Diff             @@
##              main     #291      +/-   ##
===========================================
- Coverage   100.00%   98.43%   -1.57%     
===========================================
  Files           77       38      -39     
  Lines         5548     2559    -2989     
===========================================
- Hits          5548     2519    -3029     
- Misses           0       40      +40     
Flag Coverage Δ
unittests 98.43% <23.07%> (-1.57%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/rail/evaluation/metrics/pit.py 68.00% <23.07%> (ø)

... and 55 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@torluca
Copy link
Collaborator Author

torluca commented Jan 30, 2023

@sschmidt23 Biprateep is wrapping up his code into a package. This way we can get rid of hard coded stuff in RAIL and just import his external package. I will keep you updated on when I'm implementing these new changes. For the moment, I would pause the merging with the main branch.

@sschmidt23
Copy link
Collaborator

Ok, just message me again when Biprateep's code is in place and you think the code is ready for review.

@biprateep
Copy link

Hi the package is more or less ready and available here: https://github.com/lee-group-cmu/Cal-PIT , will update you in a week or so when the final version of the package is public and pip installable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation evaluation metrics of photo-z posteriors refactor Moving code around to make the world a better place
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants