Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check the data returned by JWST queries #366

Open
troyraen opened this issue Dec 18, 2024 · 1 comment
Open

Check the data returned by JWST queries #366

troyraen opened this issue Dec 18, 2024 · 1 comment
Labels
use case: spectroscopy Spectroscopy use case

Comments

@troyraen
Copy link
Contributor

troyraen commented Dec 18, 2024

Due to recent changes on the JWST and/or MAST end, there are two potential issues with the data being returned by our JWST_get_spec function, described below. More detail is in #360, including input from MAST folks. The result is that our notebook gets different JWST data than it used to for the same targets. A scientist should check whether the current results are appropriate and make adjustments if needed.

  1. Some of the files we were previously retrieving no longer exist in the archive due to JWST reprocessing ("which can cause some source extractions to no longer exist"). For now, we've relaxed our requirements to include calibration level 2 data (in addition to levels 3 and 4) in order to get results back for some of our targets.

  2. Some of the query results have a calibration level of "-1" (remove the calib_level filter to see this). The JWST doc Processing Levels and Product Stages says "-1 = Planned, but not yet executed". That makes me wonder if it's related to the reprocessing and indicates that those results are forthcoming. But, I don't know whether these "-1" values are new or whether it's an issue we even need to address.

Also, this function now takes about 14 min (on Fornax), while all the other archive calls in the notebook take about 3 min total. I don't know how long this was taking before the recent changes. If this is unexpectedly long, maybe one option is to change our target sample?

@troyraen troyraen added the use case: spectroscopy Spectroscopy use case label Dec 18, 2024
@jkrick
Copy link
Contributor

jkrick commented Dec 19, 2024

It sounds to me like you have done a good job with the tech review of this code, and maybe it is time to ask some of the JWST experts on the Fornax team for help with these questions as well as the timeout error in #339

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
use case: spectroscopy Spectroscopy use case
Projects
None yet
Development

No branches or pull requests

2 participants