Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Uncertainty type ignored in Cubeviz spectral extraction #2713

Open
pllim opened this issue Feb 19, 2024 · 1 comment
Open

BUG: Uncertainty type ignored in Cubeviz spectral extraction #2713

pllim opened this issue Feb 19, 2024 · 1 comment
Labels
bug Something isn't working cubeviz plugin Label for plugins common to multiple configurations

Comments

@pllim
Copy link
Contributor

pllim commented Feb 19, 2024

The current documentation (at the time of original posting this issue) at https://jdaviz.readthedocs.io/en/latest/cubeviz/plugins.html#spectral-extraction does not mention at all how uncertainty and mask arrays are used in the extraction process. The mask is yet another issue (#2714); here I am only focusing on uncertainty:

uncert_cube = self._app._jdaviz_helper._loaded_uncert_cube
# This plugin collapses over the *spatial axes* (optionally over a spatial subset,
# defaults to ``No Subset``). Since the Cubeviz parser puts the fluxes
# and uncertainties in different glue Data objects, we translate the spectral
# cube and its uncertainties into separate NDDataArrays, then combine them:
if self.aperture.selected != self.aperture.default_text:
nddata = spectral_cube.get_subset_object(
subset_id=self.aperture.selected, cls=NDDataArray
)
uncertainties = uncert_cube.get_subset_object(
subset_id=self.aperture.selected, cls=StdDevUncertainty
)
else:
nddata = spectral_cube.get_object(cls=NDDataArray)
uncertainties = uncert_cube.get_object(cls=StdDevUncertainty)

This was originally implemented by @bmorris3 via #2039 . In the code above, uncertainty cube is assumed to be always be in standard deviation.

However, Cubeviz parser actually allows at least 3 different kinds of uncertainty (inverse variance, variance, standard deviation, and who knows what else). The uncertainty type was only to detect whether a cube falls in that category, we do not have any extra processing to make sure it is actually always standard deviation.

EXT_TYPES = dict(flux=['flux', 'sci', 'data'],
uncert=['ivar', 'err', 'var', 'uncert'],
mask=['mask', 'dq', 'quality'])

As a result, if the uncertainty cube is not really standard deviation, it might be wrong to assume so, and that wrong assumption would propagate to it being handled improperly during extraction.

While, there appears to be some checks in https://github.com/glue-viz/glue-astronomy/blob/main/glue_astronomy/translators/nddata.py . I am not sure if those checks are actually used always due to the way we store uncertainty cube as a separate Data object, and not as data.uncertainty.

For instance, at one point, glue-astronomy translator looks for data.meta.get('uncertainty_type', 'std') but we never set that in Cubeviz parser. As a result, any of our code that goes through that logic would always return 'std' regardless.

The scope of this ticket is to confirm whether or not my concern of a bug is real. And if it is, what is necessary to fix. Thanks.

🐱

@pllim pllim added bug Something isn't working cubeviz plugin Label for plugins common to multiple configurations labels Feb 19, 2024
@bmorris3
Copy link
Contributor

The glue-astronomy NDData translator will represent any uncertainties in glue Data entry for the uncertainty as StdDevUncertainty by default (see NDData source here) when you ask for data.to_object. But note that the relevant translator from input->Data depends on what the input data are. I believe Cubeviz spectral extraction returns NDDataArrays, but the jwst products from MAST will be parsed via the glue-astro translator for Spectrum1D objects (in this block).

Both the NDData and Spectrum1D translators check for a metadata keyword called uncertainty_type, and uses the associated value to determine which type of uncertainties are given (source here). I believe I implemented this approach knowing that we could set an uncertainty_type key in meta in the parser, if we knew how to identify the uncertainty type correctly.

For your ticket, the next step is to check if (1) any JWST cubes have non-stddev uncertainties, (2) if there's a keyword that specifies the uncertainty type in the JWST spectral cubes used in Cubeviz, (3) setting data.meta['uncertainty_type'] if you find (2) and it's not stddev.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working cubeviz plugin Label for plugins common to multiple configurations
Projects
None yet
Development

No branches or pull requests

2 participants