Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cubeviz parser to load all sorts of s3d cubes #911

Closed
wants to merge 2 commits into from

Conversation

pllim
Copy link
Contributor

@pllim pllim commented Sep 27, 2021

See javerbukh#9

Description

This pull request is to address s3d load failures reported by @havok2063 .

This builds on what work @javerbukh started in #547 .

Good news: I got all the files to load.
Bad news: This is very hacky and broke two plugins.

🐱 🐱

Checklist for package maintainer(s)

This checklist is meant to remind the package maintainer(s) who will review this pull request of some common things to look for. This list is not exhaustive.

  • Are two approvals required? Branch protection rule does not check for the second approval. If a second approval is not necessary, please apply the trivial label.
  • Do the proposed changes actually accomplish desired goals? Also manually run the affected example notebooks, if necessary.
  • Do the proposed changes follow the STScI Style Guides?
  • Are tests added/updated as required? If so, do they follow the STScI Style Guides?
  • Are docs added/updated as required? If so, do they follow the STScI Style Guides?
  • Did the CI pass? If not, are the failures related?
  • Is a change log needed? If yes, is it added to CHANGES.rst?
  • Is a milestone set? Milestone is only currently required for PRs related to Imviz MVP.
  • After merge, any internal documentations need updating (e.g., JIRA, Innerspace)?

Concept of loading spectrum1d as cube in CubevizExample

Get slider working correctly

SpectralCube to specutils transition in cubeviz

Manga cube loaded using Spectrum1D, still some things to iron out

Add support for loading Spectrum1D cubes

Fix codestyle checks

Use correct axes without np swapaxes

Set correct RA and Dec to x y axes, respectively

Remove excess commits and get slider working
unit = u.Unit(hdulist[1].header.get('BUNIT', 'count'))
flux = hdulist[1].data << unit
wcs = WCS(hdulist[1].header, hdulist)
data = Data(flux=flux, coords=wcs) # Spectrum1D too slow
Copy link
Contributor Author

@pllim pllim Sep 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided to bypass Spectrum1D parsing because of two problems:

  1. Spec1D read is slow compared to astropy.io fits.open astropy/specutils#688 (this will be fixed once Speed up jwst s3d loader astropy/specutils#874 is merged)
  2. Even if I wait patiently for it to load, Glue tries to swap something in the data translation phase (Spectrum1D -> Data) and fails.

This is what I tried (something like that):

sp = Spectrum1D.read(file_obj, format='JWST s3d') 
app.add_data(sp, data_label)

Because I did not pass in Spectrum1D, but rather native Glue object, line list and unit conversion plugins break because they make certain assumption of the data types.

@pllim pllim added this to the 2.1 milestone Sep 27, 2021
@pllim
Copy link
Contributor Author

pllim commented Sep 27, 2021

p.s. Ginga can load the cubes fine and without lag. It does not use specutils, so it is able to take advantage of FITS memory mapping.

@codecov
Copy link

codecov bot commented Sep 27, 2021

Codecov Report

Merging #911 (8f33857) into main (86fce7b) will decrease coverage by 0.46%.
The diff coverage is 36.06%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #911      +/-   ##
==========================================
- Coverage   68.38%   67.92%   -0.47%     
==========================================
  Files          66       66              
  Lines        4799     4823      +24     
==========================================
- Hits         3282     3276       -6     
- Misses       1517     1547      +30     
Impacted Files Coverage Δ
...z/configs/default/plugins/line_lists/line_lists.py 39.90% <0.00%> (ø)
jdaviz/configs/cubeviz/plugins/parsers.py 57.57% <35.71%> (-24.48%) ⬇️
...specviz/plugins/unit_conversion/unit_conversion.py 72.84% <50.00%> (-0.81%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 86fce7b...8f33857. Read the comment docs.

@pllim
Copy link
Contributor Author

pllim commented Sep 28, 2021

I think it is clearer if we look at javerbukh#9

@pllim pllim closed this Sep 28, 2021
@pllim pllim deleted the pr547-vs-ice-cube branch September 28, 2021 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants