Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for 16 and 32 bit input #41

Open
wfarah opened this issue Dec 7, 2021 · 6 comments
Open

Support for 16 and 32 bit input #41

wfarah opened this issue Dec 7, 2021 · 6 comments
Assignees
Labels
enhancement New feature or request

Comments

@wfarah
Copy link

wfarah commented Dec 7, 2021

rawspec currently supports an 8bit and more recently a 4bit input.
The ATA beamformer will output samples at 16 and 32 bit (half and single precision float).
The beamformer output will be saved in a guppi format, which I currently assume can support this bit rate. It would be a great addition to rawspec to make it handle the above input bit size.
It's also common in astronomy to use 16 and 32 bit integers, so maybe this is worth a discussion as to how this could be treated within the code. Maybe an additional guppi header parameter

@wfarah wfarah added the enhancement New feature or request label Dec 7, 2021
@texadactyl
Copy link
Contributor

All of my GBT testing has been with 32-bit floats. So, this works. I do not have access to a .raw file with 16-bit floats.
@david-macmahon : tried 16-bit floats yet?

Integer element formats? That's another question for @david-macmahon .

The FBH5 output code doesn't care what its writing.

@wfarah
Copy link
Author

wfarah commented Dec 9, 2021

@texadactyl I can produce a half-precision file, but the values there will kinda be non-sense.
At least until I can calibrate the beamformer properly

@david-macmahon
Copy link
Contributor

rawspec currently supports 8+8 bit complex integer data and 16+16 bit integer data. The GUPPI RAW format has historically been limited to integer data. There is no "datatype" header field defined to indicate what type of values are in the data blocks other than NBITS which specifies how many bits each real/imaginary component has. We could invent a new header field to indicate integer or float, but other GUPPI RAW readers (e.g. dspsr) would ignore it until they get modified to accommodate it. I'm not sure it's really worth all the effort though. Why not just re-quantize to 8 or 16 bit integers before writing out the coherent beams? The data are coming in as 8 or 16 bit values anyway, would the dynamic range would increase so much that we require floating point output?

@texadactyl
Copy link
Contributor

@wfarah
Have you reached consensus with @david-macmahon ?
Does this issue need a redefinition or ?

@siemion
Copy link
Member

siemion commented Mar 11, 2022

nbits=32 is widely recognized implicitly in pulsar land as meaning floating point. I think NBITS = 32 is a reasonable hack at this point

@radonnachie
Copy link
Contributor

radonnachie commented Mar 11, 2022

When it comes to how to trigger the integer_data flag, I believe we need primarily to decide how we want other programs to react to floating point data: do we want them to 🎉 break/error out/scream, or do we want them to 🚀 merely continue ignorantly.
Negative NBITS values as a flag should fall into intention 🎉 .
A new FITS key-value such as DATATYPE!='INTEGER' should fall into intention 🚀 .

Could we tally votes with reactions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants