ojph_expand error/warning behaviour with quality layer placeholder passes

[michaeldsmith]

While I'm not suggesting that using HT with quality layers is a great idea for mainstream use, it could be important in some special use cases and also when transcoding to/from J2K Part-1 codestreams which do commonly use quality layers. I think the HTJ2K standard allows the representation of quality layers using the placeholder passes feature.

Ideally, OpenJPH would be able to decode these files and if not, have a consistent response to files that are valid HTJ2K files yet not yet decodeable by OpenJPH.

ojph_expand compiled from current master branch has a different response to the attached input files with placeholder passes, depending on the input file

different errors and warnings are displayed
A malformed codeblock that has more than one coding pass, but zero length for 2nd and potential 3rd pass.
Error decoding a codeblock
We do not support more than 3 coding passes; This codeblocks has 5 passes.
error in parsing a tile header; missing msbs are larger or equal to Kmax. The most likely cause is a corruption in the bitstream.
Segmentation Fault

In the case of the pathological test with 65335 quality layers ht_216p_65535_layers.j2c, a gray image is decoded without any error or warning output.

ojph_expand -i ht_216p_01_layers.j2c -o ht_216p_01_layers.j2c.tif
Elapsed time = 0.002735

ojph_expand -i ht_216p_02_layers.j2c -o ht_216p_02_layers.j2c.tif
ojph warning 0x00010001 at ojph_block_decoder_ssse3.cpp:1034: A malformed codeblock that has more than one coding pass, but zero length for 2nd and potential 3rd pass.
ojph error 0x000300A1 at ojph_codestream.cpp:4067: Error decoding a codeblock

ojph_expand -i ht_216p_07_layers.j2c -o ht_216p_07_layers.j2c.tif
ojph error 0x000300A1 at ojph_codestream.cpp:4067: Error decoding a codeblock

ojph_expand -i ht_216p_19_layers.j2c -o ht_216p_19_layers.j2c.tif
ojph warning 0x00010002 at ojph_block_decoder_ssse3.cpp:1042: We do not support more than 3 coding passes; This codeblocks has 5 passes.
ojph error 0x000300A1 at ojph_codestream.cpp:4067: Error decoding a codeblock

ojph_expand -i ht_216p_65535_layers.j2c -o ht_216p_65535_layers.j2c.tif
Elapsed time = 0.002206

ojph_expand -i ht_216p_01_layers.j2c -o ht_216p_01_layers.j2c.tif
Elapsed time = 0.002665

ojph_expand -i ht_2160p_01_layers.j2c -o ht_2160p_01_layers.j2c.tif
Elapsed time = 0.112171

ojph_expand -i ht_2160p_02_layers.j2c -o ht_2160p_02_layers.j2c.tif
ojph error 0x00030092 at ojph_codestream.cpp:1898: error in parsing a tile header; missing msbs are larger or equal to Kmax. The most likely cause is a corruption in the bitstream.

ojph_expand -i ht_2160p_10_layers.j2c -o ht_2160p_10_layers.j2c.tif
Segmentation fault

The input files were encoded with Kakadu v.8.2.1 using the following commands:

kdu_compress -i meridian_216p.tif -o ht_216p_01_layers.j2c Cmodes=HT -rate 1 Clayers=1 -no_info
kdu_compress -i meridian_216p.tif -o ht_216p_02_layers.j2c Cmodes=HT -rate 1 Clayers=2 -no_info
kdu_compress -i meridian_216p.tif -o ht_216p_07_layers.j2c Cmodes=HT -rate 1 Clayers=7 -no_info
kdu_compress -i meridian_216p.tif -o ht_216p_19_layers.j2c Cmodes=HT -rate 1 Clayers=19 -no_info
kdu_compress -i meridian_216p.tif -o ht_216p_65535_layers.j2c Cmodes=HT Clayers=65535 -no_info

kdu_compress -i 3840x2160_10bit_444_BT709_SDR.tif -o ht_2160p_01_layers.j2c Cmodes=HT -rate 1 Clayers=1 -no_info
kdu_compress -i 3840x2160_10bit_444_BT709_SDR.tif -o ht_2160p_02_layers.j2c Cmodes=HT -rate 1 Clayers=2 -no_info
kdu_compress -i 3840x2160_10bit_444_BT709_SDR.tif -o ht_2160p_10_layers.j2c Cmodes=HT -rate 1 Clayers=10 -no_info

The HJT2K J2C files generated by these commands are contained in the attached zip files:

ht_2160p_01_layers.zip
ht_2160p_02_layers.zip
ht_2160p_10_layers.zip
ht_216p_01_layers.zip
ht_216p_02_layers.zip
ht_216p_07_layers.zip
ht_216p_19_layers.zip
ht_216p_65535_layers.zip

[aous72]

Hi Mike,

Thank you for the detailed report.

Currently, the precinct parser can only parse simple one-layer HT codestreams; there are no specific paths that detect other cases, let alone handle them. That's why the message can be different depending on where the code hits a problem; the code interprets the packet header incorrectly, and then try to decode with this wrong interpretation.

The code needs to interpret the data correctly before we can have consistent messaging.

If I understand things correctly, there are at least two cases that needs to be handled: multi-ht sets, placeholders passes.

I think the next step should be passing the conformance test -- passing these should address most of the concerns in this issue.

Kind regards,
Aous.