DngConverter not writing black level correctly

[olivertai]

Hi, I'm running your latest version under ubuntu with google hdr+ official burst images, I tried to get raw align and merge result by running binary "stack_frames", the bursts of inputs have black level at 64 and white level at 1023, then I used an image previewing software to compare my output with the output of google group (offered together with the bursts), I found out that my output was visually more "purple" which I thought due to wrong black level in the .dng file. I checked the black_level array before TIFFSetField(tiff, TIFFTAG_BLACKLEVEL, 4, &black_level); but they were all set to 64 already. Then I set the output .dng as input and got, bl at 0 and wl at 1023 for my output, and bl at 1024 and wl at 16368 for google's output. Could you help me with that? Thank you.

[brotherofken]

Hi!
Thank you for the bug report. I can take a look and it would be really helpful if you share this burst somehow. I have HDR+ data, so could you write the name of a burst?

Black level equals to 1024 in the output of google's group since they increase the number of gray levels, which is not implemented here. But it's trivially to do and maybe I'll implement it shortly.

[olivertai]

Hi, thank you for quick response!
I’m currently running tests on the whole subset(153 bursts) and it seems to me that the problem appears not only on a single or several bursts but on all of them.
What I do now is appending the black_white_level step in stack_frames then the bl stores in .dng file should be 0, so I get visually more similar output to google’s, which however I don’t consider is a proper method.
From here, I encountered two additional problems. Firstly, I multiplied the images with 16, as well as the bl and wl, to simulate google’s output, I got really close result but still a little lack of saturation(mine is less saturated), is this something to do with the color matrix? Because I don’t think align and merge step make such effort on the result, maybe it’s just some other bad-saved metadata causing preview problem.
Secondly, the black_level in function contrast confused me since black level adjustment has been done at the beginning of post-processing, can I get to know the reason and what value of bl should be set here? Thank you.

Btw, I’m using libtiff of version 4.0.6 if this is an useful information for you.

[brotherofken]

I’m currently running tests on the whole subset(153 bursts) and it seems to me that the problem appears not only on a single or several bursts but on all of them.

I've checked stack_frames on the following samples from the subset and cannot see purplish color in the result or wrong black level in the results. Look for merged.dng in subdirectories. Download them first, don't trust the preview. However...

I got really close result but still a little lack of saturation(mine is less saturated), is this something to do with the color matrix?

I found, that DngConverter doesn't write proper color matrices to merged DNG. :-( Seems, that it's broken now and requires fixing. There are lack of full set of tags now: Color Matrix X, Camera Calibration X and Calibration Illuminant X are missing and previews on cloud storage broken for that reason.

Secondly, the black_level in function contrast confused me since black level adjustment has been done at the beginning of post-processing, can I get to know the reason and what value of bl should be set here?

I looked at the code and if I've got it right black level in contrast means that all values below that level would be clipped and this level becomes new 0.

[brotherofken]

Right now I am very limited in time, but maybe you will be able to fix this issue by yourself?
I think that here we need to copy appropriate DNG tags from libraw.
Maybe I'll have a few hours to look at this on the weekend, but can't say for sure.

[olivertai]

I've checked stack_frames on the following samples from the subset and cannot see purplish color in the result or wrong black level in the results. Look for merged.dng in subdirectories. Download them first, don't trust the preview. However...

Thank you for the response. I've downloaded the results and decoded with libraw, eventually I still found black level at 0, that's really weired. But it's not a big deal for now since I do black level substraction before saving.

I found, that DngConverter doesn't write proper color matrices to merged DNG. :-( Seems, that it's broken now and requires fixing. There are lack of full set of tags now: Color Matrix X, Camera Calibration X and Calibration Illuminant X are missing and previews on cloud storage broken for that reason.

I set CameraCalibration to identity matrix, Color Matrix to the matrix stored in the data folder(rgb2rgb.txt) and Calibration Illuminant was already set as far as I know (D65). I think that's enough for a color correction process. Is that right?
I still have some difficulties understanding the implementation when digging into merge process. But I'll start another issue since it's not relevant to the current subject. Thank you.