How to get closer accuracy with FP16 in demo/Diffusion 8-bits PTQ?

[TheBge12138]

Hello, notice that tensorrt 9.3 had updated the 8-bit quantization to accelerate diffusion models, it's excellent !
But when running the demo, i can't get the same result with fp16, I have modified the get_pipeline in model.py to my own model, and used over 1000 prompts for calibration, there is a big gap under some prmopts, and some prmopts are close but still not completely similar.

my sdxl model pipeline has 30steps, I compared the cosine similarity of the unet output tensor. In 60 prompt tests, compared with fp16, the similarity fluctuated between 89% and 97%.

in https://developer.nvidia.com/blog/tensorrt-accelerates-stable-diffusion-nearly-2x-faster-with-8-bit-post-training-quantization/
I saw you can get images that are nearly identical to original FP16 precision.
Is there anything I can do to improve the accuracy of my in8 model？

Thanks!

Tasks

Beta Give feedback

No tasks being tracked yet.

[zerollzeng]

@ttyio ^ ^

[TheBge12138]

@jingyu-ml
thanks for your reply! i saw it on email
Can I ask when release/10.0 will be released?

[jingyu-ml]

@jingyu-ml thanks for your reply! i saw it on email Can I ask when release/10.0 will be released?

I don't have the release date, but I would say its in the near future and the team is working on it.

[zerollzeng]

TRT 10 EA has been released.

[TheBge12138]

TRT 10 EA has been released.

Excellent, thanks for your work!

[jingyu-ml]

@TheBge12138 please check the GA release instead of the EA release
d3107c8

[TheBge12138]

@TheBge12138 please check the GA release instead of the EA release d3107c8

thanks!