FP8 inference performance question, possible W8A8 and 8-bit flash/sage attention acceleration using H100?

[TheTinyTeddy]

Hi,

Thank you for the amazing work!

I observed the throughput using bf16 is 34.2s/it, but the thought goes up to 34.4s/it when using fp8 that is slightly worse than bf16 DiT model. The GPU used is H100 which has FP8 Tensor Core compute capability. So I was wondering the possibility of W8A8 in linear layer and fp8 flash/sage attention acceleration for HunyuanVideo?

Many thanks