Inneficient and not-standarized way to upload buffers to OpenGL in WATERMeDIA

[SrRapero720]

LETS TALK

133a000 introduced by @ZenoArrows changes the target format and buffer type to BGRA
PR #65 introduced by @zFERDQFREZrzfq changes the impl and also applies BGRA uploading a ARGB buffer

CONCERN

Zeno args BGRA is the most efficient format for the GPU and the drivers while FEDO args is better just pass the ARGB format natively.
Right now glTeexImage2D burns on profilers, specially on videos.

WHAT DOCS SAYS

After some research, i find this:
https://www.khronos.org/opengl/wiki/Common_Mistakes#Texture_upload_and_pixel_reads
complemented with this: https://www.khronos.org/opengl/wiki/Pixel_Transfer#Pixel_format

TL;DR: BGRA is preferred on windows? nope

Aparently (assuming) that was a very old information involving windows phone and many other rusty techs. but all concludes ARGB and BGRA performs the same. So this information is inconcluse

Also it mentions a little bit about image convertion but nothing than states about performance (only functionality)

REASON OF THE DISCUSSION

Format, Buffers and many other rendering stuff is not well done

This need to place a spec on the table to workout using default and interfaces to manage properties about a picture/video, its format and pixel type, and start preparing the API rewrite of the RenderAPI, PlayerAPI and ImageAPI making them work with almost the same spec

TOPIC OF DISCUSSION

• What is the better format for GPUs? Is divided by GPU or by OS?
• Is better delegate to GPU the format change? or keep using the off-thread impl
• anything state above is correct? what says gpu vendors about?

[zFERDQFREZrzfq]

My PR sends BGRA with the fromat set to GL_UNSIGNED_INT_8_8_8_8_REV, telling OpenGL that the buffer is reversed.
According to things I read, this let the GPU do the byteswap in hardware if needed, which is indeed much fasted than anything we could do on CPU, especially in Java :)

The issue we have here is that we need to handle different images format, so everything we can delegate to hardware should be faster than doing on CPU.

The reason why I introduced is not a matter of ARGB vs BGRA, I did this because Java internally uses BGRA, and OpenGL can handle it natively, so we saves all the byteswapping by delegating it to the GPU's dedicated hardware.

Even if it would happens that BGRA would not be the best for GPU transfer, I don't see how it would be faster to do the byteswap in software rather than on dedicated hardware, or could it be that OpenGL do it is software before sending it GPU? O_O

Also : beware of profiler % notation, because it shows % relative to the calling method, so because we have removed all the CPU byteswapping, only the OpenGL is left. In that case it may be better to compare absolute timings in ms? Or did I miss something?

It looks like this does not applies to Spark, as it shows % relative to the thread.

I did a quick test with spark while playing a youtube video on a WaterFrame and the OpenGL itself part looks fine so far?

[zFERDQFREZrzfq]

- What is the better format for GPUs? Is divided by GPU or by OS?
- Is better delegate to GPU the format change? or keep using the off-thread impl

I'm not an expert in graphics, but I digged a bit and found this so far :

According to https://registry.khronos.org/OpenGL-Refpages/gl4/html/glTexImage2D.xhtml, the importants parameters on glTexImage2D are the following :

• format (tells OpenGL the format (channels) of our source buffer)
• internalformat (sets the format (channels) we want for the texture on the GPU)
• type (tells OpenGL how the data is encoded in our source buffer)

According to https://www.khronos.org/opengl/wiki/Pixel_Transfer#Format_conversion, the best would be to match the OpenGL parameters to the source buffer we have as much as we can, so that OpenGL will handle the conversion work, possibly in hardware (so MUCH faster).
We should avoid any format conversion or byteswapping on CPU as long as there is a matching OpenGL support.

For exemple, if we have a buffer made of type BufferedImage.TYPE_INT_ARGB.
TYPE_INT_ARGB in Java side means 4 channels of 8 bits in ordered ARGB, so the OpenGL parameters should be :

• internalformat : GL_RGBA8 (4 channels of 8 bits, the order is not relevant here)
• format : GL_BGRA (4 channels ordered BGRA because OpenGL don't have ARGB)
• type : UINT_8_8_8_8_REV (4 channels 8 bits packed as a a 32 bits UINT, reversed so our source ARGB is converted to BGRA)
  So we tell OpenGL to use an RGBA texture and we feed it with an ARGB buffer, and tells OpenGL that the order in the buffer is reversed, so that it will handle the byteswapping for us.

Another exemple, if we had an image with a BufferedImage.TYPE_3BYTE_BGR (BMP for exemple), we should have the following parameters for glTexImage2D :

• internalformat : GL_RGB8 (3 channels of 8 bits, the order is not relevant here)
• format : GL_BGR (3 channels ordered BGR)
• type : GL_UNSIGNED_BYTE (means 8 bits per channel)
  Here we tell OpenGL we want an RGB texture without alpha with channels ordered BGR and that our buffer is in the same size and order, so no conversion will occur.

When I did my PR, I assumed that the order you had set was mendatory because of the byteswapping you used.
So my goal was to find a faster way to replace this byteswapping and I used Java's Graphics instead, which uses native code, so it's faster we don't have to bother about the format or the source image.
But according to Khronos, this step could be avoided alltogether if we always pick the right parameters for glTexImage2D as demonstrated, but that means we need some big switch to handle all the BufferedImage.TYPE_* to handle all cases XD