D3D11 vs Other backends

[st0rmbtw]

D3D11 gives the best performance among all the rendering backends that LLGL supports.

D3D12 is much slower than D3D11 (even though it's supposed to be faster according to tests on the internet and because it gives more control).
Vulkan is also slower than D3D11 but faster than D3D12.

Is it possible to achieve the D3D11's performance with the other backends?

In my project I get:

Backend	FPS
D3D11	~6000 😲
D3D12	850-1000
Vulkan	~2700
OpenGL	?*
Metal	?**

*: I didn't manage to test OpenGL because I can't attach to OpenGL renderer via RenderDoc for some reason. I get this error:

failed to create extended OpenGL profile
in 'ErrUnsupportedGLProc': illegal use of unsupported OpenGL procedure: glCreateShader

**: Didn't manage to test Metal properly as well, because I have a poor GPU on Mac.

Some notes

WriteTexture is very slow on D3D12 (On D3D11 it takes less than 1 ms, while on D3D12 it takes ~2 seconds to write the same amount of data to a texture)
WriteTexture is also slower on Vulkan than on D3D11 but faster than on D3D12.

Vulkan reports a ton of validation errors every frame for some reason.
Vulkan crashes often when I recreate large buffers.

Tests were done with the following conditions

OS: Windows 11 23H2
CPU: Ryzen 2700X
GPU: RTX 2060
Compiler: clang 18.1.8
Compiler options: -g0, -O3, -fomit-frame-pointer, -fno-math-errno, -ffinite-math-only, -fno-signed-zeros, -fno-trapping-math, -fassociative-math
Debug layer: Disabled

[LukasBanana]

I'm curiously following your project :) I'm also working on a project that's a lot larger than the examples included in this repository, in hope to have a benchmark/demo project to showcase and test LLGL with a larger use case, but that project is far from ready to be published.

I can also confirm that the D3D11 backend is the fastest LLGL has and beating it is a tough task. Having said that, in my project I'm seeing D3D12 at about 50% speed compared to D3D11, so 850 FPS compared to 6000 sounds extreme and so does ~2 seconds sound in the WriteTexture call. These calls are indeed poorly synchronized with the GPU, so they use a worst case approach that stalls the GPU, but 2 seconds like a different problem. Can you share some more details about these calls specifically? What type of texture data and size?

I have no trouble debugging the GL backend in RenderDoc. I assume you have the latest version? Can you share your RenderDoc parameters?

Vulkan is still in an early state and I'm trying to address validation errors as I get reports, i.e. I see people actively using it.
Can you share some details about what buffer sizes you re-create? I can certainly spin up a unit test for that.

[st0rmbtw]

Oh, nice to hear that you're following my project 😊. Feel free to use it as a test field for LLGL.

In my project, I recreate buffers (tile data) of a chunk when it gets modified (when a block is set or removed), you can see the code here: https://github.com/st0rmbtw/TerrariaCloneCpp/blob/6b38f18012b489192c0ebe373278f31e41b3e163/src/world/chunk.cpp#L102-L124

Honestly, I don't know exactly how long WriteTexture takes on D3D12, I judged by feelings :). But it really muuuch slower than on D3D11. I use it here: https://github.com/st0rmbtw/TerrariaCloneCpp/blob/6b38f18012b489192c0ebe373278f31e41b3e163/src/renderer/world_renderer.cpp#L120

Though 6000 fps on D3D11 is real, I measured it via RenderDoc.

I have no trouble debugging the GL backend in RenderDoc. I assume you have the latest version? Can you share your RenderDoc parameters?

I think it may be something with compilation flags because I can debug it when it is not built for release (i.e. zero optimizations).

[LukasBanana]

I hope you aren't using a debug build for the performance test. You mentioned debug layer is disbaled, so I presume you did not, but just to be sure, because I see you're using DebugDevice flags here, which is great for runtime validation, I still want to emphasize that it's tremendously slower on D3D12 than D3D11 to my experience.

With GL acting weird in optimized code, I would add some logs/printfs into GLCoreExtensionLoader.cpp to see what extensions are available when launching through RenderDoc and if ARB_shader_objects is called to initialize those function pointers.

A little bit off topic:
While it's certainly desirable to get D3D12 and all of its feature set on par with D3D11 - and one day hopefully beyond that - I'd like to point out that the RenderSystem::Write... functions are generally not tuned for the highest performance, their CommandBuffer counterparts are, such as UpdateBuffer. These of course come with limitations and textures don't even have such functions (you would have to update a buffer and then copy that buffer into the texture). Not only are they favorable for performance, you also have to keep in mind that the RenderSystem functions should not be inlined with command buffer recording, especially not in D3D12, Vulkan, and Metal. In other words, command buffers cannot rely on buffer modifications via WriteBuffer while they are being recorded because those commands have not been submitted to the GPU yet. Even if you use an immediate context (i.e. CommandBufferFlags::ImmediateSubmit), those commands will only be submitted to the GPU once recording is finished. So when you update a buffer via WriteBuffer more than once during command buffer recording, its previous content will be overwritten before a command buffer was able to use it. Maybe an example can illustrate that better:

cmd->Begin();
cmd->SetResource(0, *buffer);
renderer->WriteBuffer(*buffer, /* Value A */);
cmd->Dispatch(1,1,1); // Wants to read value A from "buffer"
renderer->WriteBuffer(*buffer, /* Value B */);  <-- BOOM: overrides value A before the command buffer had a chance to read it
cmd->Dispatch(1,1,1); // Reads same value as previous Dispatch
cmd->End();

I see you prepared your project for large buffer updates. Calling WriteBuffer during command recording is not a problem per se, but you have to keep track of where you are writing to and point your shaders to the next location until you submit your command buffer or reach End() on an immediate context. This could look something like this:

uint64_t currentOffset = 0;
cmd->Begin();
cmd->SetResource(0, *buffer);

renderer->WriteBuffer(*buffer, currentOffset, &valueA, sizeof(valueA));
cmd->SetUniform(0, currentOffset); // Tell shader from where to read in "buffer" for next Dispatch
currentOffset += sizeof(valueA);

cmd->Dispatch(1,1,1); // Reads value A

renderer->WriteBuffer(*buffer, currentOffset, &valueB, sizeof(valueB));
cmd->SetUniform(0, currentOffset); // Update offset where to read from
currentOffset += sizeof(valueB);

cmd->Dispatch(1,1,1); // Reads value B
cmd->End();
queue->Submit(*cmd); // This is where "buffer" is finally being accessed by the GPU and all writes are becoming visible

With UpdateBuffer, LLGL already does that for you in the APIs that require explicit command buffer submission, including pooling when a buffer exceeded its size. The size limitation comes directly from Vulkan's vkCmdUpdateBuffer() specification as these updates may be part of the command buffer memory.

I updated the documentation on those functions in 34e5041.

[st0rmbtw]

I hope you aren't using a debug build for the performance test.

No, I used a release build. I provided the optimisation flags I used for the tests in the description.
I define the DEBUG macro for debug build and if it's defined I initialize the validation layer.

the RenderSystem functions should not be inlined with command buffer recording, especially not in D3D12, Vulkan, and Metal

What's the recommended way to use it then? End recording of a command buffer before using a function and begin recording again after? Doesn't often ending and beginning command buffer recording hit performance?

Would splitting the data into chunks of 65536 bytes and uploading each of them to a buffer using CommandBuffer::UpdateBuffer be more efficient?

[LukasBanana]

End recording of a command buffer before using a function and begin recording again after? Doesn't often ending and beginning command buffer recording hit performance?

There are multiple ways to achieve this:

1. Your suggestion would indeed be a way. I honestly can't tell you what the performance impact is, that's something only empirical data can tell. I can imagine the D3D11 driver does this type of scheduling.
2. If you have to update buffers larger than 2^16 bytes during command recording, you can use the approach I illustrated above using segments of a larger buffer and point your shaders to the correct offset.
3. You could also allocate multiple buffers of the same size and then update the buffer and swap it out with the previous one using SetResource.
4. If your buffers are only slightly larger than the 2^16 limit, you could also encode the update with multiple UpdateBuffer commands, since the destination offset has 64 bit. Only the length of each update is limited to 16 bit.
5. Maybe you even have a way of delaying the update to the next frame, so you can call WriteBuffer between command recordings.

I recently added a new example for a mini puzzle game that deals with a similar problem to make it all work in WebGL where SSBOs are not supported, i.e. you are restricted to uniform buffers that are quite limited in size, too. Take a look at the InstanceBuffer class and how it splits up draw calls when storage buffers (aka SSBOs) are not supported.