Memoryless tiler

[eugenegff]

RSC_TILER + TextureFlags::Tiler[Depth]Memoryless => MTLStorageModeMemoryless and VK_IMAGE_USAGE_TRANSIENT_ATTACHMENT_BIT

[dyunchik]

First of all, we should be able to globally disable the use of memoryless. Because some very heavy scenes can give an out of tile memory error and simply not render anything (we had such cases on Metal in Live Home 3D). For this purpose we introduced "WindowMemoryless" param. Actually it's almost the same as "memoryless_depth_buffer". But if "WindowMemoryless"/"memoryless_depth_buffer" is false, then we have to remove MTLStorageModeMemoryless from depth buffer and also from MSAA buffer as well. Indeed, this way Metal will be able to split rendering into several passes with intermediate results stored in these buffers, and Metal does this. I don't know about Vulcan, but I think he rules in the same way.

[dyunchik]

Second, we should be able to use memoryless for depth only when we render to MSAA texture. For this purpose we need something like TextureFlags::TilerDepthMemoryless.
For example:

compositor_node PlanarReflectionsFSAAReflectiveRenderingNode
{
	in 0 rt_renderwindow
    in 1 planarReflect_msaa_rtt
 
	target planarReflect_msaa_rtt
	{    
		pass render_scene
		{
			load
			{
				all				clear
				clear_colour	0.2 0.4 0.6 1
			}
			store
			{
				colour	store_or_resolve
				depth	dont_care
				stencil	dont_care
			}

			overlays		off
			visibility_mask 0xfffffffe
   
            rq_first    0
            rq_last     250
		}
	}
 
    target rt_renderwindow
    {
        pass render_quad
        {
            load { all dont_care }
            material Ogre/Copy/4xFP32
            input 0 planarReflect_msaa_rtt
        }
        
		pass generate_mipmaps
		{
			mipmap_method api_default
		}
    }
}

Here we use ordinary MSAA texture planarReflect_msaa_rtt and memoryless depth when we render into target planarReflect_msaa_rtt create with following code:

            Ogre::RenderSystem* rs = root->getRenderSystem();
            if(rs)
            {
                Ogre::TextureGpuManager *textureGpuManager = rs->getTextureGpuManager();
                if(textureGpuManager)
                {
                    const RenderSystemCapabilities *capabilities = rs->getCapabilities();
                    const bool isTiler = capabilities->hasCapability( RSC_IS_TILER );
                    const size_t uniqueId = Id::generateNewId<CSceneManager>();
                    Ogre::PixelFormatGpu pixelFormat = Ogre::PFG_BGRA8_UNORM_SRGB;
                    uint32 textureFlags = TextureFlags::RenderToTexture ;
                    if(isTiler)
                        textureFlags |= TextureFlags::TilerDepthMemoryless;
                    msReflectionSharedMSAAWorkTexture = textureGpuManager->createTexture(
                                        "ReflectionSharedMSAAWorkTexture" + StringConverter::toString( uniqueId ),
                                        Ogre::GpuPageOutStrategy::Discard, textureFlags, TextureTypes::Type2D );
                    msReflectionSharedMSAAWorkTexture->setOrientationMode(orientationMode);
                    msReflectionSharedMSAAWorkTexture->setResolution( resolution, resolution );
                    msReflectionSharedMSAAWorkTexture->setPixelFormat( pixelFormat );
                    msReflectionSharedMSAAWorkTexture->setSampleDescription(sampleDesc);
                    msReflectionSharedMSAAWorkTexture->setNumMipmaps(1u);
                    msReflectionSharedMSAAWorkTexture->_transitionTo( GpuResidency::Resident, (uint8*)0 );
                }
            }

So to be flexible, it would be great to have texture flags: TilerMemoryless and TilerDepthMemoryless

[dyunchik]

Hm, may be the special memoryless depth pool is enough and we may avoid TilerDepthMemoryless

[dyunchik]

However, we still need the ability to disable memoryless mode for implicit MSAA along with depth. For example when too many triangles are visible and we run out of tile memory.

[darksylinc]

Because some very heavy scenes can give an out of tile memory error and simply not render anything (we had such cases on Metal in Live Home 3D).

Oh! I didn't know this. Then yes, we need a global toggle.

The only issue I see is that such toggle requires quite a bunch of resources to be recreated (if you want it to be toggled at runtime and not at startup).

Regardless Vulkan, ARM Mali has a 180MB limit (memoryless or not). That 180MB is in the data that gets sent from VS to PS, i.e. position + interpolators data.

ARM published more details about this yesterday, see page 13.

Hm, may be the special memoryless depth pool is enough and we may avoid TilerDepthMemoryless

Yes, the memoryless depth pool should be enough.

[eugenegff]

Because some very heavy scenes can give an out of tile memory error and simply not render anything (we had such cases on Metal in Live Home 3D).

Oh! I didn't know this. Then yes, we need a global toggle.

So, what should we do? Take current iteration as is, and add support for opt-out for memoryless window color buffers later, or keep tweaking? In current state with memoryless depth buffers and render textures - it is already quite useful

[darksylinc]

OK I just pushed the necessary change:

Added "Allow Memoryless RTT" RenderSystem option which calls TextureGpuManager::setAllowMemoryless.

If all is well, it can be merged.

PS: I noticed clang format script was ignoring the Vulkan folder 😱
We'll fix that once that is merged to avoid any potential merge conflict (fortunately the changes are not much).