[BUG] Weird crosstalk between multiple canvas when using GPU-accelerated WPF with WindowsFormsHost

[AlexandreLaborde]

Description

I have a WPF application and I am using SkiaSharp to render my graphics.
This app creates multiple windows with several layers of big bitmaps and I was hitting a CPU bottleneck that was lowering my framerate down to 8 FPS.
I decided to look into ways to accelerate the rendering by using the GPU.
I tried to adapt the code from the SkiaSharpSample examples that uses a WindowsFormsHost to do the GPU accelerated rendering and, it works for a single window but it starts breaking when I have multiple windows.

In my original app, I have my bitmap data stored as byte[] and I thought that there might be some weird pointer-related voodoo going on there, so I decided to simplify my code to the point that all the windows have to do now is to fill the canvas with a solid color.
Nonetheless, it still fails to display so the bug must be on SkiaSharp's side.

I have looked into issues #665 #688 #745 #755 #764 and I could not find any solution there.
I have also changed the code from https://github.com/freezy/wpf-skia-opengl to work the way I need and it fails in similar ways so it might not be related to the WindowsFormsHost but with SKGLContext or SKGLControl instead, maybe 🤔? .

Code
This is the Window that is being rendered in the GPU

using SkiaSharp;
using SkiaSharp.Views.Desktop;

using System;
using System.ComponentModel;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Forms.Integration;

namespace SkiaSharpGPU
{
    public partial class SkiaSharpSampleWindow : Window
    {
        const int FPS = 10;

        long previousTicks = 0, currentTicks = 0;
        int interFrameInterval = (int)(1000.0 / FPS);
        double framerate = 0;
        Stopwatch framerateStopwatch = new Stopwatch();


        SKPaint paint = new SKPaint()
        {
            TextSize = 20f,
            IsAntialias = true,
            Color = SKColors.White,
            IsStroke = false,
        };

        int windowID;
        static int lastID = 0;

        SKColor color;


        bool windowIsClosing = false;

        public SkiaSharpSampleWindow(SKColor windowColor)
        {
            InitializeComponent();

            windowID = lastID++;
            Title = windowID.ToString();
            color = windowColor;

            framerateStopwatch.Start();
        }


        private void Window_Loaded(object sender, RoutedEventArgs e) => new Task(UpdateLoop).Start();

        private void Window_Closing(object sender, CancelEventArgs e) => windowIsClosing = true;


        private void UpdateLoop()
        {
            Thread.CurrentThread.Name = windowID.ToString();

            while (!windowIsClosing)
            {
                Thread.Sleep(interFrameInterval);
                Dispatcher.Invoke(glhost.Child.Invalidate);
                //Dispatcher.Invoke(skelement.InvalidateVisual);
            }

            Console.WriteLine($"Window {windowID} terminated.");
        }


        private void OnGLControlHost(object sender, EventArgs e)
        {
            var glControl = new SKGLControl();
            glControl.PaintSurface += OpenGL_PaintSurface;
            glControl.Dock = System.Windows.Forms.DockStyle.Fill;
            glControl.Name = "glControl " + windowID.ToString();

            var host = (WindowsFormsHost)sender;
            host.Child = glControl;
        }

        private void OpenGL_PaintSurface(object sender, SKPaintGLSurfaceEventArgs e)
        {
            PaintCanvas(e.Surface.Canvas, e.BackendRenderTarget.Width, e.BackendRenderTarget.Height);
        }

        private void Skelement_PaintSurface(object sender, SKPaintSurfaceEventArgs e)
        {
            PaintCanvas(e.Surface.Canvas, e.Info.Width, e.Info.Height);
        }

        private void PaintCanvas(SKCanvas canvas, int width, int height)
        {
            canvas.Clear();

            // Draw color
            canvas.DrawColor(color);

            // Draw window ID
            canvas.DrawText($"ID {windowID}", 50, 50, paint);

            //Draw simple FPS counter
            currentTicks = framerateStopwatch.ElapsedTicks;
            framerate = 1000.0 / ((currentTicks - previousTicks) * 1000.0 / Stopwatch.Frequency);
            previousTicks = currentTicks;
            canvas.DrawText($"FPS {framerate:F2}", 50, 100, paint);

            //Draw canvas handle
            canvas.DrawText($"Canvas {canvas.Handle}", 50, 150, paint);
        }
    }
}

Expected Behavior

I have created a new Window that uses a Task that invalidates the visual elements at a specific frequency.
When redrawing the element, it uses a predefined SKColor to fill the SKCanvas and then draws some text with the window ID, current framerate and canvas handle(just for debugging purposes).
There is a main Window with a button that when pressed creates one these GPU accelerated windows with a given color.
The expected behavior would be that each window is independent from each other and they are all able to update their color and text info.

Actual Behavior

When opening multiple windows simultaneously, only the last window will display any information the remaining being black.
Horizontally resizing the blacked windows trims the texts on the window that is working as if it was being resized but the background stays colored.
Vertically resizing the blacked windows causes the texts on the window that is working to move up and down.

When opening multiple windows taking some time in between, the first two windows open fine, from the third window on, all the windows became corrupted except the first and the most recent ones.
In this corrupted images, the color and text change randomly between then causing the windows to flicker and the text to be broken.
Once again resizing the corrupted windows causes the same resizing effects on the most recent window, but this time it blanks the broken window that was resized.

My guess is that there is some OpenGL buffer that is being shared across the windows or the canvases but I don't think there is any way for me to test this unless I go way deeper into the SkiaSharp's source code which I don't feel comfortable doing.

I have also tried the same rendering pipeline but using just the standard SKElement with SKPaintSurfaceEventArgs and all the windows work with no problem.
I am not sure if this is related but, when using the SKElement I noticed that the canvas handle changed every frame which does not happen when using the GPU accelerated version.

Basic Information

• Version with issue: 1.68.0
• IDE: Visual Studio 2019 Community
• Platform Target Frameworks:
  • Windows Classic: Windows 10 Pro 64bit 1903

Screenshots

When opening multiple windows simultaneously, only the last one is working even tough the tasks are still requesting the update.
0 should be red
1 should be green

By opening one window at a time it is possible to have several windows open.

Having more than three windows open causes random glitches in the windows.
Window 5 keeps changing between the correct form (green) and a abnormal version (red)

Resizing broken windows affects the content of other windows

Reproduction Link

Code for the classes used in the test project
code.zip

[AlexandreLaborde]

@Redth @mattleibow I have found and fixed the bug in the SKGLControl that is causing this issue. Should I make a pull request?

[idotta]

I have the same issue. Do you have the solution in a fork? Or maybe you could do the PR.

[AlexandreLaborde]

@idotta Since I did not receive any approval from any of the mods I decided not to do the PR.
The problem lies in the way the SKGLControl uses the control it gets from OpenTK.

The OpenTK documentation specifies that when using multiple GLControls you need to specify which control you are using at that time.

You do that by calling MakeCurrent() on one control. This will make all other controls non-current in the calling thread.

Going back to SkiaSharp. I had to create my own SKGLControl that calls the MakeCurrent() function before painting the canvas.

You can do that by doing something like this:

    class ConcurrentSKGLControl : SKGLControl
    {
        protected override void OnPaint(PaintEventArgs e)
        {
            MakeCurrent();
            base.OnPaint(e);
        }
    }

Now you just follow SkiaSharp's documentation but you use this new class in the place of the standard SKGLControl.

I'm not sure if it is the most optimal implementation of this, of even if you need to call MakeCurrent every time your drawing but at least this fixes the weird cross-talk issues I had.

Let me know if this helps fixing your problems as well.

[Gillibald]

Just spawn a new thread for each window and call MakeCurrent once per thread / window.

[AlexandreLaborde]

@Gillibald In my example, every window was running on its own thread and I've tried what you suggested and that does not seem to fix this issue.

In addition, the MakeCurrent can only be called through the SKGLControl. The only way I got your suggestion not to crash was by removing the glHost variable from the OnGLControlHost function and keep it as a class variable. Then use Dispacher.Invoke so that the correct thread is calling MakeCurrent

[Gillibald]

SKGLControl should call MakeCurrent when it is created. If you really spawn a new UI thread per window there shouldn't be an issue. Heaving multiple SKGLControl per thread should not be allowed unless you can control the GRContext like you did in your example.

[AlexandreLaborde]

But don't I need to all of this if I want to render all objects using the GPU ? I was just following the official examples.

[Gillibald]

You can only have one GRContext per thread. If you want to have more than one you have to deal with MakeCurrent calls. That is expected behavior. Calling MakeCurrent on every render call works but I am not sure if that is the best way to do it when you want to draw on a window's surface.

[mattleibow]

Hi @AlexandreLaborde sorry about the incredibly long wait. Just so you know, this is an open-source project and you can just send in PRs - no need to ask!

The SKGLControl has not received that much love at all, so could very well have a few bugs. It does not get the same level of support as Android and iOS.

@Gillibald thanks for getting in on the discussion.

@AlexandreLaborde did you try calling MakeCurrent in the constructors - as opposed to the paint method?

[jensbrak]

I just found this and it explains why I get black surfaces when trying to have two SKGLControls. I tried two controls just out of curiosity (previously had one single control and did all painting from a single PaintSurface but thought it would be more logical to have two separate canvases and PaintSurfaces, one for each control).

One GRContext per thread - guess that means one SKGLControl per thread. If the need to do MakeCurrent on every render - would it be better to have separate threads instead?

[mattleibow]

I was looking at the general OpenGL world, and it does seem to be a thing to call "make current" before rendering - even if there is multiple views and a single thread. I did some tests (thanks to the attached code) and this seems to fix it.

I am going to go with this a fix unless there is some serious downside. But, I think the alternative is only a single GL view per thread, which is more of a downside.

[AlexandreLaborde]

@mattleibow Sorry for being away from this thread for such a long time and forgetting to reply to the message you sent in November. Is there still something I can help with ?

[mattleibow]

@AlexandreLaborde I think you gave enough information, so all is good. I got what I think is a fix just merged in. I'll try get a preview out soon so you can test it. I basically just do what you did and call MakeCurrent as this is the correct (as far as I can see) way to handle multiple drawing surfaces on a single thread.

Thanks for the issue and continuing info over several months. SkiaSharp just got better.