Support sharing texture from cef on Windows #1217

ducthiem90 · 2019-06-08T04:42:25Z

Solve the problem at #1177
Need update cef to 3.3578.1870 (for support sharing texture)
Support only Windows

Normal not use sharing:

Use sharing:

Julusian · 2019-06-08T16:21:20Z

Thanks.
I'll give this a proper test and review soon.
Have you tried running multiple html producers? It would be good to confirm it does work with many active. In theory this should be limited by the gpu hitting 100% usage.

ronag · 2019-06-08T16:28:19Z

Please move the dx11 stuff out to a separate file

ducthiem90 · 2019-06-08T16:39:07Z

Thanks.
I'll give this a proper test and review soon.
Have you tried running multiple html producers? It would be good to confirm it does work with many active. In theory this should be limited by the gpu hitting 100% usage.

I tested with 4 layer html with video mode 1080p50. And results:

Diagnostics

- Screen with mixer grid 2:

- And GPU usage(NVIDIA Quardro M2000M)

ducthiem90 · 2019-06-08T16:50:56Z

Please move the dx11 stuff out to a separate file

Ah sorry. I don't know to add files with cmake :(

Julusian · 2019-06-10T17:41:38Z

Ah sorry. I don't know to add files with cmake :(

Yeah, I have found that Visual studio does not make this easy to do.

The best way I have done this is to manually create the file in src/modules/html/... and add it to src/modules/html/CMakeLists.txt copying the lines for other files, adjusting as needed.

Julusian · 2019-06-08T16:10:37Z

src/modules/html/producer/html_producer.cpp

+
+                            auto image_size = rowBytes * height;
+
+                            memcpy(frame.image_data(0).begin(), reinterpret_cast<uint8_t*>(map.pData), image_size);


I'm not keen on this memcpy. It would be better if we could do a native gpu copy into a opengl texture, to save having to reupload it before the mixer can use the frame.

We can't do a native gpu copy into opengl texture, because shared texture can't access by cpu

Julusian · 2019-06-08T16:12:49Z

src/modules/html/producer/html_producer.cpp

+
+                        // Mapping surface data to memory
+                        D3D11_MAPPED_SUBRESOURCE map;
+                        if (SUCCEEDED(dx11_device_->immedidate_context()->context()->Map(


It looks like we are doing some potentially quite slow work in this callback. It might be better to throw this onto an executor to do, so that we don't block up the cef event loop.

The shared texture will be used by next render, so, could not throw it onto an executor to do

Julusian · 2019-06-08T16:18:45Z

src/modules/html/producer/html_producer.h

 namespace caspar { namespace html {

+#ifdef WIN32
+
+class dx11_device final


I think this should be made into a singleton, as it is being constructed a few times just to check if it is available.

Julusian · 2019-06-08T16:27:48Z

src/CMakeModules/Bootstrap_Windows.cmake

-	set(CEF_BIN_PATH "${NUGET_PACKAGES_FOLDER}/cef.redist.x64.3.3239.1723/CEF")
-	set(CEF_RESOURCE_PATH "${NUGET_PACKAGES_FOLDER}/cef.redist.x64.3.3239.1723/CEF")
-	link_directories("${NUGET_PACKAGES_FOLDER}/cef.sdk.3.3239.1723/CEF/x64")
+	set(CEF_INCLUDE_PATH "${NUGET_PACKAGES_FOLDER}/cef.sdk.3.3578.1870/CEF")


Any particular reason for this version of CEF? There look to be newer on nuget. I ask because I will need to bump to 74 sometime soon for #669

I tested in cef 74, but cef 74 seem have bug with fps, it render very fast

ronag · 2019-06-10T17:59:56Z

Can we use WGLEW_NV_DX_interop?

ducthiem90 · 2019-06-10T18:14:34Z

Can we use WGLEW_NV_DX_interop?

May be. I will try

Julusian · 2019-06-10T18:26:45Z

I looks like I was using WGLEW_NV_DX_interop when trying this myself, but its been a few months since I was doing this so I don't really remember what I was doing. In case any of it is useful for you, but be warned some of it is a hacky mess: https://github.com/CasparCG/server/compare/master...Julusian:feature/cef_gpu_texture?expand=1

ducthiem90 · 2019-06-10T18:58:19Z

Ah sorry. I don't know to add files with cmake :(

Yeah, I have found that Visual studio does not make this easy to do.

The best way I have done this is to manually create the file in src/modules/html/... and add it to src/modules/html/CMakeLists.txt copying the lines for other files, adjusting as needed.

I moved the dx11 stuff out to a separate file(dx11.h and dx11.cpp)

ducthiem90 · 2019-06-18T19:07:28Z

Hi @Julusian
I used WGLEW_NV_DX_interop to copy directx texture2d to gl texture2d.
You can recheck it.
#1217

ronag · 2019-07-07T19:11:44Z

I haven't actually compiled this. But I'm quite impressed and happy with what I see. @Julusian if you can compile and verify I'm ok with this being merged.

ronag · 2019-07-07T19:12:46Z

src/modules/html/producer/html_producer.cpp

+                    std::lock_guard<std::mutex> lock(frames_mutex_);
+
+                    frames_.push(dframe);
+                    while (frames_.size() > 8) {


This buffer size should probably be configurable.

mryvz · 2019-09-03T13:16:07Z

I've applied changes and looks like working well, but html text scroll (Ticker) is not smooth, is it possible its not synchronized with requestanimationframe loop ?

Julusian

Overall this looks good.
The only things from me are to update CEF to 73, and to fix it so that caspar can still run on non-nvidia gpus.

Julusian · 2019-09-14T20:10:47Z

src/accelerator/ogl/util/texture.cpp

@@ -74,6 +74,14 @@ struct texture::impl

    void clear() { GL(glClearTexImage(id_, 0, FORMAT[stride_], TYPE[stride_], nullptr)); }

+#ifdef WIN32
+    void copy_from(int texute_id)


Is there a reason this method is windows only?
typo in variable name.

Julusian · 2019-09-14T20:13:41Z

src/accelerator/ogl/util/device.cpp

+            GL(glFlush());
+
+            deadline_timer timer(service_);
+            for (auto n = 0; true; ++n) {


Will the opengl thread be blocked while this copy is happening?

Julusian · 2019-09-14T20:14:33Z

src/accelerator/ogl/util/device.cpp

    io_context                          service_;
    decltype(make_work_guard(service_)) work_;
    std::thread                         thread_;

    impl()
        : device_(sf::ContextSettings(0, 0, 0, 4, 5, sf::ContextSettings::Attribute::Core), 1, 1)
+#ifdef WIN32
+        , d3d_device_(d3d::d3d_device::get_device())


We should be able to handle failure when creating this gracefully.

Julusian · 2019-09-14T20:16:41Z

src/modules/html/html.cpp

+        if (enable_gpu) {
+            auto dev = accelerator::d3d::d3d_device::get_device();
+            if (!dev)
+                CASPAR_LOG(warning) << L"Failed to create directX device";


This log line could be better

Julusian · 2019-09-14T20:18:51Z

src/CMakeModules/Bootstrap_Windows.cmake

+	set(CEF_INCLUDE_PATH "${NUGET_PACKAGES_FOLDER}/cef.sdk.3.3578.1870/CEF")
+	set(CEF_BIN_PATH "${NUGET_PACKAGES_FOLDER}/cef.redist.x64.3.3578.1870/CEF")
+	set(CEF_RESOURCE_PATH "${NUGET_PACKAGES_FOLDER}/cef.redist.x64.3.3578.1870/CEF")
+	link_directories("${NUGET_PACKAGES_FOLDER}/cef.sdk.3.3578.1870/CEF/x64")


We should bump this to cef 73. Newer than that has issues, but I havent found any in 73 yet.

Julusian · 2019-09-20T19:44:04Z

src/accelerator/ogl/util/device.cpp

@@ -100,6 +113,18 @@ struct device::impl : public std::enable_shared_from_this<impl>

        device_.setActive(false);

+#ifdef WIN32
+        if (d3d_device_) {
+            interop_handle_ = std::shared_ptr<void>(wglDXOpenDeviceNV(d3d_device_->device()), [](void* p) {


This can segfault as wglDXOpenDeviceNV is not available on intel.
If this happens then just skip the whole shared texture stuff there, we can look into alternative api in a later PR.

Julusian · 2019-11-22T12:04:15Z

This has been manually merged to master.
It is still using CEF71 as 73 has some framerate issues

Support sharing texture from cef on Windows

2fbb5e0

Julusian requested changes Jun 10, 2019

View reviewed changes

move the dx11 stuff out to a separate file

e1f7e91

ducthiem90 added 4 commits June 17, 2019 11:17

move directx11 to accelerator project

6f0865d

Create cmake for directx11

e785a08

create executor for create frame from d3d texture on html producer

29d49aa

Use WGLEW_NV_DX_interop to copy directx texture to gl texture

cfb0581

Move d3d_device_interop to ogl device

252e108

ronag reviewed Jul 7, 2019

View reviewed changes

Julusian mentioned this pull request Jul 8, 2019

JavaScript Engine Update #1221

Closed

Julusian requested changes Sep 20, 2019

View reviewed changes

Julusian mentioned this pull request Oct 6, 2019

CEF Upgrade #1235

Open

Julusian closed this Nov 22, 2019

sirfnomi mentioned this pull request Feb 19, 2020

Texture Sharing Support nrkno/sofie-casparcg-server#37

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support sharing texture from cef on Windows #1217

Support sharing texture from cef on Windows #1217

ducthiem90 commented Jun 8, 2019

Julusian commented Jun 8, 2019

ronag commented Jun 8, 2019

ducthiem90 commented Jun 8, 2019

ducthiem90 commented Jun 8, 2019 •

edited

Loading

Julusian commented Jun 10, 2019

Julusian Jun 8, 2019

ducthiem90 Jun 10, 2019

Julusian Jun 8, 2019

ducthiem90 Jun 10, 2019

Julusian Jun 8, 2019

Julusian Jun 8, 2019

ducthiem90 Jun 10, 2019

ronag commented Jun 10, 2019

ducthiem90 commented Jun 10, 2019

Julusian commented Jun 10, 2019

ducthiem90 commented Jun 10, 2019

ducthiem90 commented Jun 18, 2019

ronag commented Jul 7, 2019 •

edited

Loading

ronag Jul 7, 2019 •

edited

Loading

mryvz commented Sep 3, 2019 •

edited

Loading

Julusian left a comment

Julusian Sep 14, 2019

Julusian Sep 14, 2019

Julusian Sep 14, 2019

Julusian Sep 14, 2019

Julusian Sep 14, 2019

Julusian Sep 20, 2019

Julusian commented Nov 22, 2019


		auto image_size = rowBytes * height;

		memcpy(frame.image_data(0).begin(), reinterpret_cast<uint8_t*>(map.pData), image_size);

Support sharing texture from cef on Windows #1217

Support sharing texture from cef on Windows #1217

Conversation

ducthiem90 commented Jun 8, 2019

Julusian commented Jun 8, 2019

ronag commented Jun 8, 2019

ducthiem90 commented Jun 8, 2019

ducthiem90 commented Jun 8, 2019 • edited Loading

Julusian commented Jun 10, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ronag commented Jun 10, 2019

ducthiem90 commented Jun 10, 2019

Julusian commented Jun 10, 2019

ducthiem90 commented Jun 10, 2019

ducthiem90 commented Jun 18, 2019

ronag commented Jul 7, 2019 • edited Loading

ronag Jul 7, 2019 • edited Loading

Choose a reason for hiding this comment

mryvz commented Sep 3, 2019 • edited Loading

Julusian left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Julusian commented Nov 22, 2019

ducthiem90 commented Jun 8, 2019 •

edited

Loading

ronag commented Jul 7, 2019 •

edited

Loading

ronag Jul 7, 2019 •

edited

Loading

mryvz commented Sep 3, 2019 •

edited

Loading