New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Requests: GetStreamScreenshot #1189

Open

ghostzero wants to merge 1 commit into obsproject:master from ghostzero:feature/get-stream-screenshot

ghostzero commented Dec 11, 2023 •

edited

Loading

Description

Adds a new request called GetStreamScreenshot which returns a Base64-encoded screenshot of the stream (program).

Motivation and Context

I require this feature for a remote production where I cannot use remote desktop software to see the actual program in real time. Currently there is no way to take a screenshot of the program, which is "immutable" until the transition button is pressed. This feature allows you to take a screenshot of the main texture frame (which also contains stinger transitions). This PR also closes #1130.

WindowsTerminal_I5HJ3QmOBm.mp4

How Has This Been Tested?

Tested OS(s): Windows 11 22H2 22621.2715

I checked out the obs-studio repository, followed the build instructions, made changes to the plugins\obs-websocket folder, and built obs-studio to test my changes. Since this is mostly copy/paste from the source screenshot request with tweaks to capture the main texture, I expect no problems.

Types of changes

New request/event (non-breaking)

Checklist:

I have read the Contributing Guidelines.
All commit messages are properly formatted and commits squashed where appropriate.
My code is not on master or a release/* branch.
The code has been tested.
I have included updates to all appropriate documentation.

Author

ghostzero commented Jul 19, 2024

Is there anything blocking this?

norihiro reviewed

View reviewed changes

src/requesthandler/RequestHandler.cpp

               	{"StartStream", &RequestHandler::StartStream},
               	{"StopStream", &RequestHandler::StopStream},
               	{"SendStreamCaption", &RequestHandler::SendStreamCaption},
+              	{"GetStreamScreenshot", &RequestHandler::GetStreamScreenshot},

Contributor

norihiro Dec 19, 2023

I feel GetProgramScreenshot would be a better name because Stream is used for the output layer APIs. And, the implementation should be in RequestHandler_Outputs.cpp instead of RequestHandler_Stream.cpp. I want tt2468 to comment about the name as he has overhauled a lot of API names in version 5.0.

Author

ghostzero Jul 25, 2024

I agree, the GetProgramScreenshot seems like a better name since "Program" is also used in the user-facing part. But I wasn't sure about the naming conventions on obs-websocket, so yes, we may need @tt2468 feedback on this :)

My first idea was GetOutputScreenshot, but that was too generic. Then I renamed it to GetStreamScreenshot.

src/requesthandler/RequestHandler_Stream.cpp

Comment on lines +71 to +85

+              	gs_texrender_reset(texRender);
+              	if (gs_texrender_begin(texRender, imgWidth, imgHeight)) {
+              		vec4 background;
+              		vec4_zero(&background);
+              		gs_clear(GS_CLEAR_COLOR, &background, 0.0f, 0);
+              		gs_ortho(0.0f, (float)streamWidth, 0.0f, (float)streamHeight, -100.0f, 100.0f);
+              		gs_blend_state_push();
+              		gs_blend_function(GS_BLEND_ONE, GS_BLEND_ZERO);
+              		obs_render_main_texture();
+              		gs_blend_state_pop();
+              		gs_texrender_end(texRender);

Contributor

norihiro Dec 19, 2023

The function obs_get_main_texture() returns the main texture. This texture should be sufficient for this purpose.
If the requested geometry is same as the base size, just pass the texture to gs_stage_texture. If not, still need to use gs_texture_render to rescale.

src/requesthandler/RequestHandler_Stream.cpp

+              	obs_leave_graphics();
+              	return ret;
+              }

Contributor

norihiro Dec 19, 2023

The majority of this function TakeStreamScreenshot looks same as TakeSourceScreenshot.
Can we share the code? For example, creating QImage and copying the pixel data to QImage can be a common function.

src/requesthandler/RequestHandler_Stream.cpp

+               *
+               * @responseField imageData | String | Base64-encoded screenshot
+               *
+               * @requestType GetOutputScreenshot

Contributor

norihiro Dec 19, 2023

Is GetOutputScreenshot a typo? (This name sounds also good though.)

Author

ghostzero Jul 25, 2024

It's a typo

src/requesthandler/RequestHandler_Stream.cpp

Comment on lines +87 to +88

		gs_stage_texture(stageSurface, gs_texrender_get_texture(texRender));
		if (gs_stagesurface_map(stageSurface, &videoData, &videoLinesize)) {

Contributor

norihiro Dec 19, 2023

Just for your information and for a future improvement,
gs_stagesurface_map followed by gs_stage_texture is sometimes slow.
It might be better to release the graphics context, wait, and take the context again.

src/requesthandler/RequestHandler_Stream.cpp

Comment on lines +89 to +92

+              			int lineSize = ret.bytesPerLine();
+              			for (uint y = 0; y < imgHeight; y++) {
+              				memcpy(ret.scanLine(y), videoData + (y * videoLinesize), lineSize);
+              			}

Contributor

norihiro Dec 19, 2023

Just for your information,
between gs_stagesurface_map and gs_stagesurface_unmap, we may release the graphics context so that the graphics thread won't be blocked while copying the pixel data.


          Requests: GetStreamScreenshot

3faf2de

Adds a new request called `GetStreamScreenshot` which returns a
Base64-encoded screenshot of the stream (program).

ghostzero force-pushed the feature/get-stream-screenshot branch from 23a1543 to 3faf2de Compare

July 25, 2024 08:48

Xiaohei2-cn commented Sep 4, 2024

Hi, I want to get stream screenshot of the scene, how should I modify it, because the current function is to get the current stream, when I have more than one scene, he can only get the current stream.

Contributor

norihiro commented Sep 5, 2024 •

edited

Loading

Hi, I want to get stream screenshot of the scene, how should I modify it, because the current function is to get the current stream, when I have more than one scene, he can only get the current stream.

This is out of the topic but use GetSourceScreenshot.

Even if the request GetStreamScreenshot is not implemented, user can retrieve the screenshot by two steps; GetCurrentProgramScene and GetSourceScreenshot. Though the transition cannot be captured, assuming transition takes not a long period of time and the request won't be called frequently, I don't think there is much difference. So, I don't think this PR is necessary. In my opinion, if the request is called frequently, pipelined implementation to stage the texture and other encodings such as H.264 would be preferable.

Author

ghostzero commented Sep 11, 2024 •

edited

Loading

There is a significant discrepancy between using GetCurrentProgramScene and GetSourceScreenshot. The screenshot may not accurately reflect the current program content, especially if the source is being edited while the program scene remains static. This can lead to missleading displaying of programm content using both requests, which is why this PR is necessary to address the behavior.

I agree that for frequent calls, a pipelined implementation to stage the texture or use encodings like H.264 would be ideal. However, I’ve already encountered challenges with this implementation (as I am not an experienced C programmer), and if further optimization is required, I will definitely need assistance here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet