Hardware accelerated DynamicSprite

[ericoporto]

Describe the problem
Dynamic sprites are currently manipulated in software mode only, writing an implementation for 3D renderers too might enhance rendering perfomance and make further expansion of its API with new effects more interesting. But this may also require adjusting a way of how sprites are refered to and handled in the engine.

Suggested change
Looking at the ScriptAPI for Dynamic Sprites, the Crop, Resize, Rotate and Tint are things that can make sense either by applying it to all pixels and actually modifying the pixel data (the surface) or through a transformation matrix or some graphics renderer command (for Tint).

There is an issue that once these are done through transformation, then operating the pixel data for these will require an API for it (DrawingSurface?), and that this change is not backwards compatible (ags4?). Or perhaps some parameter would specify if the change is in the data or in the containers (eSurface, eAccelerated or eDestructible, eNondestructible).

This will create an additional hierarchy/abstraction for the Sprites, and maybe an additional relationship between character, object sprite transformation on top of this additional transformation.

We may also back-out of this if we decided is not needed, overall there was no ticket for it reference in #1298 , so I just wanted to open one. I also wondered if #969 was related to this.

additional context

• https://www.adventuregamestudio.co.uk/forums/engine-development/ags-high-priority-improvement-needs/msg636636366/#msg636636366

  I think a more powerful sprite manipulation API via DynamicSprite and DrawingSurface â€" or perhaps support for shaders â€" would provide more value (...) Better performance for certain graphical operations

• https://www.adventuregamestudio.co.uk/forums/engine-development/ags-high-priority-improvement-needs/msg636636380/#msg636636380
• allegro 5: al_hold_bitmap_drawing, allegro help on it.
• allegro 5: fastest way to draw many (many!) primitives

[ivan-mogilko]

The idea about accelerated dynamic sprite is to make it a "operation batch", where the operations are stored as a list of commands and performed at the render stage, instead of being applied immediately to pixels.

In other words, when you call DynamicSprite.Rotate, but also DrawingSurface.DrawImage, or DrawLine, and so forth, the function does not apply this change right away, but pushes a command into the list. Ideally, this command list is executed whenever is most convenient and efficient for the current renderer. Like, for software renderer it does not matter, so it may do that right away, video texture renderer executes 3D drawing and transformation at the render stage instead.

This implies that graphics driver interface should be expanded, instead of having just DrawSprite, it should have an interface of a draw command (or "operation") items and lists. An example of this may be found in SDL2's Renderer code, I believe (I might find this again later).
EDIT: here's the random example in SDL2's code:
https://github.com/libsdl-org/SDL/blob/a4ad293d4a184057c6dc370538ddd808ca9b0c90/src/render/SDL_render.c#L136

Somewhat on the downside there's a case where you want to access DynamicSprite's surface to get pixels. In this case, it's likely that all the saved operations will have to be performed right away.
Optionally, this could be optimized (maybe) by setting up a DynamicSprite's mode, like you mentioned (accelerated/non-accelerated). But this is more of a secondary question.

This task should not be rushed, and requires a good task list.

• Probably starting with changes to graphics driver interface, adding an extendable "operation" support, including drawing primitives,
• Then modifying the DynamicSprite into a list of commands, and changing DrawingSurface to push commands instead of drawing them, depending on the mode.
• Anything else I did not think of here.

[ericoporto]

Uhm, the way you propose is very different than what I had in mind, but with what you propose, I think the hardest thing but that can be useful would be having a command for the text drawing (thinking about drawing text in screen resolution).

Ah, dear imgui has a command system too, with compatibility with lots of different renderers, perhaps something to look into.

[ivan-mogilko]

Right, this may be done in two major stages:

1. Stage one: DynamicSprite transformations.

• DynamicSprite is a list of operations, which include only image transformation and effects for now: Rotate, Resize, Tint, etc.
• DynamicSprite's list of transforms is translated to the texture settings when adding sprites to the renderer. I think our texture (DDB) interface already supports all these that dynamic sprite supports, with the exception of Crop and ChangeCanvasSize(). One thing that I am not certain about is whether DDB interface is enough for multiple consecutive transforms, as DynamicSprite may be transformed several times with operations going in any order; for example: Rotate, Resize, Rotate again, and so on. It's possible that we might have to support passing a prepared matrix into DDB instead.
• Graphics driver interface does not require any major changes at this stage, it seems.
• DynamicSprite will have to execute (flush) its operations in software mode when you request a DrawingSurface.
• Objects with this sprite assigned must be using the dynamicsprite's matrix in addition to their own when testing for bounds, and also pixels in pixel perfect mode ("is pos on character" and so forth).
  EDIT: for the reference, see GraphicsSpace struct in ags4:
  

  ags/Common/gfx/gfx_def.h

  Line 58 in 6d7ec60

  class GraphicSpace

2. Stage two: accelerated DrawingSurface.

• DynamicSprite extends its operations list to drawing primitives, clearing with color, etc.
• DrawingSurface keeps the API, but internally switches between executing a raw drawing or adding a command into the dynamicsprite's list.
• Graphics driver must support the drawing operations, including drawing primitives, and text (apparently?). Instead of "list of sprites" it has to have a "list of commands".

These stages may of course be documented as separate tickets.