GS/HW: Improve Round Sprite upscaling fix to cause less problems [TEST]

[refractionpcsx2]

Description of Changes

Attempt to reduce the problems associated with Round Sprite (Especially Full Round Sprite)

Rationale behind Changes

It was pretty awful and caused a whole bunch of problems, this tries to mitigate the worst ones.

Suggested Testing Steps

Test any game which uses round sprite, or needed it but it caused problems.

[bigol83]

Couple of games tested

This greatly improves Dragon Quest VIII menu highlight

PR

Master

But it makes text in Final Fantasy X slightly worse

PR

Master

Dragon Quest VIII uses Half Round Sprite to fix text
Final Fantasy X uses Full Round Sprite to fix text

[refractionpcsx2]

yeah, there's not a lot in that, it looks wrong to begin with :P

[cyanea-bt]

Hi, here's some comparisons using Full Round Sprite on MGS2 and MGS3. First MGS2:
Text artifacts on some UI elements look about the same on both versions (vertical lines next to some letters):

old:

new:

old:

new:

But one big improvement on this new version - it seems like it doesn't break DoF like the old version used to do. Awesome!
old:

new:

---

Now for MGS3, which doesn't use roundSprite at the moment but imo could benefit from it. For example this upscaling artifact on the right and bottom sides of videos that play on the codec screen:

without roundSprite:

old roundSprite:

new roundSprite (looks the same as old version here):

But sadly in MGS3 this new version still breaks DoF, just like the old version did. Maybe it doesn't play nicely with the Renderfix (#8991) that MGS3 needs for it's DoF effect? Or idk what else could be the reason for it working in MGS2 but not in MGS3. I added the roundSprite fix to MGS3 via its entry in GameIndex.yaml to ensure that all other fixes it needs stay intact. Just in case, this is what the modified entry looks like:

SLUS-21359:
  name: "Metal Gear Solid 3 - Subsistence [Disc 1 of 3]"
  region: "NTSC-U"
  compat: 5
  gameFixes:
    - BlitInternalFPSHack # Fixes internal FPS detection.
    - InstantDMAHack # Fixes missing letters in text such as E.
  gsHWFixes:
    recommendedBlendingLevel: 3 # Fixes skin colour and banding.
    autoFlush: 2 # Fixes lens flare.
    halfPixelOffset: 2 # Fixes blurriness.
    roundSprite: 2
    getSkipCount: "GSC_MetalGearSolid3" # Fixes depth of field blur.

old:

new:

[refractionpcsx2]

yeah unfortunately this isn't going to revolutionise round sprite, but I hope to reduce the amount of times it screws things up.

Thanks for testing!

[refractionpcsx2]

Actually can you provide a GS dump of the slicey Depth of Field effect? Maybe we can improve that too.

[cyanea-bt]

Sure thing, I'll attach 2 GS dumps from that cutscene. If you need any more or if I did them wrong, just let me know.

MGS3_roundSprite_slices_1.zip
MGS3_roundSprite_slices_2.zip

[refractionpcsx2]

Sure thing, I'll attach 2 GS dumps from that cutscene. If you need any more or if I did them wrong, just let me know.

That's perfect, thanks :)

[bigol83]

This PR also improves post processing being broken in Prince of Persia Sands of Time when using Round Sprite Full to improve post processing misalignment due to upscaling.

Master

PR

Look at the effect around the sun. It's not completely fixed but it's much better. Upscaling is at 3x.

[refractionpcsx2]

yeah that was actually one of my targets. lol

[bigol83]

It breaks Valkyrie Profile while using Round Sprite both Half and Full

Valkyrie Profile 2 - Silmeria_SLES-54647_20240317003508.gs.zip

Notice those horizontal dotted lines, that doesn't happen on Master.

I wonder though, what is Round Sprite used for in Valkyrie Profile? When it's off cutscenes look clean.

[refractionpcsx2]

It breaks Valkyrie Profile while using Round Sprite both Half and Full
I wonder though, what is Round Sprite used for in Valkyrie Profile? When it's off cutscenes look clean.

"Fixes area transition vertical lines and lessens red forest vertical lines." apparently.

[bigol83]

It breaks Valkyrie Profile while using Round Sprite both Half and Full
I wonder though, what is Round Sprite used for in Valkyrie Profile? When it's off cutscenes look clean.

"Fixes area transition vertical lines and lessens red forest vertical lines." apparently.

Yeah, i checked ingame and with Round Sprite OFF those lines on area transition are distracting. It's a shame because without it cutscenes look so beautiful.

[refractionpcsx2]

It seems to be the combination of Special Texture, Unscaled Palette and Half Round Sprite

If I change to align to native it looks better

however the forest squares look a little worse (not that they were okay before)

Also to note, if I do master with half round sprite and special texture but no unscale palette, that looks pretty bad too, so I suspect some coincidental reason it doesn't work properly

Master with no unscale palette

[bigol83]

With Align to Native and Round Sprite cutscenes look a lot better, vertical lines area transitions are not visible, but the top and left side have some thin lines (those could be cropped).

[refractionpcsx2]

That's always the problem with upscaling, isn't it :P

[bigol83]

That's always the problem with upscaling, isn't it :P

I also checked two other games that need Round Sprite, DQ VIII and Bully. Dragon Quest VIII menu box orange highlight has improved like i reported back in November, with no visible side effects. Bully doesn't show any difference compared to Master.

[refractionpcsx2]

I resolved the Valkyrie Profile issue for those not on the discord. We were running round sprite on native palette draws, which probably wasn't the best.