Add bdm hdd support #1365

KrahJohlito · 2024-10-25T07:07:04Z

Pull Request checklist

Note: these are not necessarily requirements

I reformatted the code with clang-format
I checked to make sure my submission worked
I am the author of submission or have permission from the original author
Requires update of the PS2SDK or other dependencies
Others (please specify below)

Pull Request description

This adds bdm hdd support adapted from @grimdoomer fork. Only additions are if one hdd option is ON the other will be disabled to try avoid conflicts.

BatRastard · 2024-10-25T10:11:12Z

Got a test binary?

KrahJohlito · 2024-10-25T12:30:02Z

Got a test binary?

CI builds pending prs if you ever want to try them before merge

https://github.com/ps2homebrew/Open-PS2-Loader/actions/runs/11513539409

src/bdmsupport.c

AKuHAK · 2024-11-03T09:36:18Z

src/bdmsupport.c

+        else {
+            dmaType = 0x40;
+#ifdef ATA_UDMA_PLUS
+            dmaMode = pDeviceData->ataHighestUDMAMode;


Chinese IDE-to-SD adapters are faking this value in adapter specs. If the user still can set UDMA mode manually it is ok, but I am not sure if settings for "classic" HDD (HD Loader) will be taken in place in this case. Also some adapters sets this value to zero, while actually supports UDMA4.

I'm not sure what to do here about that? Any suggestions?

Huh, I am not sure too, maybe read udma mode from HDD apa settings somehow?

but that's what it's doing already? on device connection it calls the cmd to retrieve the highest udma mode here

Open-PS2-Loader/src/bdmsupport.c

Line 765 in 723dedd

pDeviceData->ataHighestUDMAMode = fileXioDevctl("xhdd0:", ATA_DEVCTL_GET_HIGHEST_UDMA_MODE, NULL, 0, NULL, 0);

Open-PS2-Loader/modules/hdd/xhdd/xhdd.c

Line 58 in 723dedd

case ATA_DEVCTL_GET_HIGHEST_UDMA_MODE: {

and store the value in pDeviceData->ataHighestUDMAMode then at launch its just applying it in this line we're quoting.

Or I'm just misunderstanding your latest comment and you mean read a manually defined udma user setting?

Huh, I am not sure too, maybe read udma mode from HDD apa settings somehow?

Oh wait apa settings.. I didn’t read this properly.. what about this? If ataHighestUDMAMode > 0 use it, otherwise use the user defined setting from the cfg?

Yes, I was talking about user defined value that are per game.

Yes, I was talking about user defined value that are per game.

ok now it works very similar to apa hdd, if the automatically resolved udma is <= 0 it will use the user defined value.. we could have it always use the user defined value but then the whole ataHighestUDMAMode is pointless and unused..

so now we check ataHighestUDMAMode is greater than 0 if its not due to a faked value we use the user defined value

src/hddsupport.c

CosmicScale · 2024-11-03T13:10:34Z

I've been testing the CI build and I've found a couple of bugs.

OPL settings are always saved to the HDD's exFAT partition no matter what. On launch OPL does not read the settings config from the HDD's exFAT partition.
The 'autoLaunchBDMGame' function is now completely broken again.

I tried to compile this PR with the latest build and bdm hdd support doesn't work at all.

I did a little debugging on KrahJohlito's gd2 build trying to find the issue with 'autoLaunchBDMGame'. It gets as far as reading the game's config file from the exFAT partition on the HDD but then just crashed when trying to launch the game.

KrahJohlito · 2024-11-03T21:58:57Z

I've been testing the CI build and I've found a couple of bugs.

OPL settings are always saved to the HDD's exFAT partition no matter what. On launch OPL does not read the settings config from the HDD's exFAT partition.

The 'autoLaunchBDMGame' function is now completely broken again.

I tried to compile this PR with the latest build and bdm hdd support doesn't work at all.

I did a little debugging on KrahJohlito's gd2 build trying to find the issue with 'autoLaunchBDMGame'. It gets as far as reading the game's config file from the exFAT partition on the HDD but then just crashed when trying to launch the game.

2 is likely not “completely broken” but broken for hdd as I forgot to add the loading of bdm hdd modules for auto launching, that’s probably all it is

Edit: ok now I'm actually on pc, no its more than that haha i'll try push an update if you can give it a quick test

CosmicScale · 2024-11-04T00:32:16Z

Edit: ok now I'm actually on pc, no its more than that haha i'll try push an update if you can give it a quick test

I can test it whenever you're ready :)

KrahJohlito · 2024-11-04T01:19:54Z

Edit: ok now I'm actually on pc, no its more than that haha i'll try push an update if you can give it a quick test

I can test it whenever you're ready :)

I think both issues 1 & 2 should work properly now.. tbh the whole cfg saving/loading system needs a rewrite (probably a more static approach like maximus32 has suggested multiple times is the best way to go) but I personally don't have time to it at the moment.

I'll rebase if those problems are solved.

CosmicScale · 2024-11-04T17:47:23Z

Edit: ok now I'm actually on pc, no its more than that haha i'll try push an update if you can give it a quick test

I can test it whenever you're ready :)

I think both issues 1 & 2 should work properly now.. tbh the whole cfg saving/loading system needs a rewrite (probably a more static approach like maximus32 has suggested multiple times is the best way to go) but I personally don't have time to it at the moment.

I'll rebase if those problems are solved.

OPL settings now save to memory card and are loaded at launch however, OPL always starts up on the menu instead of the games list.

BDM Auto Launch didn't working on HDD or USB but I did some debugging. opl.c line 1846, change the delay to 10 and it works!

KrahJohlito · 2024-11-04T20:58:47Z

Edit: ok now I'm actually on pc, no its more than that haha i'll try push an update if you can give it a quick test

I can test it whenever you're ready :)

I think both issues 1 & 2 should work properly now.. tbh the whole cfg saving/loading system needs a rewrite (probably a more static approach like maximus32 has suggested multiple times is the best way to go) but I personally don't have time to it at the moment.
I'll rebase if those problems are solved.

OPL settings now save to memory card and are loaded at launch however, OPL always starts up on the menu instead of the games list.

BDM Auto Launch didn't working on HDD or USB but I did some debugging. opl.c line 1846, change the delay to 10 and it works!

I’m going to drop the cfg changes commit, the whole system is the problem and needs a rewrite.. the cfg entry to know whether or not to load the module is in the cfg.. but how can we read it from the cfg if the module isn’t loaded :-s .. chicken and the egg situation.

ok so we gotta increase the delay now thanks for trying it, I’ll try make it more dynamic so it’s not just a set x delay, we’ll just delay as long as we need to until the device is found that way it’s not longer than it needs to be.

KrahJohlito · 2024-11-05T23:47:41Z

@CosmicScale I've just set a delay loop if you're free to test the auto launching again?

The cfg issues will need to be addressed in a separate pull request imo as a rewrite to the system will require some big changes.

CosmicScale · 2024-11-06T12:12:45Z

@CosmicScale I've just set a delay loop if you're free to test the auto launching again?

The new loop with delay(2) fixed loading from HDD but USB was still broken. I experimented with delay lengths, the shortest delay that reliably worked with both was 6.

KrahJohlito · 2024-11-06T22:41:03Z

force pushed

Mooninites11 · 2024-11-07T23:15:30Z

I could not get this to run i just get a black screen

CosmicScale · 2024-11-08T01:25:15Z

I could not get this to run i just get a black screen

Working perfectly for me. Could be the way you formatted your drive. I assume, just like Grimdoomer's build, the HDD must have a sector size of 512. I remember people have issues when using certain USB enclosures. It's best to connect the HDD directly to your PC then format it.

Mooninites11 · 2024-11-08T03:25:44Z

I could not get this to run i just get a black screen

Working perfectly for me. Could be the way you formatted your drive. I assume, just like Grimdoomer's build, the HDD must have a sector size of 512. I remember people have issues when using certain USB enclosures. It's best to connect the HDD directly to your PC then format it.

i was talking about opl not even launching its in a boot loop and if unplug the hard drive opl works i just reformatted me drive and made sure it was 512 and its still doing it

CosmicScale · 2024-11-08T10:42:53Z

i was talking about opl not even launching its in a boot loop and if unplug the hard drive opl works i just reformatted me drive and made sure it was 512 and its still doing it

Try deleting your OPL config files.

KrahJohlito · 2024-11-08T23:21:02Z

I could not get this to run i just get a black screen

likely due to this #1380 which imo is from a ps2sdk update, if you tried the latest version of this build; I rebased which used a newer sdk to compile.. try this one before that sdk update and see if the issue persists https://github.com/ps2homebrew/Open-PS2-Loader/actions/runs/11694352998 .. after the recent sdk update I get random spam messages in pcsx2 logs Unknown Device 'host' .. something seems to be causing issues

Mooninites11 · 2024-11-09T00:59:34Z

I could not get this to run i just get a black screen

likely due to this #1380 which imo is from a ps2sdk update, if you tried the latest version of this build; I rebased which used a newer sdk to compile.. try this one before that sdk update and see if the issue persists https://github.com/ps2homebrew/Open-PS2-Loader/actions/runs/11694352998 .. after the recent sdk update I get random spam messages in pcsx2 logs Unknown Device 'host' .. something seems to be causing issues

Yep its working now

mystyq · 2024-11-10T17:17:51Z

Happy to help test this. 8TB HDD exFAT. Grimdoomer's build works.

The current PR build switches to GPT and then is quite laggy. Loader icon just keeps spinning and inputs are don't register unless on a specific frame

362053534 · 2024-11-11T09:49:11Z

I could not get this to run i just get a black screen

likely due to this #1380 which imo is from a ps2sdk update, if you tried the latest version of this build; I rebased which used a newer sdk to compile.. try this one before that sdk update and see if the issue persists https://github.com/ps2homebrew/Open-PS2-Loader/actions/runs/11694352998 .. after the recent sdk update I get random spam messages in pcsx2 logs Unknown Device 'host' .. something seems to be causing issues

Hello! I tested this build too. And I found that loading the HDD game list is too slow,but Grimdoomer's build works good.
I don't know why ,I guess maybe hdd or bdm cache doesn't work.

KrahJohlito · 2024-11-13T10:30:40Z

I could not get this to run i just get a black screen

likely due to this #1380 which imo is from a ps2sdk update, if you tried the latest version of this build; I rebased which used a newer sdk to compile.. try this one before that sdk update and see if the issue persists https://github.com/ps2homebrew/Open-PS2-Loader/actions/runs/11694352998 .. after the recent sdk update I get random spam messages in pcsx2 logs Unknown Device 'host' .. something seems to be causing issues

Hello! I tested this build too. And I found that loading the HDD game list is too slow,but Grimdoomer's build works good. I don't know why ,I guess maybe hdd or bdm cache doesn't work.

Happy to help test this. 8TB HDD exFAT. Grimdoomer's build works.

The current PR build switches to GPT and then is quite laggy. Loader icon just keeps spinning and inputs are don't register unless on a specific frame

Hi guys, unfortunately these differences appear to be the result of a newer build environment because the code here is almost exactly the same as grimdoomer builds (can’t really do anything about that in this pr, may need to be addressed in the sdk idk).

https://github.com/KrahJohlito/Open-PS2-Loader/actions/runs/11811553194

This version that I’m linking above is the same as this PR build except the build environment used is very similar to OPL revision 2049 rather than “latest” and seems to have much better performance; conclusion this PR is not causing the issues but there can still be improvements made whether it be in the build environment or OPL itself. I do not intended to try rectify any further issues in this PR (with the exception of current pending reviews) as I don’t believe the problem to be here.

Best Regards

RaptorZX4 · 2024-12-07T14:47:32Z

So does that mean the exFAT support work better now?

I remember trying out KrahJohlito's build with it, and it had some problems with some games., but that was a while ago. Maybe it got improved by then before being integrated into the main build?

mystyq · 2024-12-07T15:01:06Z

@RaptorZX4

It does work. It just takes VERY long (in my instance I have a 8TB drive with a full romset) and is arguably not very reliable (there can be improvements made). If I'm understanding correctly @KrahJohlito has made a point that he believes the issue is related to the build environment vs. this specific pull request.

If it is true that there is a difference in performance between build environments, the newest build environment could use improvements

If you are looking for the exFAT support I recommend sticking with Grimdoomer's fork until someone can look into this deeper

RaptorZX4 · 2024-12-07T15:51:52Z

If you are looking for the exFAT support I recommend sticking with Grimdoomer's fork until someone can look into this deeper

yeah this is what i figured a while ago. Though i don't use a massive HDD storage like you do, i just use a SSD of less than 1Tb.

KrahJohlito · 2024-12-07T23:09:19Z

@RaptorZX4

It does work. It just takes VERY long (in my instance I have a 8TB drive with a full romset) and is arguably not very reliable (there can be improvements made). If I'm understanding correctly @KrahJohlito has made a point that he believes the issue is related to the build environment vs. this specific pull request.

If it is true that there is a difference in performance between build environments, the newest build environment could use improvements

If you are looking for the exFAT support I recommend sticking with Grimdoomer's fork until someone can look into this deeper

I will rebase when I get a chance to see if the recent changes to iomanX in sdk has improved anything, the reason I think its environment related is just because the code is exactly the same as grimdoomers so that’s the only change.. seems like cache might have broken in newer builds but there’s no reason for this other than io changes which would come again from the build environment :) anyway I’ll let you know when I rebase if you wanted to give it a quick test again

Co-authored-by: grimdoomer <[email protected]>

KrahJohlito · 2024-12-15T23:38:41Z

rebased

https://github.com/ps2homebrew/Open-PS2-Loader/actions/runs/12343233139

@mystyq @RaptorZX4 idk if you guys wanna test it again see if the sdk updates have improved anything

mystyq · 2024-12-16T00:56:28Z

Deleted opl_conf.cfg -> enabled caching -> bdm mode manaul -> sub setting enabled HDD -> save config -> game list -> start BDM and then after a while as the cache is building the application hangs and becomes unresponsive. (Loading icon not spinning, no input accepted, no audible hard drive activity)

Edit: went through this process twice, and the same results

With ~1500 titles the caching takes ~5 minutes to create the games cache with the build environment used in GD's binary

The latest rebase with the build environment you used it took over 9 minutes and then hung up.

KrahJohlito · 2024-12-16T03:32:36Z

Deleted opl_conf.cfg -> enabled caching -> bdm mode manaul -> sub setting enabled HDD -> save config -> game list -> start BDM and then after a while as the cache is building the application hangs and becomes unresponsive. (Loading icon not spinning, no input accepted, no audible hard drive activity)

Edit: went through this process twice, and the same results

With ~1500 titles the caching takes ~5 minutes to create the games cache with the build environment used in GD's binary

The latest rebase with the build environment you used it took over 9 minutes and then hung up.

lol so rebase made it even worse.. are you on discord?

CosmicScale · 2024-12-16T11:24:02Z

Just a heads up, one of the older builds I was testing with HDD BDM support is now being used a part of my PSBBN-Definitive-English-Patch. It's working great for that use case.

mystyq · 2024-12-16T21:17:59Z

@KrahJohlito Yes I am on discord, is there a server to join? Otherwise what is your discord name, I can help troubleshoot

KrahJohlito · 2024-12-17T02:13:38Z

Just a heads up, one of the older builds I was testing with HDD BDM support is now being used a part of my PSBBN-Definitive-English-Patch. It's working great for that use case.

well if its working well then I'm glad its getting some use..

@KrahJohlito Yes I am on discord, is there a server to join? Otherwise what is your discord name, I can help troubleshoot

theres a discord badge on the readme of this repo click it for an invite link, my name is same as here

AKuHAK requested changes Nov 3, 2024

View reviewed changes

KrahJohlito force-pushed the gd2 branch from c3e7ffa to 00bf63e Compare November 5, 2024 23:16

KrahJohlito force-pushed the gd2 branch from 7c8f7a7 to 723dedd Compare November 6, 2024 22:38

KrahJohlito requested a review from AKuHAK November 6, 2024 22:41

KrahJohlito force-pushed the gd2 branch 3 times, most recently from cd1e675 to c300a02 Compare November 10, 2024 23:42

KrahJohlito force-pushed the gd2 branch 2 times, most recently from 4e1f198 to 4e917c8 Compare November 18, 2024 02:56

KrahJohlito force-pushed the gd2 branch from 4e917c8 to 4b232dc Compare November 29, 2024 05:18

AmyRoxwell mentioned this pull request Dec 5, 2024

[FR] Implement Grimdoomer's ExFat support to help UX users #1043

Closed

1 task

KrahJohlito and others added 4 commits December 16, 2024 10:04

Add support for BDM HDD from grimdoomers fork

626e5e4

Co-authored-by: grimdoomer <[email protected]>

settings: only allow one hdd option to be on at a time

40656e6

lang: make bdm options more understandable

9e788df

u64 datatype for vmc

b05257c

KrahJohlito force-pushed the gd2 branch from 4b232dc to b05257c Compare December 15, 2024 23:34

AKuHAK merged commit a626d98 into ps2homebrew:master Dec 27, 2024
17 checks passed

Add bdm hdd support #1365

Add bdm hdd support #1365

Conversation

KrahJohlito commented Oct 25, 2024

Pull Request checklist

Pull Request description

BatRastard commented Oct 25, 2024

KrahJohlito commented Oct 25, 2024

AKuHAK Nov 3, 2024

Choose a reason for hiding this comment

KrahJohlito Nov 5, 2024

Choose a reason for hiding this comment

AKuHAK Nov 8, 2024

Choose a reason for hiding this comment

KrahJohlito Nov 8, 2024

Choose a reason for hiding this comment

KrahJohlito Nov 10, 2024

Choose a reason for hiding this comment

AKuHAK Nov 10, 2024

Choose a reason for hiding this comment

KrahJohlito Nov 10, 2024

Choose a reason for hiding this comment

CosmicScale commented Nov 3, 2024 • edited Loading

KrahJohlito commented Nov 3, 2024 • edited Loading

CosmicScale commented Nov 4, 2024

KrahJohlito commented Nov 4, 2024 • edited Loading

CosmicScale commented Nov 4, 2024

KrahJohlito commented Nov 4, 2024

KrahJohlito commented Nov 5, 2024

CosmicScale commented Nov 6, 2024

KrahJohlito commented Nov 6, 2024

Mooninites11 commented Nov 7, 2024

CosmicScale commented Nov 8, 2024

Mooninites11 commented Nov 8, 2024 • edited Loading

CosmicScale commented Nov 8, 2024

KrahJohlito commented Nov 8, 2024 • edited Loading

Mooninites11 commented Nov 9, 2024

mystyq commented Nov 10, 2024

362053534 commented Nov 11, 2024 • edited Loading

KrahJohlito commented Nov 13, 2024 • edited Loading

RaptorZX4 commented Dec 7, 2024

mystyq commented Dec 7, 2024

RaptorZX4 commented Dec 7, 2024

KrahJohlito commented Dec 7, 2024

KrahJohlito commented Dec 15, 2024

mystyq commented Dec 16, 2024 • edited Loading

KrahJohlito commented Dec 16, 2024

CosmicScale commented Dec 16, 2024

mystyq commented Dec 16, 2024

KrahJohlito commented Dec 17, 2024

CosmicScale commented Nov 3, 2024 •

edited

Loading

KrahJohlito commented Nov 3, 2024 •

edited

Loading

KrahJohlito commented Nov 4, 2024 •

edited

Loading

Mooninites11 commented Nov 8, 2024 •

edited

Loading

KrahJohlito commented Nov 8, 2024 •

edited

Loading

362053534 commented Nov 11, 2024 •

edited

Loading

KrahJohlito commented Nov 13, 2024 •

edited

Loading

mystyq commented Dec 16, 2024 •

edited

Loading