Out of memory error with isos that have invalid TOC entries

[LittleStinky123]

With some games there are invalid entries even in the redump ISO.
When I use xbox backup creator to get the main partition of the PAL game Burnout for example and try to extract it with extract-xiso, it hangs at the directory trafcars/models/minibus and then crashes after some time with an out of memory error.
This directory is probably the last entry as the size of the extracted files exactly matches the size when extracting the disc with xbox back creator directly from the disc.
With xdvdmulleter with the redump as well as the xbox backup creator images I get an invalid toc entry error and can't even try to extract the file.
Is there some way to ignore this error or fix it?

[JayFoxRox]

I believe I've never seen this. Can you explain this better?

---

Are you using our fork of extract-xiso? There are other versions (outdated or specialized) of this tool maintained by other people.

With some games there are invalid entries even in the redump ISO.
[...]
With xdvdmulleter with the redump as well as the xbox backup creator images I get an invalid toc entry error and can't even try to extract the file.

Are we talking about confirmed images (at least 2 independent reference dumps which match)?

• How did you dump the disc / How did you confirm your dump?
• Why would the TOC be invalid / Could it be some form of copy protection?

When I use xbox backup creator to get the main partition of the iso of Burnout for example [...]

There've been 4 different games in the Burnout series on Xbox; and there's at least 12 different discs on Redump (only 2 of which have been confirmed).

• Which Burnout?
• Which disc / release?
• What hashes do you get?

[...] and try to extract it with this tool, it hangs at the directory trafcars/models/minibus and then crashes after some time with an out of memory error.

• How much RAM do you have?
• How much RAM does extract-xiso consume?
• Where does it crash?

Is there some way to ignore this error or fix it as it crashes in beginning of the extraction and therefore only a small part of the files are extracted.

We'd need more information for a fix.

extract-xiso probably shouldn't crash, but it is possible that it hangs for a long time (if the input is bad, but could be valid). We could potentially add some sanity checks if need-be.

[LittleStinky123]

How did you dump the disc / How did you confirm your dump?

I dumped it via a kreon drive and made a redump dump with DICUI where the md5 matches with the one from redump.org and additionally made a dump of the main partition with xbox backup creator where the dump of the main partition matches from 0x00010000 to 0x0CFF99C78 the redump dump from 0x018310000 to 0x0E8299C78 so I think the xbc dump I used with extract-xiso was dumped without errors.

Which Burnout?
Which disc / release?
What hashes do you get?

I used the first PAL Burnout and got all the same hashes that are on here: http://redump.org/disc/14469/

How much RAM do you have?
How much RAM does extract-xiso consume?
Where does it crash?

I have 16 GB of RAM, while extracting it constantly takes 25,4 MB until it reaches trafcars\models\minibus\ then the RAM usage constantly rises until the PC is out of memory and then it prints failed to extract xbox iso image burnout.iso.
It also doesn't crash near the beginning, only ~4 MB of data are missing after the failed extraction.

I am really sorry to waste your time with this but as the xbox emulators improve I think people will try to use their own original discs with them.
And if you dump it yourself and don't pirate them you will have to extract the redump image as cxbx needs the extracted files and xqemu needs a rebuilt image from the extracted files as it can't handle redump isos for now.
The first suggested programs I found were extract-xiso and xdvdmulleter and only after trying out a lot of programs I found xbox backup creator which finally worked.
So if extract-xiso could also handle this use case others maybe won't have to search for so long anymore.

[JayFoxRox]

I dumped it via a kreon drive and made a redump dump with DICUI where the md5 matches with the one from redump.org
[...]
I used the first PAL Burnout and got all the same hashes that are on here: http://redump.org/disc/14469/

I did not find time to submit to redump yet, but I can confirm that same hash for my German disc of Burnout 1. So your disc / dumping method is fine.

dump of the main partition matches from 0x00010000 to 0x0CFF99C78 the redump dump from 0x018310000 to 0x0E8299C78

Independent of the rest, this sounds suspicious: The data partition should be 7419363328 bytes in size. Your's is only half that size.

I assume that xbc already compressed the ISO (removing padding sectors).

I have 16 GB of RAM, while extracting it constantly takes 25,4 MB until it reaches trafcars\models\minibus\ then the RAM usage constantly rises until the PC is out of memory and then it prints failed to extract xbox iso image burnout.iso.

I was able to reproduce the issue.

There's many independent issues in extract-xiso which cause this:

1. On 64 bit, the size of some datatypes is not the expected width.
2. On big endian, some fields are not endianess swapped.
3. The traverse_xiso function uses the l_offset (named tmp in code) and r_offset to traverse the file-tree. This is bad. These fields are meant for faster file-search for specific names, but not for scanning over all files.

Point 3 is obviously the most critical one, as it means that extract-xiso is fundamentally broken. There's different exit conditions for XISO:

• For array-like usage: if cursor offset directory->file_size is reached. if l_offset and r_offset are both -1, then the current sector is skipped, but the directory continues.
• For tree-like usage: if cursor offset is 0, the file doesn't exist; it's an error if cursor offset is greater or equal to the directory->file_size. it's an error if the cursor moves backwards (except offset 0).

I've prototyped the array approach in JayFoxRox#1 - now that I understand it better, I'll look into a solution for the tree-like search (however, I'll not add the error conditions, because the code is fundamentally broken / bad; See #12 ).

I am really sorry to waste your time with this but as the xbox emulators improve I think people will try to use their own original discs with them.

It's not a waste of time. Thanks for reporting this bug!

XboxDev was started by XQEMU developers, with the intention to work together with Cxbx-R people and other stakeholders.
That's also why we took over some of these 15 year old projects: to keep them alive and provide users with good tools.

And if you dump it yourself and don't pirate them you will have to extract the redump image as cxbx needs the extracted files and xqemu needs a rebuilt image from the extracted files as it can't handle redump isos for now.

Once you extract the data partition you should just be able to use it in XQEMU.

Example:

dd if=redump.iso of=xqemu_compatible.iso skip=198144 bs=2048

• Repacking is only recommended for storage size and temporary bugs in XQEMU.
• Extraction is only necesary for Cxbx-R.

In the near future, XQEMU should be able to support redump ISOs exclusively (and will also require SS.bin, DMI.bin and PFI.bin).

Cxbx-R will probably still require extraction, but I assume they'll promote some fuse-filesystem to mount Xbox Redump ISOs, too.

The first suggested programs I found were extract-xiso and xdvdmulleter and only after trying out a lot of programs I found xbox backup creator which finally worked.

Yes, it's annoying.

We'd rather want to avoid tools which are primarily used by the piracy scene. We also want open-source and cross-platform tools.

XQEMU specifically recommends this version of extract-xiso on its website.

So if extract-xiso could also handle this use case others maybe won't have to search for so long anymore.

Agreed.

---

Issue remains open until we have a fix in master.