Author Topic: A tool for Dedupe/diff/xdelta... anything for redundancy in ROMS?  (Read 511 times)

Offline tolie

  • Newbie
  • *
  • Posts: 4
It's no secret there is identical data in multi-regional titles, but is there a tool that can split the identical data from the duplicate data in say a .ISO or anything that can be unpacked for ROMs?

Years back Microsoft had this ISO tool that people used to create "multiOS" images (home, pro, enterprise, etc...) that basically just symlinked/relinked the TOC of a image to allow duplicate data to remain.... unduplicated (:-/). Anyhow, I'm wondering if there is a tool that does something similar as having multiple copies of the same ROM with basically just a difference of language files is kind of bloated.



Offline Maddog

  • Global Moderator
  • Full Member
  • *****
  • Posts: 182
Re: A tool for Dedupe/diff/xdelta... anything for redundancy in ROMS?
« Reply #1 on: December 29, 2019, 01:12:32 PM »
None that I know of, except NKit ISO and Nkit GCZ (https://wiki.gbatemp.net/wiki/NKit) which can shrink significantly the GameCube and Wii ISOs, while maintaining usability. TOSEC doesn't dat Wii at all for the time being however.

In theory it could be possible to create patches that would convert a release to the similar ones from other regions, but this involves too much manual work and I am not aware of any efforts towards this, especially since nowadays disk space is very cheap to buy.

Offline tolie

  • Newbie
  • *
  • Posts: 4
Re: A tool for Dedupe/diff/xdelta... anything for redundancy in ROMS?
« Reply #2 on: December 30, 2019, 02:55:12 AM »
Thanks, I might take an initiative towards this (or start and give up half way). I'm not a "ROM" person, I'm just curious about how they are organized. I guess you could say I'm more interested in the meta collection and not the actual playing of ROMS.

I'm surprised to see that at first glance a lot of these ROM managers are very confusing. "Offlinelist" seems O.K., point at a .dat, point at a directory... done. But Romcenter, which seems widely popular, I can't understand. I got a redump SEGA Dreamcast .dat to "load" (I guess it loads) in Romcenter, but they all show red (even though the crc's in the .zip's are correct).

But the space,  it kind of erks me to see so much redundancy. I'm wondering just how much space is just being flat out wasted. Although you're right, space is cheap, so maybe I just kind of want to know how much is wasted more than actually saving that space.

But off that, thanks to TOSEC for the Dreamcast collection. Until very recently (very) it was my favorite console, and there's still something about Jet Set Radio that was never repeated.

Offline Maddog

  • Global Moderator
  • Full Member
  • *****
  • Posts: 182
Re: A tool for Dedupe/diff/xdelta... anything for redundancy in ROMS?
« Reply #3 on: December 30, 2019, 10:25:57 AM »
I generally use ClrMAME Pro and Romvault for ROM management. Specifically for TOSEC, Romvault is by far the easiest to work with. I am not familiar with Romcenter.

Thanks for the kind words, I have spent more than 12 years building DC dats, so I am glad to hear they are appreciated. :)

Offline Kodoichi

  • Full Member
  • ***
  • Posts: 143
  • ❤️ 💾 🕹
Re: A tool for Dedupe/diff/xdelta... anything for redundancy in ROMS?
« Reply #4 on: January 01, 2020, 01:22:58 PM »
I got a redump SEGA Dreamcast .dat to "load" (I guess it loads) in Romcenter, but they all show red (even though the crc's in the .zip's are correct).
DAT files have to be converted to RDT files, but RomCenter should do that automatically if you add/load it for the first time.

Offline Cassiel

  • Administrator
  • Hero Member
  • *****
  • Posts: 1471
    • Email
Re: A tool for Dedupe/diff/xdelta... anything for redundancy in ROMS?
« Reply #5 on: February 28, 2020, 11:48:47 AM »
Thanks for the kind words, I have spent more than 12 years building DC dats, so I am glad to hear they are appreciated. :)

Always...