TOSECdev Forum

TOSEC Project => Database / Datfiles => Topic started by: LeBurglar on February 01, 2014, 11:51:40 AM

Title: State of the "Nintendo" DATs
Post by: LeBurglar on February 01, 2014, 11:51:40 AM
Hello there TOSEC,

first let me thank you for all you efford you put into collecting and organizing this old but somewhat beautiful stuff.

I recently noticed that the Nintendo DATs (at least the one for Super Nintendo/Super Famicom) seems not *that* complete. For example, there's no entry for "Super Mario Kart" which is a quite popular game. This game is included in No-Intro, so there's definately a known good dump.

Is there currently a maintainer for the Nintendo DATs? I'll make a list of all Nintendo ROMs which are included in No-Intro, but not in the current TOSEC release. I'm quite new to the TOSEC stuff, so I don't know how to catalog and rename properly or make a proper DAT file. But I could provide the ROM names and MD5 sums at least.

Best regards
Title: Re: State of the "Nintendo" DATs
Post by: PandMonium on February 02, 2014, 06:32:32 PM
Hi there,

Cassiel worked on some of the Nintendo stuff before but i think nobody is doing it now. You're more than welcome to help us improve those dats, renaming and creating the dats is not that hard.

The basics of renaming starts with checking an existing file and discovering what it is about (title, year, publisher are mandatory but the most info we can get the better). Then you rename the setfile [the zip] and normally also the romfile accordingly. From there you add it to the right folder (representing a datfile) with all the remaining sets and use a tool (for instance, clrmame pro) to generate a new datfile based on the content of such folder. That way, the new datfile will include old and new files with all the changes. Some renamers prefer to do this directly by editing the datfile in some text editor, after all the syntax is easy. The hardest part is many times to discover the details of a set (for instance modifications and dumping errors).

It's a bit tedious and hopefully will be improved in the future but some renamers like it this way since it can be a lot faster to just test and rename a file than play around with an extra tool of form. Also, one important point is to be careful because for some systems the emulators might alter the roms when played, creating a ton of different versions.
Title: Re: State of the "Nintendo" DATs
Post by: LeBurglar on February 02, 2014, 06:51:03 PM
Hi PandMonium,

I think I'll do it the manual way and edit the datfiles directly because I really don't like clrmamepro (seems not very usable for romsets with a nested structure like TOSEC has. Fine for "flat" romsets without a subdirectory structure like MAME and MESS.

The roms I plan to work with are completely unaltered, they completely match the latest No-Intro and TOSEC datfiles, so no altering due to emulators. What about rom headers for NES and FDS? I only have the rom *with* headers, is there any problem with that? I also have some concerns with some Japanese roms, as I don't really know if they are already included or not. (Not sure if No-Intro "(Japan)" version is the same that's already in TOSEC...)

If there's a MD5 mismatch between two roms (one in No-Intro and one in TOSEC) which share the ssame name, is that a valid reason to include the ROM in TOSEC? You know, *every* known rom for *every* known system ... ;)

I'll PM Cassiel, so maybe he can give me his latest WIP dats in case he is working on it right know. I also read in the forums that someone made about 600 entries for the Gameboy dats, so I'll check that too.

Best regards
Title: Re: State of the "Nintendo" DATs
Post by: PandMonium on February 02, 2014, 07:05:27 PM
Indeed you should PM Cassiel or just wait for him to show up and post here too.

I'm not sure if he has any WIP dats, probably you can start with the latest public ones. Using cmp just to create a datfile is not that hard really, have the roms all in the folder, point cmp dir2dat to it, fill the fields and it's done. Still you can do it that way too, editing the dats and adding game/rom entries manually (carefully :P).

As for NES headers (and others), i think Cassiel used only clean header files but wait for his answer (you can then clean them with ucon64).
If two roms exist with different hashes it means they are not the same so indeed we tend to catalog them both, figuring out what is the difference (is it an hacked version? a different version of the same software? different language? a bad dump or other kind of (un)intentional modification?).

Generally, the plan is to get a copy of those files [in the right format, right headers and other details Cassiel will share :P] in a folder and scan them with tosec dats, leaving only the unknown (cmp has an option to remove matched source files, romvault does this automatically (i think) if you put them in tosort folder. Then you start with those left, checking what they are and where they belong, renaming them accordingly to TOSEC and then: a) put them in the right folder and generating the dats in the end; or b) editing the right datfile and editing / adding the respective entry with correct size, md5, crc and sha1.

Generating the dats automatically might save you time and trouble with manually adding hashes and time by just dropping the new sets in the right folder. Still, real renamers might have better tips :D
Title: Re: State of the "Nintendo" DATs
Post by: Cassiel on February 03, 2014, 05:34:15 PM
Let's discuss after we've got the next release out the way...  next week-ish   :)
Title: Re: State of the "Nintendo" DATs
Post by: LeBurglar on February 03, 2014, 06:00:30 PM
Hi Cassiel,

good to hear that we can expect a new TOSEC release soon :)

I think it's a good idea to wait with this Nintendo stuff until the next update, so we don't have to get it done too fast.

Best regards

EDIT: "next week-ish" as in "next week" or as in "10 to 12 weeks"?  >:D