XML (JSON / YAML ...) format ?

[LynXX]

Hi,

Out of curiosity, why isn't the TOSEC in a more "standard" format like XML / JSON / YAML ? Is it just for legacy reasons ?
Would there be any interest in trying to switch to one of those formats ?

Pardon me if that's a settled issue, I tried to search the forums for a discussion about formats but couldn't find anything relevant.

Cheers,

Nico

[PandMonium]

Hi Nico and welcome aboard,

The way dats/dbs are released changed over time. Currently we use clrmame pro [old] datfile format, since TIM was deprecated. CMP changed its default datformat to XML a few years ago but he (and other projects) lagged behind. We have discussed the idea of using cmp XML dats or even extending that and afaik we kinda agreed that that is they way to go. The only reason stoping us doing so is problably some legacy software we use that is yet to be updated. Once we find time to do that XML might be a reality

[LynXX]

Hi, thanks for the answer !

Is the CMP XML format the one with the DTD here: http://www.logiqx.com/Dats/datafile.dtd ? It looks very similar to the MAME XML format.
What legacy software are you using that's lacking XML support ?

I wanted to write a tool that's using the TOSEC files but when I saw that it was a custom format and that I'd have to write a parser first I was a bit discouraged That's why I'm curious to see how a few steps in direction of switching to XML could be taken.

Cheers,

Nico

[PandMonium]

No problem!

Had to check and you're right, that's indeed the dtd referenced in CMP XML headers... and it is probably the same DTD used by MAME and romcenter (another rom manager). Also, apart from MAME most of those entities/tags/attributes are unused for most of dats since they are generated automatically from the filenames. This means most used ones are datfile, game (name, description) and rom (name, size, crc, md5, sha1) are the nodes used normally.

As for the legacy software, they're just some internal tools i developed over the years and haven't had the time to update. Mainly a few libs like a tosec naming convention (TNC) parser used to check a bit the datfiles and sets' names and a few other tools around that. They all have been developed using the old deprecated format but it shouldn't take a lot to change that. Thanks for asking that, i might look at it soon

[LynXX]

Hi,

I think the MAME DTD (Available with mame -listxml) is slightly different as the root tag is <mame> and not <datafile>, and each have a couple of different tags. It looks as if the datafile one was derived from an old version of the MAME one.

Would it make sense for a TOSEC-specific XML format (or an extension of the CMP one) to put the various information and flags into specific XML tags / attributes ? It would be good to store the metadata about each entry in a structured format, rather than encoding it within the filename I guess... Especially with non-ascii game names or names that contain characters that cannot be used in file names, or even titles using characters used for metadata like parenthesis and square brackets.

I'm willing to help, are the source of these tools available somewhere ? I'm happy to look at implementing XML support assuming that's in a language I'm familiar with. Also, some features might be replaceable with built-in XML features such as syntax validation, etc ?

Cheers,

Nico

[PandMonium]

You're probably right. I said that because i saw the clrmame pro MAME datfile available at ROMManager site. The output from -listxml is probably different and cmp converts that.

As for the XML bits, it off course it makes sense. It is something i called "TOSEC XML Extended Information File" (or something a bit lame like that) years ago but never ended the idea. It involves a lot of steps and i've only implemented part of them, furthermore the thing would need to be usable by renamers and they all have distinct ways of doing their work... my i guess that that is manageable too. The main problem is really my lack of time.
We already have means of picking the current datfiles and parsing all the information from there, would just need to output to the desired format. One of the main issues is the TNC complexity (and errors) but it would be a good opportunity to get rid of some too i guess.

Support in the tool i named is also partially done (i think i started doing it but since it was never used i've never ended/tested). Some validation is indeed possible by using DTD and XSD/XML Schemas. If you have interest and free time of course we can discuss what we have and our goals with further detail and you can contribute in some way (via pm?). Small changes that actually get done are a lot better than my older complex ideas that are stopped halfway.