Cassiel WIP (v2007-03-14-v2009-12-27)

[Cassiel]

Just a (very) quick msg to let you know I've just uploaded some new DATs to the FTP.

I'll keep adding to this thread, up until the new release (easier to follow):

Updated:
Acorn BBC - Educational - [SSD] (TOSEC-v2009-10-07_CM)
Acorn Electron - Games - [UEF] (TOSEC-v2009-10-07_CM)
Atari 5200 - Demos (TOSEC-v2009-10-07_CM)
Atari 5200 - Games (TOSEC-v2009-10-07_CM)
Bally Professional Arcade & Astrocade - Games (TOSEC-v2009-10-13_CM)
Bandai WonderSwan - Demos (TOSEC-v2009-10-13_CM)
Bandai WonderSwan - Games (TOSEC-v2009-10-13_CM)
Bandai WonderSwan Color - Demos (TOSEC-v2009-10-13_CM)
Bandai WonderSwan Color - Games (TOSEC-v2009-10-13_CM)
Creatronic Mega Duck & Cougar Boy - Games (TOSEC-v2009-10-06_CM)

New:
Bondwell Model 2 - Applications (TOSEC-v2009-10-06_CM)
Bondwell Model 2 - Operating Systems (TOSEC-v2009-10-06_CM)
Camputers Lynx - Applications (TOSEC-v2009-10-13_CM)
Camputers Lynx - Demos (TOSEC-v2009-10-13_CM)
Camputers Lynx - Games (TOSEC-v2009-10-13_CM)
Camputers Lynx - Magazines (TOSEC-v2009-10-13_CM)

Removed:
Bondwell Model 2 - Various
Camputers Lynx - Various


PandMonium - Please remove the two above, they have been split out....

[PandMonium]

Hi Cassiel, nice to see you keep working
I can check your new datfiles tomorrow or so for errors, about removing those from current-wip, idoru said he would like to be in charge of doing that to know what happens so i wont do it for now at least.

I have also other subject to talk with you (and possibly others) and so will do it know:
I guy from irc (jerry2600, don't know if any of you knows him) contact me and talked he had updated the "Nintendo Super Famicom & Super Entertainment System - Games" datfile, the first problem is that we are delaying a new release so much that any contributions from random people will almost always be done to old datfiles and probably became unuseful since those dats were already changed/updated and so on, he told me he already sent it to idoru but haven't contacted him.

AFAIK you already done something to this datfile but i don't know what, i really appreciate the help of everyone but imho the intention was good but the final product is not, from what i see this datfile comes in the style of the old TOSEC updates, growing them a lot with little care for correctness. The setfiles have doubled or something (+4000 now) and i have been told it was just a kind of merge between 2 dats:
TOSEC - Nintendo Super Famicom & Super Entertainment System - Games from 2007
Super Nintendo Entertainment System (No-InTrOSEC-v2007-04-01_CM)

The 2nd being done by NGEfreak, i've never seen this datfile but it has all sets from no intro not in tosec (like nongood stuff and so on), i think the intention was good in trying to help the project, but also according to our current goals (and i know that No Intro sets are almost always good stuff, no questions in that - they should be added) i don't think adding a thousand new sets without checking names or something is a good plan, anyway you are in charge of those dats and you decide, i can send you the dat for you to scan an gather all those files and eventually add them to the datfile later, imo the datfile (at least the one from 2007) is not that great in aspects like the 100++++ similar hacks without a proper moreinfo flag and only a counter making them different [h2], [h59], [h300], ...

As a final note i can't even check the dat for tnc errors because (i think) i haven't finished implementing support for new cmp xml datfiles and this datfile uses it :S

will upload it to your space anyway

[Cassiel]

No problem, as long as someone knows to do it…

Been really busy this/last week, but will try to reply to some posts/pm’s now whilst I have an hour.

[idoru] did send me the SNES DAT via email. I briefly reviewed and replied to [idoru], but need to look at it closer. My issue (and also spotted by [idoru]) is that there seems to be WAY too many 19xx’s for dates... it doesn't exactly instil confidence in publisher's being 100% right (I mean, why would you go the trouble of checking out the publisher on the title screen but NOT note the year as well...?).

Though if you saying they’ve just copied what is in No-Intro DAT, then this may seem explainable (do No-Intro use year? can’t rem off top of head) though hardly ideal for our purposes. Whilst the current SNES DAT is far from perfect (yet!), I don’t necessarily think dumping in No-Intro re-names without expanding for TOSEC is the right way to go.

Like I say, haven’t had enough time to investigate more though…..

[PandMonium]

i just saw it briefly, the first problem is it uses the new XML format (which we don't use yet, if you like we can start using it in the future or even fork a new one with more flags to describe a lot of set details one day), but for now we will keep the old one.

I asked how it was done and was told it was a merge (done with datworkshop or something) of the old snes dat + the nointrosec dat by NGEfreak, my concerns are exactly those, with the quality of added sets, i don't know NGEfreak or jerry2600 and i prefer to have new guys working undercontrol of others (like gorski did, with TKaos helping him out so he didn't made mistakes) instead of having now +2000 sets added. First this new datfile and your newer one will probably have differences in sets and i never used datworkshop, no idea of what it is capable of, 2nd these sets use the older country / language codes, also like you said they most certainly will have errors or few data simply because the initial idea was not to do it properly but to just add all no intro sets to it, so quality was not the first goal, that's why you probably found a lot of 19xx and other wrong information. NoIntro project is great and i'm sure they have some king of naming with at least publisher and year but don't know how it was converted (nointro -> tnc).

I guess you will have to investigate, adding the new dumps from them is good and they should be all good dumps i suppose but if we do that at least we should do it with quality or anyone can just blame us of inserting all those files without even renaming them properly

[TKaos]

Exactly, I don't like that they took other peoples work, done not much changes in filenames without even investigating the images more and then put the files in our DATs.
Doing it that way will just start alot of trouble between us and the No-Intro people and also with having more mistakes in our database.

[Cassiel]

Well, remember a huge number of dumps in the No-Intro DATs are already in the console DATs in TOSEC. I regularly pick up (full, current) No-Intro ‘ROM sets’ and do a rebuild to see what TOSEC is ‘missing’.

All the 8bit Sega DATs (SMS, GG, SC, SF7 etc) already include ALL images in No-Intro, MESS, GoodTool etc for example.

Any DATs that currently don’t, eventually will include all images from No-Intro or any other major DAT project.

Not to take anything away from the No-Intro boys (I think they’ve done lots of good work), but I don’t think we should just be ‘importing’ No-Intro renames into TOSEC wholesale, without them being checked individually (like you normally would when renaming images).

[TKaos]

Well yea that was also my point, check them the way you always do and not just include them with minimal name change.
btw: where's my PM ?

[Cassiel]

Hang on a minute, think I misread:

jerry2600 just took latest No-Intro DAT, and then our latest (public) TOSEC DAT and merged them (with a few cleanups here and there)?

jerry2600 not part of No-Intro then no?

If so then, no – really not liking the sound of any of this. Doesn’t sound very robust, and not very respectful to No-Intro project either.

TKaos - you posted as I was typing, but yes we both on same page exactly      PM coming, 2min....! (damn distractions)

[Cassiel]

Crap, ping you later..... running late, just seen time   

[PandMonium]

well i think we all agree in that, they did and do an excellent work and that new dats should be added carefully (if manually checked it would be a good time to even take some screenshots of publishers or groups ) instead of converting them, for example i could just code something to parse non intro dats and rename them to tnc adding extra info to more info flag but i think this is NOT the way we should do it in our quest for correctness
(we have the fine example of this kind of checks in Tandy and how bad they got )

I don't know jerry2600 (just know we hangs around foreverchat irc network) nor NGEfreak, he just told me the new dat was a MERGE between our latest official SNES dat + one dat called NoInTrOSEC or something that was something like: NoIntro datfile - TOSEC datfile = NoInTOSEC (a kind of diff).
Just that, thats why i think it is bad, he told me he just did the merge and the dif dat or something was done by NGEfreak, don't know how made the changes to TNC or if there are some without changes because i can't really check manually or automatically (new xml format).

I will have to give some feedback to jerry2600 because i appreciate his interest on helping the project but have to say him that this is not something we need, will probably say that Cassiel is already working on these dats and is adding new sets manually or something, normally he will take it good but we never know, there is always strange people around emu scene (he could start thinking we don't work/have a dead project and don't accept help anyway ).
Will see, he seemed a nice guy anyway

[idoru]

well, saves me talking to him   i gave him some comments on it then he never said anything about it to me again after so how he can say i didn't contact him i don't know.  from being in an irc channel with him for years, i very much doubt jerry will be a regular contributor - i'd definitely see this as a one-off, especially if that's how the dat came to being

[idoru]

cassiel,

can you fix Camputers Lynx - Demos :-

game (
   name "Pattern Box Demo (1984)(Camputers)"
   description "Pattern Box Demo (1984)(Camputers)"
   rom ( name "Pattern Box Demo (#2 Page 7 (19xx)(-)"
   description "R. Penrose Lynx User #2 Page 7 (19xx)(-)"
   rom ( name "R. Penrose Lynx User #2 Page 7 (19xx)(-).tap" size 646 crc a1db7e86 md5 d98e73208a77bd3a2a8477a2c30fdb68 )
)

I'm presuming should be :-

game (
   name "Pattern Box Demo (1984)(Camputers)"
   description "Pattern Box Demo (1984)(Camputers)"
   rom ( name "Pattern Box Demo (1984)(Camputers).tap" size 1222 crc 0f8bde7e md5 0b9970bbecfbc887a4a2de79726dd965 )
)

As R. Penrose... is in the Magazines dat, but the dat also includes files that are in other split dats & the dat file itself also cuts off before the end at :-

game (
   name "Star Rover Lynx User #1 (19xx)(-)"
   description "Star Rover Lynx User #1 (19xx)(-)"
   rom (

<end of file>

ta

[Cassiel]

WFT has happened there...?!

Will check and reupload......

[idoru]

Thanks, I left the set of Camputers Lynx files in your renamers dir, lemme know when re-uploaded & I'll move across to main dats stem.

[Cassiel]

Forgot to say, re XML.... think this an excellent idea for the physical DATs. Not even CMP uses the old DAT format any more, and you never know when Roman might pull the option from CMP's DIR2DAT all together, which would be a mild pain the ass for us.

Not a huge deal, but forward looking... and forward looking always good (even for a project obsessed with the past     )

Obviously a question (and decision) for [idoru] though…..