Hi,
The development for TiM in progress?
I find a old version 0.6.14 beta2 on net. This is not usable?

 

PandMonium fixed alot bugs in TIM like 1,5 years ago and released a beta version but he said that the sourcecode was too broken to make a good version.
The beta was fine because it was alot faster with loading the DATs but there were still some bugs in it, he then stopped working on it and said you'd need to re-do the whole code to make something usefull but fixing more in TIM's old code would be wasted time because of the broken source.

Hi Guillaume,
Where did you find that? :o
TIM development stopped long ago, it was bad planned (or not quite planned at all :P) and had a lot of things that didn't work properly, so it wasn't never reliable and wasn't being used for long, even delaying releases (thats why the last official one was not applied in tim.db. I once started to fix it but it was plain broken and would require an immense amount of time to fix but even then there were parts that needed to be redone so i just stopped.
It had some nice parts / ideas but others weren't that good and there were/are already excellent tools to do so (clrmame to rom auditing), maybe we will have some more tools in the future but nothing like TIM (afaik), it was too big and with too much unnecessary features and should had been split in smaller tools :P
AFAIK Xang (an user here) posted a tool he is using to maintain his sets, not sure how it works and if it does work correctly but at least it is something similar to TIM/tugid :)

ok, thank you for your answers

Hello PandMonium,

...perhaps you remember me throwing weekly bug reports at you... :)

...as TIM is now officially discontinued, is there any chance to release the source & db schema for studies? Just to enable people who want to create new applications to not make the same faults again? I still like the basic concept of TIM and I think it could have been auccessful if it was more streamlined...

Anway: any chance for a (final) TIM source package?

Kind Regards,
derJörg

Hi derJörg,

First of all welcome aboard, yes i (+-) remember getting some feedback from you and TKaos when i first started bug fixing it long ago.

Source code was public, i haven't it here now but if you really want it i can look for it and maybe upload it to the site so we have a bit more garbage preserved :P
You can grab the last public src here (http://www.tosec.org/userfiles/(tosec-v0.6.12_tim_source).zip) and the last setup here (http://www.tosec.org/userfiles/(tosec-v0.6.12_tim_setup).zip) (0.6.12), i don't know if it will work and it will be able to fetch the DB but looking at the source you can find how it worked and where they were/are located, for the db it is here (http://www.tosec.org/xoops/tim/database.zip) and updates list at update.txt (you won't need it with that db.

After that i did some updates and fixes to up to 0.6.14 i think but it is ALL really awful believe me, i can try to package it in the next days and give it to you anyway so you can see how it should NOT be done :)
There was really no documentation about it or the DB, you have to learn by just checking the messy code or by opening the db (is a SQLite db, you can open it with something like SQLite Database Browser, that was one of the first things i did and created a diagram on how it was 'designed', you may find it useful so here (http://panda.pt-server.com/tosec/old_tim-wip/timdb.png) it is. If you look at it and you know a bit about DBs you will see it is really BAD done (i suppose it wasn't planed at all, someone just added a table and started adding columns when they were needed), there is no normalization, values were not atomic and no nothing.


The main problem in tim was that they wanted it to do everything imaginable but the persons coding it couldn't make it do anything properly, i understand and agree that the idea wasn't bad at all and if i had the time i would like to try and do something useful but nothing like TIM.
You may also check the work of Xang, he is a member here and had a tool that resembled it, while it is far from perfect it may have some good features and you both can discuss about it here or privately and get good ideas.

My suggestion if someone wanted to do something like that was, first cut off all unnecessary features, clrmame is already here and really powerful, there is no need to loose time doing another rom manager (some basic and useful features could be there but focus on the main idea), don't try to have an all in one  with tools for renamers, rom management, browsing of the db and so on.

The main idea at least for me would be something to browse dats information, anyway you would need to:
-> have a proper db (and this would give you a bit of a headache when you start thinking about dat updates) or use the dats (a lot less power i guess)
-> a parser for tnc rules (the one in tim is HORRIBLE, i updated it a bit to fix parsing errors but still really bad) - doing this is also a nightmare because tnc has a lot of weird rules and a few stupid cases (still) that can be parsed correctly without knowing some fixed values.
-> plan it properly before coding anything so you can focus on desired features and do something reusable, solid and usefull, avoiding duplicating code and rewrites later
-> REALLY evaluate your time and knowledge so you don't end up losing a lot of your time planing something that will never take off.

Thats why i quit of it, too much garbage and too much work, fixing it properly would be worst than doing it from scratch because there was really nothing valuable to keep.

Currently i'm lost in academic work that i can't leave undone and may be like this until summer (with a bit more free time sometimes off course) but my idea after tim was start evaluating our needs and fill those small gaps, producing small but more useful tools that are useful now and could be useful later too. So i have a parser to extract information, some tests with a proper db that is not perfect but for now may help us eliminating errors and one or two more apps that help me checking mistakes in current dats. Avoiding monolithic / giant all in one apps that would take me a lot of time and never be ready to do something useful with my lack of time, instead i just do or plan to do small useful tools like i described here somewhere, keeping one or other bigger projects/ideas to be done some day.


Thats all, feel free to explore, play, break it and so on, if you need something more just ask and i will try to help you out again :)

Hello PandMonium,

...a very detailed answer. Thank you for clearing things a little up and thank you for the links/resources.

In my opinion it would be a good thing "to preserve a bit more garbage" :) and put the latest TIM version online (including source code).

I thought over your points and must admit that you're right. clrmamepro does already 98% (at least of what I want it to do), and there had to be a really good reason to reinvent the wheel. clrmamepro has two drawbacks which I can think of spontaneously, which are:

• that empty folder issue, which is a real road block in my opinion and should be solved some day
• that clrmamepro has to scan/rebuild all DATs separately. However, this not a missing feature, it just raises the amount of time needed to scan large folders (it was indeed a great advantage of TIM that it could scan everything in a single pass)

kind regards,
derJörg

hey,
the problem with releasing tim has it is is that it would be really broken again (so would be only for you or others to test), since the db was really bad and fixing it would take tons of time, i did a couple of fixes to be able to handle the last dat release but for that some updates would need to be applied (which i don't remember quite well) to add / change stuff in the db before continuing and after that all dats should be re imported which would take ages and randomly crash sometimes ;D (hint: i don't have access to tim updates server and don't want to change much more stuff tim related :P)

Anyway i may take a look at some folders and prepare a zip with the source code i had changed without a problem, also note that it will never be able to import newer dats (if they get released one day :P) because of the latest changes in TNC that weren't applied in tim parser too.

Clrmame pro is THE tool to use when talking about managing roms, it is powerful, fast, reliable and so on. TIM had a few good things, having a single db would mean you could check what a file was at first try instead of going trough all the dats (but i suppose no one has tons of sets without having a small idea of what they can be and even with some thousands batch run of clrmame and time does the work), other could be the creation of folders tree, but those advantages are small for such a huge tool and should be done with small apps :P
The empty folders think would be cool, i don't know why it doesn't work on cmp, if by option or because it doesn't makes sense in the view of Roman or even some techical problem with having that.

Lets see what i can do in the next days :)

Hi,

First of all, I'm chuffed to finally see this up and running again!!! It's exciting to actually have some where that will bring all the TOSEC activities back together again. I remember following the site when it was originally up and we could submit screenshots, etc, etc.

I too remember using TIM and thinking how naff if was. It had all the right ideas, but had tooo many options. Well, certainly for a first release.

I love CLRMAME, but I hate it too. It's a great renamer, but it really is poor if you want to manage multiple sets. I've often wondered why it doesn't have a feature to gather all DATs up into a database and scans this when it comes across a file. IE. A Check everything known and rename. Ok, this is complicated, but is what computers are good at.

Well, why am I posting.. If I had more time, I would continue writing a tool I've been coding in Java (yes, I know, but I figured it'd be multiplatform so ideal for me where I run Linux, Windose and now and again, Mac). I've often dabbled and created a TNC parser for the files, a database to store the general structure of a comprehensive set. However, the design lends itself for both adding new sets (i.e. comparing old. This basically deals with new sets) and also for scanning purposes. I've also coded the general file scanning down trees, a bit of the GUI stuff (in Swing for those technical) and dealing with CRC32, MD5 and SHA hashes.

I know I want this tool and never understood why someone hasn't written it. In my view, the initial release should be:

1) Database with good design for set loading (I've used hsqldb which is open source and very powerful). The initial V0.000001 of database is complete, although I might compare the above schematic.
2) Ability to add new sets (support for CLRMAME Dats and XML initially)
3) Ability to set scan of a directory, with option sub path scanning.
4) Ability to move/copy files to destination and zip if required (this is where I thought CLRMAME could be used initially once copied to).. So a half way house to getting this going.
5) A simple gui to drive

This is all pipe dream as I have a professional IT job and thus don't have time. I've really used this to create multiple small java apps to do these small little things (i.e. scan folder, crc32 check, read/write to database etc) as ways of learning how best to implement these (i.e. I've got the scanning very efficient with buffers/tricks).

I'm writing this to just get people's thoughts and ideas of what they would want to initially see?

At the moment there are no promises of doing anything. I'm not sure if I'd even want to release my code right now.. However, I've often wondered whether setting up a new open source project to do this might move this tool forward????

...I'm not here to waste peoples time.... After all, I have lots of other things to do. However, I've now got my cabinet built and up and running. A tool that manages my set would be great....

Regards

Sp33dy

I forgot to say, I've been tinkering with this for at least 6yrs. This should give you an impression on how likely I am to put anything concrete together! However, it took me over 10 years to finally build my arcade cabinet (which is now running and great fun!)...

I'd also like to have a new tool but clrmame is really powerfull also my programming knowledge is not the best, some tools I wrote for myself for various file checking which helps me renaming a bit.
For more information about TIM or databases you'll have to wait for PandMonium, he'll be back in 2 days as far as I know.

..and you touch on an important point here. I look for a tool that does all the renaming and managing for me. However, a later release is where I'm worried about the renaming/TNC processing. I.E. get the tool to manage the roms by moving them first, then worry about creating new lists and changing the flags in the file name.

In fact, my argument all along is that I don't think the filename should have all the flags in it. I've always felt a DB is required with all this detail and that you rename the files to meaningful names that are stored in heirarchy folders/zips. Using the Database to manage what I do/don't have and cataloging where I've stored them etc, etc. I want to use the tool to produce a folder of roms/files that I would like to use in the cabinet. I.E. produce a list of all amiga games that the Bitmap brothers created, ony UK version and that aren't hacked. I'd expect a folder of zips to be created. Each zip containing the disks required for that game that are known to be working and potentially linked to the screenshot/artwork that is also linked. I expect the zip and contents in to be named without the extra flags etc. However, I want the tool to have all those flags in order for me to make the selections that I would like to use to create my set and also to keep a tab on what I do have.

I hope that makes sense????

Ultimately, I'd love this tool/database to be able to perform cross linkage between the entire database. I.E. show me all versions of Space Invaders created for all systems. Getting a count of games created by different manufactorers, authors, etc, etc would be sooo cool. Turning this into a cross referencing tool for all games created. Way beyond where thinking/actual toolsets are today. It's all pie in the sky and I think TIM was trying to be all of this. As mentioned previously, only small steps could possibly make this work. Those small steps would only work with a well thought out initial design.

It's why I'm writing this. I'd just like to hear other people's views. This might be worth investing my time in, or it might not. Just wondering what people think and how they perceive using such a tool.

I guess you really have to wait for PandMonium there, he can tell you alot more about our database.
Lets say, there's a way for us to see all releases of a specific publisher / person / scenegroup / scener, I think there might be also a way to export a list of all UK releases but remember that first all software needs a country flag then.

A (my)SQL database would be a great idea. It could exist along with DAT files and be a sort of source for creating DAT files and managing files. However if you'd like to use it to manage files I think you'd need a rom manager capable of connecting to a database or a separate tool. If you can design the database correctly (in terms of speed / efficiency) filling it with data will be a relatively easy. It would be platform independent + there are SQL bindings for most of the popular languages so creating a frontend for displaying/sorting results would be easy too.

Hi guys,

nice discussion that started when i was away :P

I will try to cover most of the stuff discussed here but will surely miss some since i have other things i need to do soon.

PART of the idea behind TIM was good and useful, the problem is that it would take years to do, also we would need to fix most of the aspects around TNC and TOSEC sets which difficult these ideas.

I, like you have the same idea/dream and have been pursuing it from the last 2 years or so (since i dropped TIM). Creating a set of useful tools about TOSEC collection, in my view doing a single, monolithic tool is not the solution and will end just like TIM, different small tools would be much better.
Renaming all sets with a single db should be done by some small tool and just that, not a powerful tool since cmp is already great and doing something big takes much of our free time, my current idea if i had to say something would just be a small simple tool that would pick some dats and rebuild from source to destination (creating an intermediate db with hashes and filename or not, dats already filtered with only the wanted sets for example) bla bla.
About the parser, TNC is really nasty and huge, i've a parser to extract all that information, identify some errors and so on, it is NOT perfect but it is something, currently WIP and unfortunately for you and me i guess done in C# (i was trying to use something new and of fast development).
That parser is used in a couple tools i have to check dats and so on.

Next, the first months after getting tired of TIM DB and thinking about an alternative led me to plan a new DB, i got huge with a lot of tables (100+) and i only normalized part of them (the most used), after that i decided to test it and start something that could be fun and decided for something more web instead of an local app, created some phps and imported them to mysql, that's how we can know see what TKaos said, for example from "Bencor Brothers" we have
...some of the sets use "BB" as the group abbreviation and there are also 84 sets using "Bencor Bros" which are wrong and need to be fixed, this is just an example but after having all information parsed and well organized, it's just a matter of code logic or SQL to do whatever we like.

Anyway there are a lot of issues regarding this, first and about my simple prototype, i started it just as a test and so it is not WELL designed, should be completely redone and secure before thinking about go live.
Second a lot of problems arise with not so clear TNC issues that keep changing and create the need to change the DB or DATs or parser.
3rd, the updates, this is a really nasty problem, with dats being renamed and sets fixed/added/dropped it gets really complicated to discover that just by looking at dats and not giving a LOT of extra work to renamers.
The original setnames should be maintained but the possibility for renames with some user defined name shouldn't be a problem.
Making renamers start using that system to add new sets is impossible, for ISO and new things it is possible since things are being now dumped with nice rules and added, but renamers prefer to handle files and generate cmp unfortunately, parsing their new dats is possible but gives a lot of extra code and possible points of failure :P
I guess server requirements to bring it to life would be higher than i have currently :P
Current datfiles (noniso) are full or errors that need to be fixed (values inserted has titles, scene groups, publishers and so on), that is our current goal.

Anyway those are my ideas, hope you can understand something out of this pile of un organized ideas and feel free to ask, that is my current view of it, the main thing would be a web something to have a central point of information, a few tools could be done and based on it, a rebuilder using dats/xml/db/something, a local app to browse information with a local db (eg. sqlite) or dats (cmp, xml, something), and so on.

The main problems with it are definitely time we have (at least i don't have much currently), existent rules and definitions in TOSEC that you or i can change just because they are clearly not a good option and in a few aspects knowledge :P

Time is the critical issue! I just wish I was a student again!!! All that spare time I used to have, but way before the internet thing. Lots of hacking on the Amiga, but it was just that; playing and wasting valuable energy....

I think the database is the critical item in all of this, after all, this is what you guys are kindly creating (I just don't have the time to do this. Although, I've got approximately 300 amiga disks in the loft to dump to pc. Many backups of demo disks [SAE #1->#100's] and a whole load of others. For some reason I liked to collect these, not sure if any are corrupt now). If the database is designed well, then the rest will fit around it.

I don't see the new versions of releases a problem. In fact, I believe I'd already solved this problem!!! It's one of the driving reasons why I've been considering doing this. My theory/Logic? If I load v0.01 of a dat, the database in my view will have one table that knows a single copy of every rom, other tables know about sets and linkage between the two. When v0.02 dat is loaded (this could be the same set [i.e. Amiga games] or could be by a different renamer [i.e. Good amiga set]). When that set is loaded into the database, it creates a new set entry in the appropriate table and then as it inserts the roms/disks, it checks unqiueness for CRC32/MD5/SHA/Size and either adds new version or makes an entry in a history/alternative version list. In this way, it will be easy to maintain different sets with cross overs, history (i.e. rollback sets) and other issues. For me, this is very very important. I may not have explained well enough and I may have to draw a few diagrams to explain. Howevver, I really think it's a simple problem to resolve, but would make all our lives easy enough.

Your description below of how/what you are doing to search the database is exactly what I would like to do. I'm soooo geeky with games that I'd love to find the author who has written the most games etc. However, I'd also love to have ratings from both magazines, popular websites and also open up to the community to rank and rate.

Touching on community, you've had the same thoughts as I ! This tool in my opinion should be two fold. A local tool that performs the rom renaming etc. It also needs to be a server version to be online. Professional Websites are my daily job and hence why I know Java insite and out (just the technology the company uses). This is why I started the local tool in java with an open source database and have had a play with some JSP to also make it available via a server. It works very well. This reinforces one other point. The database technology is irrelevant! It doesn't matter if I use HSQLDB, MySQL, DB2, Oracle or any other database language, the important point is the database schema. It'd be very easy to allow the user to select the database of preference. Building in one free database (HSQLDB in my view) makes sense as an out of the box, bundled version ready to go makes life easy for those that aren't bothered by what they prefer/have license for.

The more I type, the more I want to make this work. Although, I've got a local school's website and a membership system for snooker club to write in my free time before this. I've also got real life and a lot of house DIY to complete... I've started planning to see what I could do over the next 6 months..

Regards

Sp33dy

(don't have much time to detail some parts a lot atm :P)

I think you already understood that time is the key and since this is a hobby for all of us it is hard to start something very complex, it has to start with small steps (at least this is my view and what i've done). The next huge problem is how TOSEC and other collections work out, you will have to pick info from dats that may change and have mistakes in most of the cases, you can't control most of the rules and change them when they are bad and a small change somewhere may end up forcing a schema change or something that takes time.

The most important part is planing it right but in my opinion the deep using on it and complexity depends on the team developing it, in this project i did some planing but nothing huge or always documented because it is just me and i would end up wasting all my time planing something and doing zero, so i started with a big idea but some aspects not well defined, was just like a big test that was getting more and more updated adding some parts to it, now i've come to a point that shows me that i should just rewrite this core properly so i can use it as a base for something and not keeping rewriting and duplicating code in an adhoc development :P

Now, a bit more about your last post, i'm not sure i understood it all (my english isn't great):
Storing releases information (and so datfiles history) is a great idea that i haven't ended yet and don't have any idea if i will ever have time to do so (i grabbed some older packs, have some olders tncs, tugids, etc but for now just have small stats about these packs and not full information).
The main problems with that idea that i've recall and can remember just now go from the huge size it would take in the DB, to the millions of errors you find in old dats and also information that is impossible to parse.

I mean, TNC changed a lot and the older sets have flags not recognized now, information wouldn't be parsed there until a parser for that was done (and there is no great documentation about the older rules + they weren't always followed).
Next, if by any mean we happened to extract all information from flags you would end up with a TON of invalid values, from invalid dates that are easy to check to completely invalid groups of information like: inexistent publishers that are just garbage and have been fixed, descriptors, countries and language codes that are invalid and don't even exist, scene groups, persons with name not written accordingly "(Last Name, Name)" and so on, resuming a ton of errors. In a db that covers all details of a set, you will need to have this kind of information, having older sets details too means all typos and errors that got once in one of these flags needed to be added, having a list of scene groups where more than half would need to be tagged as invalid/errors.

In my view that is a little too much, too much work, too much information (half unneeded) and so my planned approach would be having that many details only for the current/actual datfiles, older versions of them should only have datfile details and stats, eventually also the existent setnames and roms but not covering flag details for each of the older sets.

After that you have to take in consideration that datfile names change in time, to fix company or system names that were wrong, changes that need to be manually tracked (you would need to tell that COMPANY System128 - Dat (2000-00-00) was an older version of "Company System 128 - Dat (2001-00-00) where the datfile was renamed, or correctly marking category changes, datfiles that were merged or split because they are not always (and we can see that along the time) a direct update where only version changed. Identifiying this automatically is near impossible or will be dangerous, causing errors.

Datfile updates like you described would be easy but would also generate an huge db with tons of duplicated sets, we have near 350.000 setfiles now, each one with AT LEAST one image (software image, not picture), one title, year, publisher, crc, md5/sha1, filesize, filename, extension and tons of extra info, having it all that info for the last release already takes tons of table entries and MBs, thinking about dozens of releases where all this info would be immense.

IMO this will lead to an idea of storing datfiles and setfile names, with setdetails only for the latest dats (or sets that didn't change much). To avoid such duplication, identify changes would be needed, so adding 2 versions of the Amiga Games ADF dat would not add 2x 20000 sets but 20000 sets first and then 1000 new ones + 500 renamed ones and so on (+ rom changes + bla bla). Even that is complex already for the available time of most, dealing with datfile renames and separation in more than one dat, set moves between dats and so on is also complicated.
It is not that hard to figure out setfile renames or moves on single rom sets just by comparing hashes+size, but in multi rom sets things get harder and harder cause of shared files between sets (crackintros, readmes, something else), redumps of some files and so on that make it impossible to always know if sets are related or not.

Adding even more stuff community related will just make it even more complex, you can have those goals but you need to define what is the basic, important part and what should be done later if still a good idea.

My view is that, currently the important part for me and the project is a way to easily browse and relate information existent (systems, companies, publishers, groups, etc, etc) in the latest existent datfiles so they can be checked and renamers can fix the identified errors, if possible adding a bit of information on older releases (+ thinking of a WIP system easier for renamers one day), this is what i've done lately (when i had more free time, currently it is in need of urgent rewrites so i don't waste so much time by repeating stuff + make it securer).

I will not talk about any technical aspects of tools or so, just answered SQLite for an app because it seemed good to use with options like Qt & c++, that is not relevant for now anyway.

...and note that i didn't even talked about the problems with using datfile values, for example with publishers you just have a string there (name), there are a lot of publishers that may have shared same name, this is really bad when happening in the same system (duplicated person names, sceners, and so on), also adding details for setfiles is complicated when you later will end loading newer setfiles.


That's it, hope you can get something out of this pile of text, also if you like i can show you what we've got now, just pm me.

I really like the idea of having a real TOSEC db. I read the discussion and it just came to my mind that using names for DATs and files on... say a low level of the whole structure (?) can be a cause of many troubles. It would be easer to operate on some kind of ID strings. A name of a file is a result of a few values (flags) and could be easy created by getting all informations from DB. Final name of a file should be the last step of the whole process. It can change from time to time but if there is an unique ID you will always know what file you working on. I guess it could be similar with the DAT names. Changing a DAT name is sometime confusing for people who try to rebuild / update their sets so having an unique ID along with the names sounds good to me. Actually we already have sort of ID for files: CRC/MD5.

Hi Diabol,

Yes, the idea is not new and has been discussed a few times, actually for the setnames it already could be done in TIMs renamer tool, where info could be entered in each field (even if it sucked anyway) and in the end the setfile would be generated.
The same idea also happens (afaik) in iso part, the toseciso guys add info like that, inserting new dumps and filling the information (it doesn't handle all flags but the only need a few of them anyway), a (+-) similar system is used in some other projects systems like dat-o-matic for example i think.

The idea is indeed great and would help in a lot of aspects (that is the main reason i defended it long ago), the main problem is for renamers since it will give them a LOT more work (or a complex / powerful solution would need to be found), unlike iso, where sets are being dumped and added from start, with disks and old stuff we have already a gazillion files, the main work left is fixing all the errors and not adding new stuff, the process of fixing it on a db and relating the correct file/set with the right set in a db, plus adding new sets, with details as size, hashs and all those flags is way more time expensive than just editing setnames in files or in the dat itself and later check the dats again and again (what i +- have to do now in our tool for validation purposes).

Finally, the datfile ids for dat renames could be useful, indeed but cmp doesn't work like that and so just warning that dat X = day Y now is enough, it wasn't done this time because changes were too much to know. Also a great part of dat changes (most i guess) are not simple renames, but separations or other, so it would generate new ids, like dat id 1 -> id 2 and 3 and so on.

Thanks for the suggestion anyway, i also like the idea in some points :)

As a quick update (seeing you responded to my other thread offer). Although I've not got any real serious time. I've had a dig around some of the free database designers. This one took my eye:

http://www.dbschema.com/

I like it as it's java based, free (to a given restriction) and works very well. It's also compatible with many different databases, including HSQLDB ! I got it hooked up over the weekend and am currently creating a few schemas whilst modifying some of my java code to load it. It's extremely good at showing the relationships of the tables and the data within!! So you can actually see the table foreign keys, blah blah blah.

Just thought i'd let you know. You never know, I might create a 1st pre alpha schema for a pre pre pre alpha tool for general loading of dat and doing renaming (zip and possibly 7z level only)... At least this would save me having to load each dat profile into clrmame and then doing a scan to see what I do/don't have..

Incidentally, what's the process for reporting having lots of roms that a set doesn't recognise? I know from many years ago I pulled down lots of roms that seemed interesting but didn't have time to deal with. CPC stuff springs to mind. Know idea if the stuff i have is of any interest to anyone. Just wondering whether there is a process to post it (say newsgroups) or a way to report the files. This probably isn't the right part of the forum to ask... so I'll accept a flame here...

Regards

sp33dy

Hi,
don't know that tool but i might take a look since i was looking for some alternatives too. Until now all my small tools and tests i've done were using PowerDesigner (http://en.wikipedia.org/wiki/PowerDesigner), guess having a free (and equally good alternative would be cool. Hope i manage to have time and will one of these days to test it out, maybe in the next month :/

About the 2nd question, i guess i didn't understood it very well, if they are unidentified files you just need to list those files, just have them in a sepparated folder and get a list of it, you can also try to scan them using other dats so you might discover what you have there (see unrenamed / garbage dats over romshepherd.com), if they are of interest i'm sure you can put them somewhere to wait until someone can check them :P

The more I play with dbschema, the more I like it.

I agree with some of your previous points. Ultimately, I think the database tooling should be client/server based. The tool in my view (being a java dev) would be easy to create in a JSP application that could be deployed to any given J2EE container (i.e. Tomcat, WASCE, etc, etc). I've always assumed the tool should be a standalone app (i.e. Java Swing GUI based or of such design). The problem with this is the time it takes to do all the GUI elements. My background is web based solutions, so this fits in better with where I am.

I'm currently looking (time allowing) at making the renaming of roms with the current sets priority 1 as I want to point a directory of lots of different images to a database of dats.

If/When I get this up and running. I'm then going to look at the naming tool. I also figure that using a client/server based solution would lend itself better to the TOSEC site. I.E. it could be hosted, in order for a collaborative means of renaming/submission....

Again, wild thoughts and where I am today.

Regards

Sp33dy

P.S. What is the problem with the TNC issues? I remember coding a TNC interpreter of the files once (unfortunately lost code due to a hard drive crash). It was very simple to do and for the life of me, can't see what problems there might be.. Except if flags are reused for meaning. I.E. if B-Flag for bad was changed to B-Flag for bootleg (lame example). This is the only problem I see.

Hi there,

In my opinion you're having ideas that are way more than the needs of the project currently and the time we have to do such a thing :P
Really, my goals currently are to gather time to do some small apps to help me/renamers or work on some sort of db that few can want to use to manage some info, i had way more free time before and still was short of time for it all, thats why i think you may end up planning something you will not have time to finish, using some heavy tech when we probably don't need such an enterprise solution, someday we will be talking about J2EE, SOA, BPEL, Hibernate, Oracle and so on ;D And also running such technologies usually costs more and needs more power (tomcat, glass fish, dedicated servers etc.).

Even if the idea is something small, we should have attention to our time and needs before trying to test some technologies and talk about servers, clients and integrations, and what the project really needs and people are willing to use, the last "too big" idea was TIM :P

For now i guess i will keep trying to find a bit of free time between days in order to advance with the small apps / ideas i have, i really don't have a lot more time unfortunately and so the db work i've planned is halted a bit, anyway feel free to have these ideas, i will always give you my opinion, answer possible questions and give you my support when i can.

About TNC:
The issues are in the last page of the pdf i think, not sure if they are online. It is known that our naming conventions way far from "very simple" and still we have tried to improve it. The issues are related with the complexity and rules of all flags put together, there isn't anything wrong, just that it isn't anything robust too (or even close to that).

Some examples from what i recall:
Will not talk of existent errors from wrong use of tnc before (eg.: dates where someone decided to use YYYY-YY to symbolize something i don't know, ending with 1989-91, but in cases like 1998-01 we will never now what they are now without checking again) because those aren't even tnc errors, just lack of checking before.

Version flag is too weak, just expecting " vSOMETHING" isn't quite great, forcing a number after that will left out " vA.0" or some strange cases, revisions aren't parsed at all and are part of title, no rule for that but i don't like the idea of adding more and more rules to something that is already complex and in some cases broken.

Most of the flags have some fixed values considered "valid", media label is the last ( ) flag and is also a flag that can have any value - the label present in the media doh :P, this means that any wrong flag that appears before this is catched as a valid media label (imagine "Diak 1 of 2", a valid media label) and so errors are not easy to automatically check without analyzing all parsed media labels...

Dump flag rules aren't precise too, cases of fix and modification flags can have 2 types of values mixed together (they have descriptors + groups): [f PAL] = you will never know if PAL was some scene group or the fix descriptor. Currently we use fixed lists of allowed descriptors that were gathered from existent values in the db, anyway it still is broken because new values need to be identified and inserted + there could be cases where values aren't a descriptor but will be identified as one.

These are the ones i recall and the worst ones when trying to parse stuff, i'm not saying it isn't possible to parse it all, it IS indeed possible and not that hard (wouldn't say very easy too :P), but it will also be a 'Okay' parser, not something really robust that will get all flags separated correctly with 100% precision always, just what happens now :)

:(

Just to say I'm still lurking, but life really has gotten stupid again. I can see the end of the tunnel, but nothing moving here. I wish I was a millionaire!!! In fact, not quite the same, but I'm looking at getting out of the IT industry and into teaching. That would give me more free time for doing other things!

Hey, i've been really busy too for the last weeks, finally have some free days now but will use most to do other stuff, real life comes first :P
As for teaching, at least here in Portugal it is really hard to get in and grab a good job/position unfortunately but good luck with your goals :)

It seems like we are all a little bit busy atm, I didn't rename a single file since 2 weeks now but should change soon. :)

Bit off-topic. I barfed out this mock-up for a very simplistic renamer:

http://i50.tinypic.com/214tdf6.png

I tried coding it in Just Basic and Rebol, but gave up after a few minutes as I'm too stupid for these coding languages. If anybody likes my mock-up, feel free to code it :)

Thanks for your suggestion :)

Tools to aid renaming of files and such are something debatable because most of the renamers tend to dislike it and prefer to rename manually, some not and if there was something at the same time really powerful and helpful but yet really simple to use that wouldn't add a few more steps to rename one file i guess most would think about it.

Anyway i've done something that can highlight / verify set names according to our tnc, it is not specific to rename sets but is useful sometimes to avoid obvious errors (currently not content errors :P):
http://www.tosecdev.org/media/tde05.png
http://www.tosecdev.org/media/tncchecker_20091113.png

It has a different purpose than what you proposed really, i like the idea of having a tool to help renaming, at least to aid new renamers that usually have questions about flag options and order. Unfortunately i (we) am really busy currently with rl projects as well as a ton of different tosec ideas and other projects :(

Keep these coming. I've nearly finished the private website for local school and the java app for snooker club. Once these are done, this is going to receive some more of my attention. The TNC checker is exactly what i'd expect in the tool.

I'm still around, but overloaded with RL.. However, someone on Pleasuredome has put a few screenshots of a tool he's writing.

http://forum.pleasuredome.org.uk/index.php?showtopic=17382&st=0

Not sure if you have a logon here (if you don't and need an invite, let me know).. Certainly seems interesting. The guy is looking for some web hosting help. I recommended he posts details here, incase anyone has the time and energy to help..

Regards

Sp33dy

Hi sp33dy :)
Yes i know about it, GordonJ is a member of toseciso afaik, he dumps discs and so on. I remember him talking about his ideas and have also seen the tool screen shot, it looks promising and may help a lot in managing large collections of dats, once he gets it running i will link it from our site off course :P

http://www.romvault.com/