New version of TiM

[Guillaume]

Hi,
The development for TiM in progress?
I find a old version 0.6.14 beta2 on net. This is not usable?

 

[TKaos]

PandMonium fixed alot bugs in TIM like 1,5 years ago and released a beta version but he said that the sourcecode was too broken to make a good version.
The beta was fine because it was alot faster with loading the DATs but there were still some bugs in it, he then stopped working on it and said you'd need to re-do the whole code to make something usefull but fixing more in TIM's old code would be wasted time because of the broken source.

[PandMonium]

Hi Guillaume,
Where did you find that?
TIM development stopped long ago, it was bad planned (or not quite planned at all ) and had a lot of things that didn't work properly, so it wasn't never reliable and wasn't being used for long, even delaying releases (thats why the last official one was not applied in tim.db. I once started to fix it but it was plain broken and would require an immense amount of time to fix but even then there were parts that needed to be redone so i just stopped.
It had some nice parts / ideas but others weren't that good and there were/are already excellent tools to do so (clrmame to rom auditing), maybe we will have some more tools in the future but nothing like TIM (afaik), it was too big and with too much unnecessary features and should had been split in smaller tools
AFAIK Xang (an user here) posted a tool he is using to maintain his sets, not sure how it works and if it does work correctly but at least it is something similar to TIM/tugid

[Guillaume]

ok, thank you for your answers

[derJörg]

Hello PandMonium,

TIM development stopped long ago, it was bad planned (or not quite planned at all ) and had a lot of things that didn't work properly, so it wasn't never reliable and wasn't being used for long, even delaying releases (thats why the last official one was not applied in tim.db. I once started to fix it but it was plain broken and would require an immense amount of time to fix but even then there were parts that needed to be redone so i just stopped.
It had some nice parts / ideas but others weren't that good and there were/are already excellent tools to do so (clrmame to rom auditing), maybe we will have some more tools in the future but nothing like TIM (afaik), it was too big and with too much unnecessary features and should had been split in smaller tools

...perhaps you remember me throwing weekly bug reports at you...

...as TIM is now officially discontinued, is there any chance to release the source & db schema for studies? Just to enable people who want to create new applications to not make the same faults again? I still like the basic concept of TIM and I think it could have been auccessful if it was more streamlined...

Anway: any chance for a (final) TIM source package?

Kind Regards,
derJörg

[PandMonium]

Hi derJörg,

First of all welcome aboard, yes i (+-) remember getting some feedback from you and TKaos when i first started bug fixing it long ago.

Source code was public, i haven't it here now but if you really want it i can look for it and maybe upload it to the site so we have a bit more garbage preserved
You can grab the last public src here and the last setup here (0.6.12), i don't know if it will work and it will be able to fetch the DB but looking at the source you can find how it worked and where they were/are located, for the db it is here and updates list at update.txt (you won't need it with that db.

After that i did some updates and fixes to up to 0.6.14 i think but it is ALL really awful believe me, i can try to package it in the next days and give it to you anyway so you can see how it should NOT be done
There was really no documentation about it or the DB, you have to learn by just checking the messy code or by opening the db (is a SQLite db, you can open it with something like SQLite Database Browser, that was one of the first things i did and created a diagram on how it was 'designed', you may find it useful so here it is. If you look at it and you know a bit about DBs you will see it is really BAD done (i suppose it wasn't planed at all, someone just added a table and started adding columns when they were needed), there is no normalization, values were not atomic and no nothing.


The main problem in tim was that they wanted it to do everything imaginable but the persons coding it couldn't make it do anything properly, i understand and agree that the idea wasn't bad at all and if i had the time i would like to try and do something useful but nothing like TIM.
You may also check the work of Xang, he is a member here and had a tool that resembled it, while it is far from perfect it may have some good features and you both can discuss about it here or privately and get good ideas.

My suggestion if someone wanted to do something like that was, first cut off all unnecessary features, clrmame is already here and really powerful, there is no need to loose time doing another rom manager (some basic and useful features could be there but focus on the main idea), don't try to have an all in one  with tools for renamers, rom management, browsing of the db and so on.

The main idea at least for me would be something to browse dats information, anyway you would need to:
-> have a proper db (and this would give you a bit of a headache when you start thinking about dat updates) or use the dats (a lot less power i guess)
-> a parser for tnc rules (the one in tim is HORRIBLE, i updated it a bit to fix parsing errors but still really bad) - doing this is also a nightmare because tnc has a lot of weird rules and a few stupid cases (still) that can be parsed correctly without knowing some fixed values.
-> plan it properly before coding anything so you can focus on desired features and do something reusable, solid and usefull, avoiding duplicating code and rewrites later
-> REALLY evaluate your time and knowledge so you don't end up losing a lot of your time planing something that will never take off.

Thats why i quit of it, too much garbage and too much work, fixing it properly would be worst than doing it from scratch because there was really nothing valuable to keep.

Currently i'm lost in academic work that i can't leave undone and may be like this until summer (with a bit more free time sometimes off course) but my idea after tim was start evaluating our needs and fill those small gaps, producing small but more useful tools that are useful now and could be useful later too. So i have a parser to extract information, some tests with a proper db that is not perfect but for now may help us eliminating errors and one or two more apps that help me checking mistakes in current dats. Avoiding monolithic / giant all in one apps that would take me a lot of time and never be ready to do something useful with my lack of time, instead i just do or plan to do small useful tools like i described here somewhere, keeping one or other bigger projects/ideas to be done some day.


Thats all, feel free to explore, play, break it and so on, if you need something more just ask and i will try to help you out again

[derJörg]

Hello PandMonium,

...a very detailed answer. Thank you for clearing things a little up and thank you for the links/resources.

In my opinion it would be a good thing "to preserve a bit more garbage" and put the latest TIM version online (including source code).

I thought over your points and must admit that you're right. clrmamepro does already 98% (at least of what I want it to do), and there had to be a really good reason to reinvent the wheel. clrmamepro has two drawbacks which I can think of spontaneously, which are:

• that empty folder issue, which is a real road block in my opinion and should be solved some day
• that clrmamepro has to scan/rebuild all DATs separately. However, this not a missing feature, it just raises the amount of time needed to scan large folders (it was indeed a great advantage of TIM that it could scan everything in a single pass)

kind regards,
derJörg

[PandMonium]

hey,
the problem with releasing tim has it is is that it would be really broken again (so would be only for you or others to test), since the db was really bad and fixing it would take tons of time, i did a couple of fixes to be able to handle the last dat release but for that some updates would need to be applied (which i don't remember quite well) to add / change stuff in the db before continuing and after that all dats should be re imported which would take ages and randomly crash sometimes (hint: i don't have access to tim updates server and don't want to change much more stuff tim related )

Anyway i may take a look at some folders and prepare a zip with the source code i had changed without a problem, also note that it will never be able to import newer dats (if they get released one day ) because of the latest changes in TNC that weren't applied in tim parser too.

Clrmame pro is THE tool to use when talking about managing roms, it is powerful, fast, reliable and so on. TIM had a few good things, having a single db would mean you could check what a file was at first try instead of going trough all the dats (but i suppose no one has tons of sets without having a small idea of what they can be and even with some thousands batch run of clrmame and time does the work), other could be the creation of folders tree, but those advantages are small for such a huge tool and should be done with small apps
The empty folders think would be cool, i don't know why it doesn't work on cmp, if by option or because it doesn't makes sense in the view of Roman or even some techical problem with having that.

Lets see what i can do in the next days

[sp33dy]

Hi,

First of all, I'm chuffed to finally see this up and running again!!! It's exciting to actually have some where that will bring all the TOSEC activities back together again. I remember following the site when it was originally up and we could submit screenshots, etc, etc.

I too remember using TIM and thinking how naff if was. It had all the right ideas, but had tooo many options. Well, certainly for a first release.

I love CLRMAME, but I hate it too. It's a great renamer, but it really is poor if you want to manage multiple sets. I've often wondered why it doesn't have a feature to gather all DATs up into a database and scans this when it comes across a file. IE. A Check everything known and rename. Ok, this is complicated, but is what computers are good at.

Well, why am I posting.. If I had more time, I would continue writing a tool I've been coding in Java (yes, I know, but I figured it'd be multiplatform so ideal for me where I run Linux, Windose and now and again, Mac). I've often dabbled and created a TNC parser for the files, a database to store the general structure of a comprehensive set. However, the design lends itself for both adding new sets (i.e. comparing old. This basically deals with new sets) and also for scanning purposes. I've also coded the general file scanning down trees, a bit of the GUI stuff (in Swing for those technical) and dealing with CRC32, MD5 and SHA hashes.

I know I want this tool and never understood why someone hasn't written it. In my view, the initial release should be:

1) Database with good design for set loading (I've used hsqldb which is open source and very powerful). The initial V0.000001 of database is complete, although I might compare the above schematic.
2) Ability to add new sets (support for CLRMAME Dats and XML initially)
3) Ability to set scan of a directory, with option sub path scanning.
4) Ability to move/copy files to destination and zip if required (this is where I thought CLRMAME could be used initially once copied to).. So a half way house to getting this going.
5) A simple gui to drive

This is all pipe dream as I have a professional IT job and thus don't have time. I've really used this to create multiple small java apps to do these small little things (i.e. scan folder, crc32 check, read/write to database etc) as ways of learning how best to implement these (i.e. I've got the scanning very efficient with buffers/tricks).

I'm writing this to just get people's thoughts and ideas of what they would want to initially see?

At the moment there are no promises of doing anything. I'm not sure if I'd even want to release my code right now.. However, I've often wondered whether setting up a new open source project to do this might move this tool forward?

...I'm not here to waste peoples time.... After all, I have lots of other things to do. However, I've now got my cabinet built and up and running. A tool that manages my set would be great....

Regards

Sp33dy

[sp33dy]

I forgot to say, I've been tinkering with this for at least 6yrs. This should give you an impression on how likely I am to put anything concrete together! However, it took me over 10 years to finally build my arcade cabinet (which is now running and great fun!)...

[TKaos]

I'd also like to have a new tool but clrmame is really powerfull also my programming knowledge is not the best, some tools I wrote for myself for various file checking which helps me renaming a bit.
For more information about TIM or databases you'll have to wait for PandMonium, he'll be back in 2 days as far as I know.

[sp33dy]

..and you touch on an important point here. I look for a tool that does all the renaming and managing for me. However, a later release is where I'm worried about the renaming/TNC processing. I.E. get the tool to manage the roms by moving them first, then worry about creating new lists and changing the flags in the file name.

In fact, my argument all along is that I don't think the filename should have all the flags in it. I've always felt a DB is required with all this detail and that you rename the files to meaningful names that are stored in heirarchy folders/zips. Using the Database to manage what I do/don't have and cataloging where I've stored them etc, etc. I want to use the tool to produce a folder of roms/files that I would like to use in the cabinet. I.E. produce a list of all amiga games that the Bitmap brothers created, ony UK version and that aren't hacked. I'd expect a folder of zips to be created. Each zip containing the disks required for that game that are known to be working and potentially linked to the screenshot/artwork that is also linked. I expect the zip and contents in to be named without the extra flags etc. However, I want the tool to have all those flags in order for me to make the selections that I would like to use to create my set and also to keep a tab on what I do have.

I hope that makes sense?

Ultimately, I'd love this tool/database to be able to perform cross linkage between the entire database. I.E. show me all versions of Space Invaders created for all systems. Getting a count of games created by different manufactorers, authors, etc, etc would be sooo cool. Turning this into a cross referencing tool for all games created. Way beyond where thinking/actual toolsets are today. It's all pie in the sky and I think TIM was trying to be all of this. As mentioned previously, only small steps could possibly make this work. Those small steps would only work with a well thought out initial design.

It's why I'm writing this. I'd just like to hear other people's views. This might be worth investing my time in, or it might not. Just wondering what people think and how they perceive using such a tool.

[TKaos]

I guess you really have to wait for PandMonium there, he can tell you alot more about our database.
Lets say, there's a way for us to see all releases of a specific publisher / person / scenegroup / scener, I think there might be also a way to export a list of all UK releases but remember that first all software needs a country flag then.

[Diaboł]

A (my)SQL database would be a great idea. It could exist along with DAT files and be a sort of source for creating DAT files and managing files. However if you'd like to use it to manage files I think you'd need a rom manager capable of connecting to a database or a separate tool. If you can design the database correctly (in terms of speed / efficiency) filling it with data will be a relatively easy. It would be platform independent + there are SQL bindings for most of the popular languages so creating a frontend for displaying/sorting results would be easy too.

[PandMonium]

Hi guys,

nice discussion that started when i was away

I will try to cover most of the stuff discussed here but will surely miss some since i have other things i need to do soon.

PART of the idea behind TIM was good and useful, the problem is that it would take years to do, also we would need to fix most of the aspects around TNC and TOSEC sets which difficult these ideas.

I, like you have the same idea/dream and have been pursuing it from the last 2 years or so (since i dropped TIM). Creating a set of useful tools about TOSEC collection, in my view doing a single, monolithic tool is not the solution and will end just like TIM, different small tools would be much better.
Renaming all sets with a single db should be done by some small tool and just that, not a powerful tool since cmp is already great and doing something big takes much of our free time, my current idea if i had to say something would just be a small simple tool that would pick some dats and rebuild from source to destination (creating an intermediate db with hashes and filename or not, dats already filtered with only the wanted sets for example) bla bla.
About the parser, TNC is really nasty and huge, i've a parser to extract all that information, identify some errors and so on, it is NOT perfect but it is something, currently WIP and unfortunately for you and me i guess done in C# (i was trying to use something new and of fast development).
That parser is used in a couple tools i have to check dats and so on.

Next, the first months after getting tired of TIM DB and thinking about an alternative led me to plan a new DB, i got huge with a lot of tables (100+) and i only normalized part of them (the most used), after that i decided to test it and start something that could be fun and decided for something more web instead of an local app, created some phps and imported them to mysql, that's how we can know see what TKaos said, for example from "Bencor Brothers" we have

Bencor Brothers Stats
Cracked:   95    Pirated:   0
Fixed:   0    Trained:   25
Hacked:   5    Translated:   0
Modified:   0    Total:   125
Published Sets:   4

found in:
Dats:   8
Systems:   2

...some of the sets use "BB" as the group abbreviation and there are also 84 sets using "Bencor Bros" which are wrong and need to be fixed, this is just an example but after having all information parsed and well organized, it's just a matter of code logic or SQL to do whatever we like.

Anyway there are a lot of issues regarding this, first and about my simple prototype, i started it just as a test and so it is not WELL designed, should be completely redone and secure before thinking about go live.
Second a lot of problems arise with not so clear TNC issues that keep changing and create the need to change the DB or DATs or parser.
3rd, the updates, this is a really nasty problem, with dats being renamed and sets fixed/added/dropped it gets really complicated to discover that just by looking at dats and not giving a LOT of extra work to renamers.
The original setnames should be maintained but the possibility for renames with some user defined name shouldn't be a problem.
Making renamers start using that system to add new sets is impossible, for ISO and new things it is possible since things are being now dumped with nice rules and added, but renamers prefer to handle files and generate cmp unfortunately, parsing their new dats is possible but gives a lot of extra code and possible points of failure
I guess server requirements to bring it to life would be higher than i have currently
Current datfiles (noniso) are full or errors that need to be fixed (values inserted has titles, scene groups, publishers and so on), that is our current goal.

Anyway those are my ideas, hope you can understand something out of this pile of un organized ideas and feel free to ask, that is my current view of it, the main thing would be a web something to have a central point of information, a few tools could be done and based on it, a rebuilder using dats/xml/db/something, a local app to browse information with a local db (eg. sqlite) or dats (cmp, xml, something), and so on.

The main problems with it are definitely time we have (at least i don't have much currently), existent rules and definitions in TOSEC that you or i can change just because they are clearly not a good option and in a few aspects knowledge