I'm new and I would like to join and help TOSEC

[LunaVorax]

Hello everyone,

First of all I would like to apologise for my clumsy english, it's not my native tongue so I may make some errors.

I recently noticied that TOSEC had a "IBM PC Compatibles - CD" set of DATs which doesn't cover a lot of discs yet.
As I started to dump all of my CDs (for my own sake so far) and therefore I would like to help the project by adding my contribution to it.

So far I can only help with some : IBM PC CD Games, Applications, OSs ; Magasines (and their respective CDs or floppies) ; IBM PC Floppies Games and Applications.

However I'm very unfamiliar with the methods you use and as the wiki seems to be contantly down I currently have no idea how to start and help you.

Also, it seems that you mostly use Windows compatible softwares only in order to make dumps and manage the DATs files ; So as I'm a Linux only (Arch Linux x86_64 fyi) I wanted to know if Windows was needed in order to participate to the project (in general, but mostly in the IBM PC CD ripping area).

Thank you all in advance for your answers!

[Neon Vincent]

I see that this post is over 6 months old with no replies, so I sure hope somebody helped in PMs because I'm in the same position.

I've read about the TOSEC project but it's just a bit daunting for a total newbie.  Exactly where do I start?  How does it work?  What are the .dat files for?  I see them requested and posted in alt.binaries.emulators.tosec but it's all a bit much for a total newbie.

All I want is C64, but I want *everything* C64.  I will also have a few things to contribute occasionally, and I'd like to properly repost stuff I find there when I see requests and I have the fills.

I can't find a "Total newbie starter guide" here in the forums.  Please help!

Thanks.

[PandMonium]

Hey there and welcome here.

I'm just leaving now but will try to give you a proper answer later.
My first suggestion if you're completely new to the project is to take a look at Wikipedia entry which is in part maintained by some of us around here and read the bits of information on the main page, namely a brief overview of the naming convention.

Although we have tried to improve it over the last years, you're right that we still lack a few more guides and posts to explain how a new member can contribute.
Will try to answer this night or tomorrow morning.

[guigui69]

as pandemonium said, wikipedia is a good guide. i used it some years ago when i renamed some C64 demos (a few because i was lazy !
Tauchy helped me too to explain some details. Welcome vincent !

[PandMonium]

I've a lot of work to end before leaving for a week so my answer will still be brief.

What is TOSEC and how is it useful to other people?
An initial idea of what TOSEC is can be obtained by reading wikipedia entry and the small description in the site ( http://www.tosecdev.org/index.php/the-project ).

Basically TOSEC began as a software cataloging initiative to identify each software image (rom files) that floats around. To this end, setfiles (usually this means the zip (container) that contains the rom (or roms) file are renamed according to our naming convention ( http://www.tosecdev.org/index.php/tosec-naming-convention ). This naming convention consists of a set of rules to organize information such as software title, date, publisher and many other flags.

In addition to this, datfiles (the .dat you see) are created to make it possible to identify the files. We have hundreds of datfiles which normally follow the organization: <company name> <system name> <categories>{1+}. An example of this is for instance the datfile: "Commodore Amiga - Applications - Public Domain - [ADF]", which contains information about public domain applications, in the ADF file format for commodore amiga.
The dats contain entries about the set/rom files (file name and size) together with hashes (a string/code) that are used to identify each file (they can be seen as an "unique" code (of 8, 32 or 40 chars depending on the hash) is calculated from the file content.

Summing this all up, these dats containing the information we gather can be used with specific tools (called rom managers - examples are clrmame pro, romvault and romulus) to automatically identify and rename files. Imagine you have a folder full of roms you don't know exactly what they are, use the tool and the datfiles and the files are organized and renamed properly. This helps people knowing exactly which software do they own, for instance if the rom they have is broken (was dumped badly) - this means we don't catalog only perfect software but hacks and bad dumps that are common so anyone can identify them too.


How can someone contribute?
Most of our members' work is focused in identifying and renaming roms. Since TOSEC already catalogs lots of it (especially in the main branch), we have been trying to improve the quality of data by reorganizing the dats structure and properly rename the existing sets, since many miss a lot of information. In addition to this, new sets are also added to the existent dats and and also "new" (doesn't necessarily means more recent) systems and dats are created. We have a few more ideas and details but this is the main idea.

So, when members have time to contribute, they go through their dats (each member works on a specific group of dats or systems) and reinspects the existing sets for errors and missing information (most of the time by launching the sets in emulators). The same is done with new roms that exist but have not been added to TOSEC. After this, new dats are generated and released after a while.


People without the time or experience may still contribute in several forms - you may see some examples around the forum, not everyone wants to rename things or has the time to make it frequently.
Some possible contributions are:
- submitting new / interesting files that are yet to be identified
- pointing out some errors that they found out in some datfiles / rom names (an example of this is amiga contributions at EAB forum and Kodoichi around here).
- fixing some dats that are not currently maintained or contributing with small fixes to some sets.

Just a small note about the two other branches:
- TOSEC-ISO is a side project with the goal of preserving / dumping optical media discs (CDs, DVDs, and so on), which then are added to dats (e.g. dats for Dreamcast discs). One of the main differences in disc based dats is that they contain only original media dumps.
- TOSEC-PIX is a similar project to TOSEC but instead of software, the goal is catalog scans of documents (manuals, magazines and so on). Aral is the guy behind it.


That's it, hopefully this sounded clear enough. Feel free to ask more.

[Neon Vincent]

Thanks guigui69!

PandMonium, I read both pages you recommended but I'm still not getting it.  I understand the project goals, and I understand the importance of learning and sticking to the naming convention, but I don't understand precisely where to begin.

1: Download and install this piece of software.
2: Download this file and open it with that software.
3: Begin a directory structure on your hard drive that looks like this: (whatever). 
4: Step 4
5: Step 5
and so forth.

What I'm asking is, *exactly* how do I get started?  Downloading material, and sharing my own material.

Please explain as you would to a sixth grader

Thanks for the welcome and for your time replying.

Ray

[guigui69]

as i remember, some years ago, someone of TOSEC sent me a bunch of files to rename (a big RAR with a bunch of C64 demos not renamed at all) . so i used an emulator & verified each rom one by one. i launched the demo, i take informations about it on the demo itself (it was for C64 demos), (i had fun to read texts of the demomaker), i quit the emulator, & i rename the file, relating to the name convention.

For details, i asked a little & after that was automatic for me. it's like an archivist work. If you like C64, it's like a pleasure to take time to see programs.

thic topic, the naming is well explained for details :
http://www.tosecdev.org/index.php/forum/index.php?topic=318.0

the first thing to do is already to get the C64 complete TOSEC at this moment, & as pandemonium said, respect the naming convention currently used for the C64 dat.

As i said, you must keep pleasure to launch programs & be curious because it's fast boring to "rename only". 

[PandMonium]

Unfortunately i'm leaving now and not sure if i will have an internet connection during the next days (vacations) so a better explanation will have to wait...

Anyway, from a user perspective:
0) imagine you have a bunch of romfiles in some folders and which to use TOSEC to identify / organize them...
1) Get a rom manager tool such as clrmame pro, romvault or romulus. To someone without previous experience and using tosec maybe romvault is the better option
2) Get the datfile packs we have here, they are like the "databases" with information to identify (hashes) and rename (filenames) for each rom.
3) Load the datfiles in the rom manager tool and start scanning your roms or rebuilding them to a new destination (which means scan a folder and move the identified ones to the right place).

I guess this is basically it. Unidentified files may mean they are not yet added by us but forget this part for now and first try to get used to the dats/rom manager thing.

When we do a new release, it means we updated some of our datfiles and you need to get a new copy of them and update your collection.

A good video covering some of these points and rom vault can be found here : http://www.youtube.com/watch?v=yUOIYYbZuAg . It might be a good starting point.

[Vaxalon]

Looking back on this, a thought occurs. Does it all seem abit complicated? Well, speaking from my point of view, just start renaming some of the games out there that are not in TOSEC as yet. CLRmame Pro scan with our dats should remove the ones that are already done. (not that a fair few dont need improvement).
From my point of view...just rename some games! look at ours to see roughly how its done.
Rename them, get them on the net, sooner or later we will have them, post direct or..torrent.whatever really.
The real trick is making sure the ones you have, have NOT been done. er..by DONE, i mean..an exact duplicate file, any new version of an existing game would have its own checksum, so it would not be a duplicate.