What to do about LONG filenames

[NinjaRygar]

What do you do with filenames like this?!

"King of Fighters '98, The - The Slugfest (1998)(SNK)(JP)(Track 01 of 40)[!][King of Fighters '98, The - Dream Match Never Ends][NGCD-2420 MT A04, NGCD-2421].iso"

That's 160 characters, on it's own, that's ok.  HOWEVER, windows can only handle filenames that are 260 characters long, with the whole path included.  Now, in order to scan a set like this in clrmamepro, it needs to be in a directory.  So....

"C:\NeoGeoCD\King of Fighters '98, The - The Slugfest (1998)(SNK)(JP)[!][King of Fighters '98, The - Dream Match Never Ends][NGCD-2420 MT A04, NGCD-2421]\King of Fighters '98, The - The Slugfest (1998)(SNK)(JP)(Track 01 of 40)[!][King of Fighters '98, The - Dream Match Never Ends][NGCD-2420 MT A04, NGCD-2421].iso"

That's WAY MORE than 260.
I can't move anything, delete anything, copy anything... windows cannot handle filenames that long.  I would like to scan my set with CLRMAMEPRO, but it's not possible with names like this.  What am I supposed to do? Do you guys have a tool, a utility, or something?  I'm really at my wits end here... there are programs that claim they can copy long filenames, but none of them are free and I'm not paying 20 dollars for something that stupid.
What do you guys do?  

[PandMonium]

Hi NinjaRygar,

I don't have sets currently so i can't tell you what i 'do' but will try to give some answers.
Unfortunately windows has a limitation of 260chars for paths (256 actually) and we have some really long setnames in our collection because in some cases there are long values for some tnc fields (like a huge title or more info flag, etc).
Said that, the longest one is 181 chars and the following ones are smaller (170, 160, 150...), i'm not saying they are small because it is clearly that they are HUGE anyway unfortunately but they are manageable.

For you description i suppose you aren't using "compressed sets", so instead of having something like:
a) "C:\TOSEC\ISO\NeoGeo CD\Games\King of Fighters '98, The - The Slugfest (1998)(SNK)(JP)[!][King of Fighters '98, The - Dream Match Never Ends][NGCD-2420 MT A04, NGCD-2421].zip"

you have:
b) "C:\TOSEC\ISO\NeoGeo CD\Games\King of Fighters '98, The - The Slugfest (1998)(SNK)(JP)[!][King of Fighters '98, The - Dream Match Never Ends][NGCD-2420 MT A04, NGCD-2421]\King of Fighters '98, The - The Slugfest (1998)(SNK)(JP)(Track 01 of 40)[!][King of Fighters '98, The - Dream Match Never Ends][NGCD-2420 MT A04, NGCD-2421].iso"

"Compressed sets" option in clrmame means you have the sets stored in zips instead of folders, using it the actual length is up to the zip name, if not the romnames also count and you end up with even huge filenames. As you can see in the example, A) length is 174 chars against B) which is 331 (and breaks the limits).
So the solution is using this option and keep the sets in zipfiles, its not the better one i suppose and if you want to play them (and if there is no way to launch them in zip or mount them directly from the zip), then you need to extract those ones to folders with a smaller name

Clrmame Pro might have some troubles with the tmp / backup folders (not sure how it works) cause of these long filename but you have to try by yourself, what i described is what people with those sets use, my best suggestion is for you activate that option in rebuilder, mark remove matched sourcefiles and try to do a rebuild of your collection.

Good luck and let me know if it works, maybe this is something useful to add to a faq or so

[derJörg]

Hi,

...just some background information:
It is a myth that file paths under Windows are limited to ~260 characters.
Actually it depends on the API the developer uses to access file system objects. Developers can use so called "extended-length paths" under Windows which allow path lengths up to 32,768 characters which should be enough for everyone...
Developers should read this and stop complaining about path length restrictions...

Regards,
derJörg

[PandMonium]

Hey derJörg,
Didn't know that page and also didn't understood very well what it means and i'm busy right know to investigate
Anyway thanks for the info but the problem really exists, even if there was an workaround for it, it is a limitation always active in the OS. I'm using WinXP and at least here it is easy to test, i can give you a simple example:
a) You can open a zipfile and rename one of the files inside it to up 250+ chars length. After that you can't extract that file anywhere except to the root of any disk, where the path will less than 260chars.
b) You can do it even worse by renaming a folder inside a zipfile to a really long name and the same to a file inside it (each file or folder may have near 254 chars each in zip files). This way you can't even extract it with the original names to anywhere at least in Windows XP.
c) In case you did a) you will not be able to move the file anywhere because the limit will always be exceeded.

So, even if there is a workaround, the problem is that the default OS behavior doesn't use it.
Anyway long name is not really something good even if they were possible

[derJörg]

Hi Pandmonium,

...the page is for developers. It describes what steps to take to make your applications capable of handling very large paths. There is no possible solution or workaround if you are only a user. It's up to each developer to make his application work like it is expected to work in 2010.

I don't deny that the problem exists, I have a couple of applications that behave the way you describe it (in fact, 95%). It bugs me that not even standard tools like WinRAR or WinZip care about long paths.
However, it is the fault of their developers. It is not a restriction of the OS but a restriction of the particular application, and it has nothing to do with the default OS behavior.

Applications that don't support such long paths are simply using an old legacy (and restricting) method of accessing files, that's why.
You will experience the same effect when you try to access files with unicode symbols in their names. Applications using the old way will fail, applications using the modern approach will succeed.

Again: it is not a matter of the OS or file system, it is a matter of the API the developer of each application uses to access file system objects within the application.

Regards,
derJörg

P.S.: What is even more startling is that not even all of Microsoft's applications support those extended paths... 

[PandMonium]

(i'm very busy so i can't check it out properly)

I will try to explain it briefly, AFAIK the file name length limitation of 255 chars is indeed a limitation of the file system, NTFS does not support filenames longer than that, as for the path length NTFS does have a higher limit of 32000+ chars. Nonetheless, the OS itself has a limit defined (Win32 API uses MAX_PATH of 255 or 260 chars), using it to check length on file operations. That is the reason why you can rename a file with over 250 chars on C:\ root but then you can not move it to any folder because it will give you the too long path error.

Apparently (i've never tried and did not know it ), you can however bypass this limit (for paths length, the filename one is not only on the winAPI) by using the "\\?\D:\<very long path>" as path and go up to 32k chars. From a brief search it seems .NET FileInfo classes for example blocked this so it is unlikely to be useful in C# for instance, however should work in others.

It left me curious but i've no time to test it right know, if that is true and works you could indeed create/move files to very long paths exceeding by far the 260 limit. However since it is limited in the OS itself it would give a lot of problems like making it possible to move those files in explorer, copy somewhere, zip it and any other operations outside the tool that would allow it. In other words it's not worth the troubles, the file system should just support it by default.

Anyway thanks for the tip, i might try it out someday

[derJörg]

Hi Pandmonium,

(i'm very busy so i can't check it out properly)

I will try to explain it briefly, AFAIK the file name length limitation of 255 chars is indeed a limitation of the file system, NTFS does not support filenames longer than that, as for the path length NTFS does have a higher limit of 32000+ chars.

Right.

Nonetheless, the OS itself has a limit defined (Win32 API uses MAX_PATH of 255 or 260 chars), using it to check length on file operations.

Right. That limit affects non-unicode functions and paths without that "\\?\" prefix.

Apparently (i've never tried and did not know it ), you can however bypass this limit (for paths length, the filename one is not only on the winAPI) by using the "\\?\D:\<very long path>" as path and go up to 32k chars.

Right.

It left me curious but i've no time to test it right know, if that is true and works you could indeed create/move files to very long paths exceeding by far the 260 limit.

I already tried it. It works.

However since it is limited in the OS itself it would give a lot of problems like making it possible to move those files in explorer, copy somewhere, zip it and any other operations outside the tool that would allow it. In other words it's not worth the troubles, the file system should just support it by default.

I think this would give a lot of compatibility issues, so I don't expect Microsoft to change that MAX_PATH thing. But anyway: there is a way to do it right, so it's up to the developers to walk that way...

Anyway thanks for the tip, i might try it out someday

Yes, I can only encourage to do it. Everyone interested can download Visual Studio at no cost and try it out him/herself.

Regards,
derJörg

[gregory2015]

I am using Long Path Tool software for such errors, and it works like charm.

[tomse]

The post is about to get old, but as derJörg mentions it's the client software that is the issue.

Even Explorer doesn't handle filenames (full path) longer than 256 chars very well, so even MS doesn't use the long path code.

The filesystem doesn't care though (there's a limit ofc which I don't know)
but if you have a file larger than 256 chars which you can't delete without special tools, you can always rename the dirs and files to something very short, going from the root to the branch.
writing is usually not a problem, but reading is.

you can also use subst.exe to create a new drive letter from a point in the middle of the dir path.

Code: [Select]

subst h: d:\my_very\long_path\with_lots\of_chars