For any of the big, multi-file [Various] DATs, a duplicate check isn't particularly useful at the moment - there are a lot of shared files, empty files etc.
Until we have a process for cleaning these up wholesale, there's not an awful lot of point picking at the edges of the problem.
I'm not sure what the deal is with the ZX Spectrum Pokes - hopefully somebody with more knowledge of the format can explain the large number of dupes there.