Best Application for Deleting/Separating Multiple files on macOS

[FRVRandAFTR]

Hello to all you experienced MacRumors members!

I have built over 10 years several files that are duplicates. Ideally, I would like a program that would allow me to purge all, but one.

I know Apple has the new feature with Photos that actually seems to work wonderfully because it will take the best quality one by default and add in, to my understanding, modifications that may have been edited out from photo prior. Per my understanding, it's pixel perfect two so if there are two that look very similar, it will not delete one that might look almost identical.

So what I'm asking, you very kind, smart people for our recommendations. You personally use for such applications that will do this on a large scale. I have so many files and folders that are similar, but not quite the exact same I would like to be able to compare the two before choosing which one to delete or separate those into a different folder, so I can sort them later. I have googled this and the reason I ask you all as community is I would like based on personal preference, what you prefer as well as advise on what may be to avoid or stay away from?

I know there are a lot of choices, or seem like at least a decent amount, but I would like the best, assuming because it wasn't an issue.

Thank you so much!

 

[Slartibart]

The easiest way to do this - IMHO - is using fdupes in the terminal.

1. Install it via home brew (brew install fdupes)
2. Then open a terminal and run e.g. fdupes -r . to find duplicate files recursively under the current folder.

• Add -d to delete the duplicates - you'll be prompted which files to keep.
• If instead you add -dN, fdupes will always keep the first file and directly delete the other files without prompting.

EDIT: fdupes -rdN . will recursively search through the current folder and its subfolders, keep the first file of the duplicates found, and delete the others.

 

[FRVRandAFTR]

Thank you for that! I'll give it a shot and see how it works for me.

Any idea how it works with pictures like the Photo update where higher image quality and metadata (and other things I assume) can combine all qualities for the best file outcome?

Is this a more focused approach for exact copies of data? Ex: Two identical Word documents with slight punctuation differences?

I hear Homebrew mentioned a lot but haven't really researched what it does and can do.

 

[Slartibart]

There are various tools available to do what you want, e.g. hash/checksum matching or combination of methods. Many come from the *nix-world, where these kind of problems are solved for quite some time, often involving the use of a shell (“terminal” in Apple speech 🤓) where you run some form of script. Home Brew offers convenience in installing some these solutions.

You might run into problems when data files are not easily accessible on file system level e.g. when they are stored in a database or some form of “container” like Apple’s Photos can do. Otherwise, like in Apple solutions for Photos, you are the one who does the final judgement.

Let’s assume you have some folder called Documents with a lot of subfolder somewhere. After installation of fdupes via brew, open a terminal. type cd (cd followed by a space) and drag and drop the Documents-folder icon on the open terminal window. The line in the terminal window should read cd /the/path/to/Documents (“/the/path/to”-part depends on your system). Press return. You (c)hanged into the (d)irectory Documents.

If you now execute fdupes -r . >list-of-duplicate-files.txt in the terminal, you’ll find a text file named list-of-duplicate-files.txt inside Documents which list all duplicates of any kind of file in the folder and its subfolders. You can use this to inspect the files e.g. in the finder, or just get an overview before running the command with the -d option.

But this is a solution for duplicates, and not similar files of any kind.

If you want to find visually similar JPEG, PNG or Gif images and delete all but the largest by file size, I suggest to use Delete Visually Redundant or, if you prefer a graphical user interface, Geequie.


nota bene: other solution are indeed available. 😄

 

[Slartibart]

Another excellent solution to find similar images in the file system is findimagedupes.