File management is a complicated task in and of itself. When you add that large volume of duplicate files that normally take up the disk space, the process becomes more difficult.

While the standard way to deal with duplicate files is to manually find and delete them. However, using a dedicated program to find duplicate files can speed up the process significantly.

So, if your plan is to get rid of duplicate files and clean up your computer, here is a list of some of the best tools to find and remove duplicate files in Linux.

fslint duplicate file remover

Fslint is a GUI and CLI based utility for cleaning up various types of clutter on your system. It calls this clutter "lint," and offers several tools to help you complete a variety of tasks, including finding duplicate files, empty directories, and problematic file names.

By offering both graphical and command line modes of operation, fslint makes it easy for new Linux users to rid their computer memory of all kinds of system glitches.

To access fslint through the GUI, all you have to do is open the terminal and the fslint-gui Command.

As for the extended functionality, the program offers 10 different functionalities in CLI mode such as findup, findu8, findnl, findtf and found. These can be used to refine the search results to increase your chances of finding certain types of duplicate files on your system.

How to install fslint

On Debian-based distributions like Ubuntu:

sudo apt install fslint

On RHEL-based distributions like CentOS and Fedora:

sudo yum install fslint
sudo dnf install fslint

On Arch Linux and Manjaro:

sudo pacman -S fslint

fdupes under linux

Fdupes is one of the easiest programs to identify and delete duplicate files in directories. It was published on GitHub under the MIT license and is free and open source.

The program uses the md5sum signature and byte-by-byte comparison checking to identify duplicate files in a directory. If necessary, you can also perform recursive searches, filter out search results and get a summarized view of the duplicate files found.

Once you've identified duplicate files in a directory, you can use fdupes to either delete the files or replace them with links to the original file.

Fdupes installation

On Debian-based distributions:

sudo apt install fdupes

On RHEL-based distributions:

sudo yum install fdupes
sudo dnf install fdupes

To install on Arch Linux and Manjaro:

sudo pacman -S fdupes

Related: How to Find and Remove Duplicate Files on Linux Using fdupes

Using rdfind to remove duplicate files

Rdfind is another Linux utility that can be used to find redundant files on your computer in different directories. It relies on comparing files by their content – rather than their name – to identify duplicates, which makes the job more effective.

To achieve this, the program works by arranging identical files in a directory and determining the original and duplicates: the highest-ranking is selected as the original, while the rest are duplicates.

If necessary, rdfind can also calculate checksums in order to compare files. And the best part is that the scanned results are rolled into one Results.txt File in the home directory so that when you delete duplicates you can refer to it to make sure you don't remove the wrong ones.

Of course, like most other file finders for duplicates, rdfind also offers some preprocessors to sort files, ignore empty files or set symbolic links. Last but not least, there is an option to delete duplicate files.

Related: What is a symbolic link (symlink)? How to create one on Linux

How to install rdfind

Under Debian / Ubuntu:

sudo apt install rdfind

On Fedora / CentOS:

sudo dnf install rdfind

Dupeguru runs on Linux

DupeGuru is a cross-platform tool for finding and deleting duplicate files on your computer. One of its best features is the ability to customize the matching engine to suit your preferences to increase your chances of finding the right kind of duplicate files in a directory. And similar to some other duplicate search programs, it also has a GUI to make it easier to use.

Speaking of functionality: dupeGuru uses its fuzzy matching algorithm to scan either file names or file contents and to find duplicates quickly and efficiently.

It's also good at handling music and image-specific information, which gives it an edge over other duplicate file finders. In addition, if necessary, you have the option to tweak the matching engine to find the exact type of duplicate files you want to remove.

You can also delete duplicate files with DupeGuru. And for that, it has a reference directory system that prevents you from accidentally deleting the wrong files. In addition to deleting them, there is also the option of moving them to another location or copying them.

DupeGuru installation

On Debian-based distributions:

sudo add-apt-repository ppa: dupeguru / ppa
sudo apt-get update
sudo apt-get install dupeguru

Under Arch Linux:

sudo pacman -S dupeguru

rmlint on Linux

Rmlint is another lint – and not just duplicate files – finder and remover for Linux. It's free to use and extremely fast in identifying duplicate files and directories on your system. You also get support for the Btrfs storage format, which sets it apart from other tools on this list.

Speaking of which, some of the other aspects where rmlint trumps the other competing duplicate file removal tools include the ability to search for files based on a specific timeframe, find files with corrupted user / group IDs, and unstripped binaries find that take up a lot of space. In addition, similar to some other programs, it also saves the scanned results rmlint.json and rmlint.sh Files that come in handy during the deletion process.

Note, however, that unlike other tools, rmlint is not the easiest to use: it generates a duplicate deletion script that requires some understanding to be used effectively.

How to install rmlint

On Debian-based distributions:

sudo apt install rmlint

On Fedora and CentOS:

sudo yum install rmlint
sudo dnf install rmlint

On Arch-based distributions like Manjaro:

sudo pacman -S rmlint

Keep duplicate files at bay on Linux

Using the duplicate files search programs listed above, you can easily identify and completely remove the duplicate files that may be taking up space on your computer. However, one piece of advice when working with such tools is to be extra careful with your actions to avoid deleting important files and documents on your system.

If you are a little skeptical about which files to delete and which to keep, you should make a backup copy of all the data on your system to be on the safe side.

Linux backup tools

The top 8 file backup apps for Linux

Regular data backups are crucial, whether it's a server or a local machine. Check out these eight file backup applications for Linux.

Continue reading

About the author

Yash Wate
(23 articles published)

Yash is Staff Writer at MUO for DIY, Linux, programming and security. Before he discovered his passion for writing, he developed for the web and iOS. You can also find his writing on TechPP where he covers other industries. Aside from technology, he likes to talk about astronomy, Formula 1 and clocks.

More
By Yash Wate

Subscribe to our newsletter

Subscribe to our newsletter for tech tips, reviews, free e-books, and exclusive offers!

Click here to subscribe