Find & remove files that are duplicates but with different extensions recursively

for f in `comm -1 -2 <(sort <(find contrib/image/ -name *.png | sed 's/\.[^.]*$//')) <(sort <(find contrib/image/ -name *.jpg | sed 's/\.[^.]*$//'))`;do rm "$f.png" && echo "deleted: $f.png";done
Very useful to get rid of backup files or wrong extension files that lure in your folders In this example first I do two searches for all filenames of the two extensions .jpg and .png, then delete the extension and only output the now duplicate files. I loop with these results and print a log and delete the file with the extension I dislike.
Sample Output
deleted: contrib/image/4legs/cat.png
deleted: contrib/image/4legs/dog.png
deleted: contrib/image/withwings/fly.png

0
2017-11-28 15:48:15

These Might Interest You

  • Using awk, find duplicates in a file without sorting, which reorders the contents. awk will not reorder them, and still find and remove duplicates which you can then redirect into another file.


    73
    awk '!x[$0]++' <file>
    din7 · 2009-12-20 02:33:21 8
  • find files recursively from the current directory, and list the extensions of files uniquely Show Sample Output


    0
    find . -type f |egrep '^./.*\.' |sed -e "s/\(^.*\.\)\(.*$\)/\2/" |sort |uniq
    dvst · 2012-11-12 17:17:55 0
  • From the cwd, recursively find all rar files, extracting each rar into the directory where it was found, rather than cwd. A nice time saver if you've used wget or similar to mirror something, where each sub dir contains an rar archive. Its likely this can be tuned to work with multi-part archives where all parts use ambiguous .rar extensions but I didn't test this. Perhaps unrar would handle this gracefully anyway?


    9
    find . -name '*.rar' -execdir unrar e {} \;
    kyle0r · 2012-09-27 02:27:03 0
  • This assumes you have the package installed necessary for converting WMF files. On my Ubuntu box, this is libwmf-bin. I used this command, as libwmf is not on my wife's iMac, so I archived the directories containing the WMF files from OS X, ran them on my Ubuntu box, archived the resulting SVGs, and sent them back to her. Quick, simple and to the point. Searches directories recursively looking for extensions ignoring case. This is much more readable and clean than -exec for find. The while loop also gives further flexibility on complex logic. Also, although there is 'wmf2svg --auto', it expects lowercase extensions, and not uppercase. Because I want to ignore case, I need to use the -o option instead. Works in ZSH and BASH. Haven't tested in other shells.


    3
    find . -type f -iname '*.wmf' | while read FILE; do FILENAME="${FILE%.*}"; wmf2svg -o ${FILENAME}.svg $FILE; done
    atoponce · 2009-03-07 22:21:01 2

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands



Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: