Commands by alafrosty (1)

  • Find Duplicate Files (based on size first, then MD5 hash)

    * Find all file sizes and file names from the current directory down (replace "." with a target directory as needed). * sort the file sizes in numeric order * List only the duplicated file sizes * drop the file sizes so there are simply a list of files (retain order) * calculate md5sums on all of the files * replace the first instance of two spaces (md5sum output) with a \0 * drop the unique md5sums so only duplicate files remain listed * Use AWK to aggregate identical files on one line. * Remove the blank line from the beginning (This was done more efficiently by putting another "IF" into the AWK command, but then the whole line exceeded the 255 char limit). >>>> Each output line contains the md5sum and then all of the files that have that identical md5sum. All fields are \0 delimited. All records are \n delimited.


    0
    find . -type f -not -empty -printf "%-25s%p\n"|sort -n|uniq -D -w25|cut -b26-|xargs -d"\n" -n1 md5sum|sed "s/ /\x0/"|uniq -D -w32|awk -F"\0" 'BEGIN{l="";}{if(l!=$1||l==""){printf "\n%s\0",$1}printf "\0%s",$2;l=$1}END{printf "\n"}'|sed "/^$/d"
    alafrosty · 2013-10-22 13:34:19 7

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

delete command line last word

Emulating netcat -e (netcat-traditional or netcat-openbsd) with the gnu-netcat
Then just nc servername 2600 and ./script.sh kill the client with ctrl+c. You can reconnect several times. kill the server with exit

Get each users commit amount

Use tee to process a pipe with two or more processes
Tee can be used to split a pipe into multiple streams for one or more process to work it. You can add more " >()" for even more fun.

invoke MATLAB functions from command line
`-r script.m` also possible

Get a BOFH excuse
Gets a BOFH excuse from the BOFH excuse server (towel.blinkenlights.nl port 666), and passes it through sed and tr to get rid of telnet connection stuff.

Recursively lists all files in the current directory, except the ones in '.snapshot' directory
This can be useful for those who have mounted NetApp file-systems with snapshot activated.

Python version 3: Serve current directory tree at http://$HOSTNAME:8000/
In Python version 3, the module was merged into http.server. Gentlemen, change your aliases.

Set laptop display brightness
Run as root. Path may vary depending on laptop model and video card (this was tested on an Acer laptop with ATI HD3200 video). $ cat /proc/acpi/video/VGA/LCD/brightness to discover the possible values for your display.

Set access and modification timestamps of a file using another one as reference
atime and mtime timestamps of $FILE2 is changed according to the ones of $FILE1. If $FILE2 doesn't exist is created.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: