You can do the filtering natively in the aws cli, without using jq (although jq is awesome!) Show Sample Output
Finds duplicates based on MD5 sum. Compares only files with the same size. Performance improvements on:
find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate
The new version takes around 3 seconds where the old version took around 17 minutes. The bottle neck in the old command was the second find. It searches for the files with the specified file size. The new version keeps the file path and size from the beginning.
Tested on MacOS and GNU/Linux. It works in dirs containing files starting with '-'. It runs 'du' only once. It sorts according to size. It treats 1K=1000 (and not 1024) Show Sample Output
Same result as with 'du -ks .[^.]* * | sort -n' but with size outputs in human readable format (e.g., 1K 234M 2G)
Access a random news web page on the internet. The Links browser can of course be replaced by Firefox or any modern graphical web browser.
This command line detect ldap hosts, by mandatory dns entry, then ping them to detect response average. based on ping response average it sorts and print the faster server in first output line Show Sample Output
In this case searches for where .desktop files are stored. The resulted is a sorted list of the top directories containing such files. Show Sample Output
You can append these commands to the bottom of the history file to access them easier with the Up key:
sort ~/.bash_history|uniq -c|sort -n|tail -n 10|tr -s " "|cut -d' ' -f3- >> ~/.bash_history
Same, without modules... Probably smarter option: just use the shuf command or even sort -R. Show Sample Output
Top 10 Memory Processes (reduced output to applications and %usage only) Show Sample Output
Sort disk usage from directories in the current directory Show Sample Output
This assumes your mail log is /var/log/mail.log Show Sample Output
Displays the duplicated lines in a file and their occuring frequency.
This command puts all the flags of the USE variable actually used by the packages you emerged to the file "use", and those which are unused but available to the file "notuse"
This prints a summary of your referers from your logs as long as they occurred a certain number of times (in this case 500). The grep command excludes the terms, I add this in to remove results Im not interested in. Show Sample Output
This works on my ubuntu/debian machines. I suspect other distros need some tweaking of sort and cut. I am sure someone could provide a shorter/faster version.
On a Gentoo system, this command will tell you which packets you have installed and sort them by how much space they consume. Good for finding out space-hogs when tidying up disk space. Show Sample Output
Sorts a character string, using common shell commands. Show Sample Output
commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.
Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.
» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10
Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):
Subscribe to the feed for: