Commands using sort (800)

  • Sort and count subjects of emails stuck in Exim queue


    1
    grep -R Subject /var/spool/exim/input/ | sed s/^.*Subject:\ // | sort | uniq -c | sort -n > ~/email_sort.$(date +%m.%d.%y).txt
    unixmonkey27702 · 2011-11-24 14:13:29 3
  • Show total size of each subdirectory, broken down by KB,MB,GB,TB


    1
    du --max-depth=1 | sort -nr | awk ' BEGIN { split("KB,MB,GB,TB", Units, ","); } { u = 1; while ($1 >= 1024) { $1 = $1 / 1024; u += 1 } $1 = sprintf("%.1f %s", $1, Units[u]); print $0; } '
    threv · 2011-12-08 17:43:09 4
  • One line Perl Script to determine the largest file sizes on a Linux Server

    This one line Perl script will display the smallest to the largest files sizes in all directories on a server. Show Sample Output


    1
    du -k | sort -n | perl -ne 'if ( /^(\d+)\s+(.*$)/){$l=log($1+.1);$m=int($l/log(1024)); printf ("%6.1f\t%s\t%25s %s\n",($1/(2**(10*$m))),(("K","M","G","T","P")[$m]),"*"x (1.5*$l),$2);}' | more
    Q_Element · 2012-02-07 15:49:19 10
  • Displays user-defined ps output and pidstat output about the top CPU or MEMory users.

    It grabs the PID's top resource users with $(ps -eo pid,pmem,pcpu| sort -k 3 -r|grep -v PID|head -10) The sort -k is sorting by the third field which would be CPU. Change this to 2 and it will sort accordingly. The rest of the command is just using diff to display the output of 2 commands side-by-side (-y flag) I chose some good ones for ps. pidstat comes with the sysstat package(sar, mpstat, iostat, pidstat) so if you don't have it, you should. I might should take off the timestamp... :| Show Sample Output


    1
    for i in $(ps -eo pid,pmem,pcpu| sort -k 3 -r|grep -v PID|head -10|awk '{print $1}');do diff -yw <(pidstat -p $i|grep -v Linux) <(ps -o euser,pri,psr,pmem,stat -p $i|tail);done
    bob_the_builder · 2012-02-16 20:54:32 23
  • Top 20 commands in your bash history


    1
    sed -e 's/[;|][[:space:]]*/\n/g' .bash_history | cut --delimiter=' ' --fields=1 | sort | uniq --count | sort --numeric-sort --reverse | head --lines=20
    WissenForscher · 2012-02-17 23:34:16 5
  • List files in directory tree with newest last


    1
    find <directory> -type f -printf "%T@\t%p\n"|sort -n|cut -f2|xargs ls -lrt
    rik · 2012-03-02 12:51:06 3
  • identify NEEDED sonames in a path

    This works in combination with http://www.commandlinefu.com/commands/view/10496/identify-exported-sonames-in-a-path as it reports the NEEDED entries present in the files within a given path. You can then compare it with the libraries that are exported to make sure that, when cross-building a firmware image, you're not bringing in dependencies from the build host. The short version of it as can be seen in the same output is scanelf -RBnq -F "+n#f" $1 | tr ',' '\n' | sort -u Show Sample Output


    1
    scanelf --nobanner --recursive --quiet --needed --format "+n#F" $1 | tr ',' '\n' | sort -u
    Flameeyes · 2012-03-29 18:30:45 3
  • Recursively find top 20 largest files (> 1MB) sort human readable format

    from my bashrc ;)


    1
    find . -mount -type f -printf "%k %p\n" | sort -rg | cut -d \ -f 2- | xargs -I {} du -sh {} | less
    bashrc · 2012-03-30 07:37:52 3
  • List all files in current dir and subdirs sorted by size

    or tree -ifsF --noreport .|sort -n -k2|grep -v '/$' (rows presenting directory names become hidden)


    1
    tree -ifs --noreport .|sort -n -k2
    knoppix5 · 2012-05-04 09:18:39 6
  • Top 15 processes with the largest number of open files

    See the summary. Show Sample Output


    1
    lsof +c 15 | awk '{print $1}' | sort | uniq -c | sort -rn | head
    SEJeff · 2012-05-25 16:31:46 3
  • du and sort to find the biggest directories in defined filesystem

    This command give a human readable result without messing up the sorting.


    1
    for i in G M K; do du -hx /var/ | grep [0-9]$i | sort -nr -k 1; done | less
    jlaunay · 2012-06-26 22:57:17 6
  • Read a tcpdump file and count SYN packets to port 80, Order column by destination.


    1
    tcpdump -ntr NAME_OF_CAPTURED_FILE.pcap 'tcp[13] = 0x02 and dst port 80' | awk '{print $4}' | tr . ' ' | awk '{print $1"."$2"."$3"."$4}' | sort | uniq -c | awk ' {print $2 "\t" $1 }'
    efuoax · 2012-08-22 21:26:10 7
  • Display IP : Count of failed login attempts

    The lastb command presents you with the history of failed login attempts (stored in /var/log/btmp). The reference file is read/write by root only by default. This can be quite an exhaustive list with lots of bots hammering away at your machine. Sometimes it is more important to see the scale of things, or in this case the volume of failed logins tied to each source IP. The awk statement determines if the 3rd element is an IP address, and if so increments the running count of failed login attempts associated with it. When done it prints the IP and count. The sort statement sorts numerically (-n) by column 3 (-k 3), so you can see the most aggressive sources of login attempts. Note that the ':' character is the 2nd column, and that the -n and -k can be combined to -nk. Please be aware that the btmp file will contain every instance of a failed login unless explicitly rolled over. It should be safe to delete/archive this file after you've processed it. Show Sample Output


    1
    sudo lastb | awk '{if ($3 ~ /([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}/)a[$3] = a[$3]+1} END {for (i in a){print i " : " a[i]}}' | sort -nk 3
    sgowie · 2012-09-11 14:51:10 4
  • Find the 10 users that take up the most disk space

    In OSX you would have to make sure that you "sudo -s" your way to happiness since it will give a few "Permission denied" errors before finally spitting out the results. In OSX the directory structure has to start with the "Users" Directory then it will recursively perform the operation. Your Lord and master, Mematron Show Sample Output


    1
    sudo -s du -sm /Users/* | sort -nr | head -n 10
    mematron · 2012-09-13 10:15:23 4
  • Get top N files in X directory

    You can simply run "largest", and list the top 10 files/directories in ./, or you can pass two parameters, the first being the directory, the 2nd being the limit of files to display. Best off putting this in your bashrc or bash_profile file Show Sample Output


    1
    largest() { dir=${1:-"./"}; count=${2:-"10"}; echo "Getting top $count largest files in $dir"; du -sx "$dir/"* | sort -nk 1 | tail -n $count | cut -f2 | xargs -I file du -shx file; }
    jhyland87 · 2013-01-21 09:45:21 7
  • Shows size of dirs and files, hidden or not, sorted.

    Enhanced version: fixes sorting by human readable numbers, and filters out non MB or GB entries that have a G or an M in their name.


    1
    du --max-depth=1 -h * |sort -h -k 1 |egrep '(M|G)\s'
    TerDale · 2013-02-14 08:56:56 6
  • (DEBIAN-BASED DISTROS) Find total installed size of packages given a search term

    Replace \-dev with whatever you wanna search for


    1
    dpkg-query -Wf '${Installed-Size}\t${Package}\n' | grep "\-dev" | sort -n | awk '{ sum+=$1} END {print sum/1024 "MB"}'
    threv · 2013-02-15 20:29:18 4
  • Display 16 largest installed RPMs in size order, largest first

    Interesting to see which packages are larger than the kernel package. Useful to understand which RPMs might be candidates to remove if drive space is restricted. Show Sample Output


    1
    rpm -qa --queryformat '%{size} %{name}-%{version}-%{release}\n' | sort -k 1,1 -rn | nl | head -16
    mpb · 2013-03-19 21:10:54 6
  • Display unique values of a column

    the column number is '6'


    1
    cut -d',' -f6 file.csv | sort | uniq
    richie · 2013-04-10 14:05:32 6
  • Prints total line count contribution per user for an SVN repository

    make usable on OSX with filenames containing spaces. note: will still break if filenames contain newlines... possible, but who does that?!


    1
    svn ls -R | egrep -v -e "\/$" | tr '\n' '\0' | xargs -0 svn blame | awk '{print $2}' | sort | uniq -c | sort -nr
    rymo · 2013-04-10 19:37:53 5
  • Mapreduce style processing

    parallel can be installed on your central node and can be used to run a command multiple times. In this example, multiple ssh connections are used to run commands. (-j is the number of jobs to run at the same time). The result can then be piped to commands to perform the "reduce" stage. (sort then uniq in this example). This example assumes "keyless ssh login" has been set up between the central node and all machines in the cluster. bashreduce may also do what you want. Show Sample Output


    1
    parallel -j 50 ssh {} "ls" ::: host1 host2 hostn | sort | uniq -c
    macoda · 2013-04-12 11:56:41 7
  • most used unix commands


    1
    awk '{print $1}' ~/.bash_history | sort | uniq -c | sort -rn | head -n 10
    nesses · 2013-05-03 16:24:30 6
  • directory size with subdirectories, sorted list


    1
    du -m --max-depth=1 [DIR] | sort -nr
    Paulus · 2013-07-26 07:04:41 12
  • List your largest installed packages (on Debian/Ubuntu)

    The other commands were good, but they included packages that were installed and then removed. This command only shows packages that are currently installed, sorts smallest to largest, and formats the sizes to be human readable. Show Sample Output


    1
    dpkg-query --show --showformat='${Package;-50}\t${Installed-Size}\n' `aptitude --display-format '%p' search '?installed!?automatic'` | sort -k 2 -n | grep -v deinstall | awk '{printf "%.3f MB \t %s\n", $2/(1024), $1}'
    EvilDennisR · 2013-07-26 23:18:20 13
  • Diff two directories by finding and comparing the md5 checksums of their contents.

    Compute the md5 checksums for the contents of two mirrored directories, then sort and diff the results. If everything matches, nothing is returned. Otherwise, any checksums which do not match, or which exist in one tree but not the other, are returned. As you might imagine, the output is useful only if no errors are found, because only the checksums, not filenames, are returned. I hope to address this, or that someone else will!


    1
    diff <(sort <(md5deep -r /directory/1/) |cut -f1 -d' ') <(sort <(md5deep -r /directory/2/) |cut -f1 -d' ')
    unixmonkey64021 · 2013-08-18 22:13:07 6
  • ‹ First  < 12 13 14 15 16 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

recursive search and replace old with new string, inside files
Using -Z with grep and -0 with xargs handles file names with spaces and special characters.

Use find to get around Argument list too long problem
Can be used for other commands as well, replace rm with ls. It is easy to make this shorter but if the filenames involved have spaces, you will need to do use find's "-print0" option in conjunction with xargs's "-0" option. Otherwise the shell that xargs uses to execute the "rm" command line will treat the space as a token separator, thereby treating the name as two (or more) names.

Convert CSV to JSON
Replace 'csv_file.csv' with your filename.

Clear current session history (bash)

Sort all running processes by their memory & CPU usage
you can also pipe it to "tail" command to show 10 most memory using processes.

Backup with versioning

BourneShell: Go to previous directory
cd - would return to the previous directory of your cd command. NB: previous dir is always stored in $OLDPWD variable.

Copy a file using dd and watch its progress
This is a more accurate way to watch the progress of a dd process. The $DDPID=$! is needed so that you don't get the PID of the sleep. The sleep 1 is needed because in my testing at least, if you run kill -USR1 against dd too quickly, it will kill it off instead of display the status. So you need to wait a second, probably so that it can configure itself to trap the USR1 signal.

Create arbitrary big file full of zeroes but done in a second
If you want to create fast a very big file for testing purposes and you do not care about its content, then you can use this command to create a file of arbitrary size within less than a second. Content of file will be all zero bytes. The trick is that the content is just not written to the disk, instead the space for it is somehow reserved on operating system level and file system level. It would be filled when first accessed/written (not sure about the mechanism that lies below, but it makes the file creation super fast). Instead of '1G' as in the example, you could use other modifiers like 200K for kilobytes (1024 bytes), 500M for megabytes (1024 * 1024 bytes), 20G for Gigabytes (1024*1024*1024 bytes), 30T for Terabytes (1024^4 bytes). Also P for Penta, etc... Command tested under Linux.

Convert all Flac in a directory to Mp3 using maximum quality variable bitrate

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: