Commands tagged awk (317)

  • This will, for an application that has already been removed but had its configuration left behind, purge that configuration from the system. To test it out first, you can remove the last -y, and it will show you what it will purge without actually doing it. I mean it never hurts to check first, "just in case." ;)


    -3
    dpkg-query -l| grep -v "ii " | grep "rc " | awk '{print $2" "}' | tr -d "\n" | xargs aptitude purge -y
    thepicard · 2009-04-28 19:25:53 3
  • Useful for removes a package and its depends, for example to remove the gnome desktop environment, also configuration files will be removed, you should be carefully and sure that you want to do this. Show Sample Output


    -4
    sudo apt-get remove --purge `dpkg -l | awk '{print $2}' | grep gnome` && apt-get autoremove
    kelevra · 2009-04-28 10:34:42 7
  • This is useful if you have need to do port forwarding and your router doesn't assign static IPs, you can add it to a script in a cron job that checks if you IP as recently changed or with a trigger script. This was tested on Mac OSX.


    9
    ifconfig en1 | awk '/inet / {print $2}' | mail -s "hello world" email@email.com
    rez0r · 2009-04-28 06:01:52 4
  • Lists the local files that are not present in the remote repository (lines beginning with ?) and add them. Show Sample Output


    1
    svn status | grep '^?' | awk '{ print $2; }' | xargs svn add
    unixmonkey4200 · 2009-04-10 21:55:37 2
  • This command will sort the contents of FILENAME by redirecting the output to individual .txt files in which 3rd column will be used for sorting. If FILENAME contents are as follows: foo foo A foo bar bar B bar lorem ipsum A lorem Then two files called A.txt and B.txt will be created and their contents will be: A.txt foo foo A foo lorem ipsum A lorem and B.txt will be bar bar B bar


    2
    awk '{print > $3".txt"}' FILENAME
    alperyilmaz · 2009-03-31 15:14:13 0
  • Here is a command line to run on your server if you think your server is under attack. It prints our a list of open connections to your server and sorts them by amount. BSD Version: netstat -na |awk '{print $5}' |cut -d "." -f1,2,3,4 |sort |uniq -c |sort -nr Show Sample Output


    14
    netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
    tiagofischer · 2009-03-28 21:02:26 4

  • 6
    awk '{sum+=$1; sumsq+=$1*$1} END {print sqrt(sumsq/NR - (sum/NR)**2)}' file.dat
    kaan · 2009-03-24 21:56:40 4
  • The arguments of "seq" indicate the starting value, step size, and the end value of the x-range. "awk" outputs (x, f(x)) pairs and pipes them to "graph", which is part of the "plotutils" package.


    2
    seq 0 0.1 20 | awk '{print $1, cos(0.5*$1)*sin(5*$1)}' | graph -T X
    kaan · 2009-03-24 21:46:59 3
  • Displays six rows and five columns of random numbers between 0 and 1. If you need only one column, you can dispense with the "for" loop. Show Sample Output


    3
    seq 6 | awk '{for(x=1; x<=5; x++) {printf ("%f ", rand())}; printf ("\n")}'
    kaan · 2009-03-24 21:33:38 0
  • This example calculates the averages of column one and column two of "file.dat". It can be easily modified if other columns are to be averaged.


    2
    awk '{sum1+=$1; sum2+=$2} END {print sum1/NR, sum2/NR}' file.dat
    kaan · 2009-03-24 21:22:14 1
  • Another combination of seq and awk. Not very efficient, but sufficiently quick. Show Sample Output


    14
    seq 50| awk 'BEGIN {a=1; b=1} {print a; c=a+b; a=b; b=c}'
    kaan · 2009-03-24 20:39:24 1
  • "seq 100" outputs 1,2,..,100, separated by newlines. awk adds them up and displays the sum. "seq 1 2 11" outputs 1,3,..,11. Variations: 1+3+...+(2n-1) = n^2 seq 1 2 19 | awk '{sum+=$1} END {print sum}' # displays 100 1/2 + 1/4 + ... = 1 seq 10 | awk '{sum+=1/(2**$1)} END {print sum}' # displays 0.999023 Show Sample Output


    4
    seq 100 | awk '{sum+=$1} END {print sum}'
    kaan · 2009-03-24 20:30:40 0
  • Displays the duplicated lines in a file and their occuring frequency.


    1
    cat file.txt | sort | uniq -dc
    Vadi · 2009-03-21 18:15:14 3
  • Show the number of failed tries of login per account. If the user does not exist it is marked with *. Show Sample Output


    23
    sudo zcat /var/log/auth.log.*.gz | awk '/Failed password/&&!/for invalid user/{a[$9]++}/Failed password for invalid user/{a["*" $11]++}END{for (i in a) printf "%6s\t%s\n", a[i], i|"sort -n"}'
    point_to_null · 2009-03-21 06:41:59 2
  • Takes a input file (count.txt) that looks like: 1 2 3 4 5 It will add/sum the first column of numbers.


    19
    cat count.txt | awk '{ sum+=$1} END {print sum}'
    duxklr · 2009-03-16 00:22:13 7
  • This command checks for the number of times when someone has tried to login to your server and failed. If there are a lot, then that user is being targeted on your system and you might want to make sure that user either has remote logins disabled, or has a strong password, or both. If your output has an "invalid" line, it is a summary of all logins from users that don't exist on your system. Show Sample Output


    8
    zgrep "Failed password" /var/log/auth.log* | awk '{print $9}' | sort | uniq -c | sort -nr | less
    dbart · 2009-03-03 13:45:56 4
  • Useful to check DDoS attacks on servers. Show Sample Output


    -2
    netstat -alpn | grep :80 | awk '{print $4}' |awk -F: '{print $(NF-1)}' |sort | uniq -c | sort -n
    nitins · 2009-02-19 04:59:32 0
  • ‹ First  < 11 12 13

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Rename files in batch

top svn committers (without awk)
list top committers (and number of their commits) of svn repository. in this example it counts revisions of current directory.

Create a listing of all possible permissions and their octal representation.

Convert GoogleCL gmail contacts to cone adress book
Full Command: $ google contacts list name,name,email|perl -pne 's%^((?!N\/A)(.+?)),((?!N\/A)(.+?)),([a-z0-9\._-]+\@([a-z0-9][a-z0-9-]*[a-z0-9]\.)+([a-z]+\.)?([a-z]+))%${1}:${3} %imx'|grep -oP '^((?!N\/A)(.+?)) ' | sort You'll need googlecl and python-gdata. First setup google cl via: $ google Then give your PC access $ google contacts list name,email Then do the command, save it or use this one to dump it in the cone-address.txt file in your home dir: $ google contacts list name,name,email | perl -p -n -e 's%^((?!N\/A)(.+?)),((?!N\/A)(.+?)),([a-z0-9\._-]+\@([a-z0-9][a-z0-9-]*[a-z0-9]\.)+([a-z]+\.)?([a-z]+))%${1}:${3} %imx' | grep -o -P '^((?!N\/A)(.+?)) ' | sort > ~/cone-adress.txt Then import into cone. It filters out multiple emails, and contacts with no email that have N/A. (Picasa photo persons without email for example...)

Listing package man page, services, config files and related rpm of a file, in one alias
Many times I give the same commands in loop to find informations about a file. I use this as an alias to summarize that informations in a single command. Now with variables! :D

Benchmark SQL Query
Benchmark a SQL query against MySQL Server. The example runs the query 10 times, and you get the average runtime in the output. To ensure that the query does not get cached, use `RESET QUERY CACHE;` on top in the query file.

Get the time from NIST.GOV
The format is JJJJJ YR-MO-DA HH:MM:SS TT L DUT1 msADV UTC(NIST) OTM and is explained more fully here: http://tf.nist.gov/service/acts.htm

Google URL shortener
(1) required: python-googl ( install by: pip install python-googl ) (2) get from google API console https://code.google.com/apis/console/

Sorted list of established destination connections
no need grep. its redundant when awk is present.

Timer with sound alarm
I'd prefer this one, you have to install espeak to get this running


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: