Commands tagged awk

Ever compress a file for the web by replacing all newline characters with nothing so it makes one nice big blob?

It is a great idea, however what about when you want to edit that file? ...Serious pain in the butt.

I ran into this today in that my only copy of a CSS file was "compressed" with no newlines.

I whipped this up and it converted back into nice human readable CSS :-)

It could be nicer, but it does the job.

This appends a random number as a first filed of all lines in SOMEFILE then sorts by the first column and finally cuts of the random numbers.

Sometimes jittery data hides trends, performing a rolling average can give a clearer view.

exported files will get a .r23 extension (where 23 is the revision number)

This command might not be useful for most of us, I just wanted to share it to show power of command line.

Download simple text version of novel David Copperfield from Poject Gutenberg and then generate a single column of words after which occurences of each word is counted by sort | uniq -c combination.

This command removes numbers and single characters from count. I'm sure you can write a shorter version.

% cat ph-vmstat.awk

# Return human readable numbers

function hrnum(a) {

b = a ;

if (a > 1000000) { b = sprintf("%2.2fM", a/1000000) ; }

else if (a > 1000) { b = sprintf("%2.2fK", a/1000) ; }

return(b) ;

}

# Return human readable storage

function hrstorage(a) {

b = a ;

if (a > 1024000) { b = sprintf("%2.2fG", a/1024/1024) ; }

else if (a > 1024) { b = sprintf("%2.2fM", a/1024) ; }

return(b) ;

}

OFS=" " ;

$1 !~ /[0-9].*/ {print}

$1 ~ /[0-9].*/ {

$4 = hrstorage($4) ;

$5 = hrstorage($5) ;

$9 = hrnum($9) ;

$10 = hrnum($10) ;

$17 = hrnum($17) ;

$18 = hrnum($18) ;

$19 = hrnum($19) ;

print ;

}

Ok so it's rellay useless line and I sorry for that, furthermore that's nothing optimized at all...

At the beginning I didn't managed by using netstat -p to print out which process was handling that open port 4444, I realize at the end I was not root and security restrictions applied ;p

It's nevertheless a (good ?) way to see how ps(tree) works, as it acts exactly the same way by reading in /proc

So for a specific port, this line returns the calling command line of every thread that handle the associated socket

This will, for an application that has already been removed but had its configuration left behind, purge that configuration from the system. To test it out first, you can remove the last -y, and it will show you what it will purge without actually doing it. I mean it never hurts to check first, "just in case." ;)

Useful for removes a package and its depends, for example to remove the gnome desktop environment, also configuration files will be removed, you should be carefully and sure that you want to do this.

This is useful if you have need to do port forwarding and your router doesn't assign static IPs, you can add it to a script in a cron job that checks if you IP as recently changed or with a trigger script.

This was tested on Mac OSX.

Lists the local files that are not present in the remote repository (lines beginning with ?)

and add them.

This command will sort the contents of FILENAME by redirecting the output to individual .txt files in which 3rd column will be used for sorting. If FILENAME contents are as follows:

foo foo A foo

bar bar B bar

lorem ipsum A lorem

Then two files called A.txt and B.txt will be created and their contents will be:

A.txt

foo foo A foo

lorem ipsum A lorem

and B.txt will be

bar bar B bar

Here is a command line to run on your server if you think your server is under attack. It prints our a list of open connections to your server and sorts them by amount.

BSD Version:

The arguments of "seq" indicate the starting value, step size, and the end value of the x-range. "awk" outputs (x, f(x)) pairs and pipes them to "graph", which is part of the "plotutils" package.

Displays six rows and five columns of random numbers between 0 and 1. If you need only one column, you can dispense with the "for" loop.

This example calculates the averages of column one and column two of "file.dat". It can be easily modified if other columns are to be averaged.

Another combination of seq and awk. Not very efficient, but sufficiently quick.

"seq 100" outputs 1,2,..,100, separated by newlines. awk adds them up and displays the sum.

"seq 1 2 11" outputs 1,3,..,11.

Variations:

1+3+...+(2n-1) = n^2

1/2 + 1/4 + ... = 1

Displays the duplicated lines in a file and their occuring frequency.

Show the number of failed tries of login per account. If the user does not exist it is marked with *.

Takes a input file (count.txt) that looks like:

1

2

3

4

5

It will add/sum the first column of numbers.

This command checks for the number of times when someone has tried to login to your server and failed. If there are a lot, then that user is being targeted on your system and you might want to make sure that user either has remote logins disabled, or has a strong password, or both. If your output has an "invalid" line, it is a summary of all logins from users that don't exist on your system.

Useful to check DDoS attacks on servers.