
Terminal - Commands using awk - 984 results
awk 'BEGIN{srand()}{print rand(),$0}' SOMEFILE | sort -n | cut -d ' ' -f2-
This is sample output - yours may be different.
This appends a random number as a first filed of all lines in SOMEFILE then sorts by the first column and finally cuts of the random numbers.
awk 'BEGIN{size=5} {mod=NR%size; if(NR<=size){count++}else{sum-=array[mod]};sum+=$1;array[mod]=$1;print sum/count}' file.dat
This is sample output - yours may be different.
Sometimes jittery data hides trends, performing a rolling average can give a clearer view.
mysql -s -e "show processlist" |awk '{print $1}'
This is sample output - yours may be different.
2
3131
3210
5633
6032
6061
6375
6588
6772
7092
7499
7663
svn log fileName|cut -d" " -f 1|grep -e "^r[0-9]\{1,\}$"|awk {'sub(/^r/,"",$1);print "svn cat fileName@"$1" > /tmp/fileName.r"$1'}|sh
This is sample output - yours may be different.
exported files will get a .r23 extension (where 23 is the revision number)
lsof -nP +p 24073 | grep -i listen | awk '{print $1,$2,$7,$8,$9}'
This is sample output - yours may be different.
httpd 24073 TCP *:80 (LISTEN)
httpd 24073 TCP *:443 (LISTEN)
kill_daemon() { echo "Daemon?"; read dm; kill -15 $(netstat -atulpe | grep $dm | cut -d '/' -f1 | awk '{print $9}') }; alias kd='kill_daemon
This is sample output - yours may be different.
Just find out the daemon with $ netstat -atulpe. Then type in his name and he gets the SIGTERM.
ip route show dev eth0 | awk '{print $7}'
This is sample output - yours may be different.
ip address show | grep eth0 | sed '1d' | awk '{print $2}'
does the same, but shows network-prefix.
netstat -ntlp | grep -w 80 | awk '{print $7}' | cut -d/ -f1
This is sample output - yours may be different.
function autoCompleteHostname() { local hosts; local cur; hosts=($(awk '{print $1}' ~/.ssh/known_hosts | cut -d, -f1)); cur=${COMP_WORDS[COMP_CWORD]}; COMPREPLY=($(compgen -W '${hosts[@]}' -- $cur )) } complete -F autoCompleteHostname ssh
This is sample output - yours may be different.
This is meant for the bash shell. Put this function in your .profile and you'll be able to use tab-completion for sshing any host which is in your known_hosts file. This assumes that your known_hosts file is located at ~/.ssh/known_hosts. The "complete" command should go on a separate line as such:
function autoCompleteHostname() {
local hosts=($(awk '{print $1}' ~/.ssh/known_hosts | cut -d, -f1));
local cur=${COMP_WORDS[COMP_CWORD]};
COMPREPLY=($(compgen -W '${hosts[@]}' -- $cur ))
}
complete -F autoCompleteHostname ssh
find . -name \*.mp3 -printf "%C+ %h/%f\n" | sort -r | head -n20 | awk '{print "\""$2"\""}' | xargs -I {} cp {} ~/tmp
This is sample output - yours may be different.
Change ~/tmp to the destination directory, such as your mounted media. Change -n20 to whatever number of files to copy. It should quit when media is full. I use this to put my most recently downloaded podcasts onto my phone.
svn st | grep '^\?' | awk '{print $2}' | xargs svn add; svn st | grep '^\!' | awk '{print $2}' | xargs svn rm
This is sample output - yours may be different.
automatically add and remove files in subversion so that you don't have to do it through the annoying svn commands anymore
chkconfig --list | fgrep :on | sed -e 's/\(^.*\)*0:off/\1:/g' -e 's/\(.\):on/\1/g' -e 's/.:off//g' | tr -d [:blank:] | awk -F: '{print$2,$1}' | ssh host 'cat > foo'
This is sample output - yours may be different.
And then to complete the task:
Go to target host;
ssh host
Turn everything off:
for i in `chkconfig --list | fgrep :on | awk '{print $1}'` ; do chkconfig --level 12345 $i off; done
Create duplicate config:
while read line; do chkconfig --level $line on; done < foo
du -sb *|sort -nr|head|awk '{print $2}'|xargs du -sh
This is sample output - yours may be different.
ip route show dev ppp0 | awk '{ print $7 }'
This is sample output - yours may be different.
awk -F\" '{print $4}' *.log | grep -v "eviljaymz\|\-" | sort | uniq -c | awk -F\ '{ if($1>500) print $1,$2;}' | sort -n
This is sample output - yours may be different.
jaymz@bowser:~/Desktop/tmp$ awk -F\" '{print $4}' *.log | grep -v "eviljaymz\|\-" | sort | uniq -c | awk -F\ '{ if($1>500) print $1,$2;}' | sort -n
517 http://kyberia.sk/id/63756
621 http://kyberia.sk/id/15
628 http://bdelive.fr/shoutbox_view.php?0?auto_refresh=1
644 http://digg.com/all/upcoming/most
658 http://winkftp.free.fr/minichat/minichatone.php
690 http://loading.se/forum.php?thread_id=125&page=412
704 http://www.reddit.com/new/
715 http://www.okoun.cz/boards/komixove_stripy
726 http://www.skyscrapercity.com/showthread.php?t=521993&page=343
759 http://www.reddit.com/r/programming/comments/7zena/reasons_why_people_who_work_with_computers_seem/
760 http://www.kalerab.sk/forum.php?forumid=1243
787 http://www.graphilla.com/viewtopic.php?t=15983&start=1000
819 http://www.google.com/reader/view/
878 http://www.reddit.com/?count=25&after=t3_7z5oj
884 http://www.diskusjon.no/index.php?showtopic=690294&st=8680
998 http://www.reddit.com/top/
1151 http://www.imilovice.cz/main/forum.php?action=showthread&threadid=20192
1239 http://www.ziapps.com/
1259 http://www.jimmyr.com/
1369 http://www.nyx.cz/index.php?l=topic;id=64
1381 http://www.stumbleupon.com/toolbar/
2559 http://twitturls.com/
2763 http://www.reddit.com/
2897 http://www.waarmaarraar.nl/opmerkelijkbeeld/0/GO/0/opmerkelijke_beelden.html
2994 http://www.reddit.com/r/pics/
4445 http://www.reddit.com/r/programming/
8222 http://www.bdelive.fr/shoutbox_view.php?0?auto_refresh=1
24220 http://www.plurk.com/getWidget?uid=423872&h=375&w=190&u_info=1&bg=cf682f&tl=cae7fd
This prints a summary of your referers from your logs as long as they occurred a certain number of times (in this case 500). The grep command excludes the terms, I add this in to remove results Im not interested in.
Q="reddit|digg"; F=*.log; awk -F\" '{print $4}' $F | egrep $Q | wc -l
This is sample output - yours may be different.
jaymz@bowser:~/Desktop/tmp$ Q="reddit"; F=*.log; awk -F\" '{print $4}' $F | egrep $Q | wc -l
32159
I use this (well I normally just drop the F=*.log bit and put that straight into the awk command) to count how many times I get referred from another site. I know its rough, its to give me an idea where any posts I make are ending up. The reason I do the Q="query" bit is because I often want to check another domain quickly and its quick to use CTRL+A to jump to the start and then CTRL+F to move forward the 3 steps to change the grep query. (I find this easier than moving backwards because if you group a lot of domains with the pipe your command line can get quite messy so its normally easier to have it all at the front so you just have to edit it & hit enter).
For people new to the shell it does the following. The Q and F equals bits just make names we can refer to. The awk -F\" '{print $4}' $F reads the file specified by $F and splits it up using double-quotes. It prints out the fourth column for egrep to work on. The 4th column in the log is the referer domain. egrep then matches our query against this list from awk. Finally wc -l gives us the total number of lines (i.e. matches).
ps -eo user,pcpu,pmem | tail -n +2 | awk '{num[$1]++; cpu[$1] += $2; mem[$1] += $3} END{printf("NPROC\tUSER\tCPU\tMEM\n"); for (user in cpu) printf("%d\t%s\t%.2f%\t%.2f%\n",num[user], user, cpu[user], mem[user]) }'
This is sample output - yours may be different.
NPROC USER CPU MEM
16 web2 10.80% 0.00%
1 rpc 0.00% 0.00%
44 web 18.30% 1.70%
9 patrol 0.00% 0.00%
11 wikimoe 0.00% 1.50%
11 gipce 0.00% 0.40%
3 68 0.00% 0.00%
1 dbus 0.00% 0.00%
1 rpcuser 0.00% 0.00%
1 named 0.30% 0.00%
11 wikiprod 0.20%
2 appli 0.00% 0.00%
1 smmsp 0.00% 0.00%
2 avahi 0.00% 0.00%
10 webdba 0.00% 0.00%
1 xfs 0.00% 0.00%
1 ntp 0.00% 0.00%
281 root 2.30% 8.10%
It's like `prstat -t` under Solaris
wget -q -O- http://www.gutenberg.org/dirs/etext96/cprfd10.txt | sed '1,419d' | tr "\n" " " | tr " " "\n" | perl -lpe 's/\W//g;$_=lc($_)' | grep "^[a-z]" | awk 'length > 1' | sort | uniq -c | awk '{print $2"\t"$1}'
This is sample output - yours may be different.
aback 1
abandon 6
abandoned 13
abase 1
abased 1
abashed 7
abated 1
abatement 1
...
This command might not be useful for most of us, I just wanted to share it to show power of command line.
Download simple text version of novel David Copperfield from Poject Gutenberg and then generate a single column of words after which occurences of each word is counted by sort | uniq -c combination.
This command removes numbers and single characters from count. I'm sure you can write a shorter version.
p=$(netstat -nate 2>/dev/null | awk '/LISTEN/ {gsub (/.*:/, "", $4); if ($4 == "4444") {print $8}}'); for i in $(ls /proc/|grep "^[1-9]"); do [[ $(ls -l /proc/$i/fd/|grep socket|sed -e 's|.*\[\(.*\)\]|\1|'|grep $p) ]] && cat /proc/$i/cmdline && echo; done
This is sample output - yours may be different.
Ok so it's rellay useless line and I sorry for that, furthermore that's nothing optimized at all...
At the beginning I didn't managed by using netstat -p to print out which process was handling that open port 4444, I realize at the end I was not root and security restrictions applied ;p
It's nevertheless a (good ?) way to see how ps(tree) works, as it acts exactly the same way by reading in /proc
So for a specific port, this line returns the calling command line of every thread that handle the associated socket
w | egrep -v '(load|FROM)' | awk '{print $2}' | sed 's/^/tty/' | awk '{print "echo \"The Matrix has you...\" >> /dev/" $1}' | bash
This is sample output - yours may be different.
The Matrix has you... (on all writable ttys)
This works just like write or wall ... cept one thing the sender is anonymous ... if you really want to drive everyone insane replace echo \"The Matrix has you...\" with cat /dev/urandom
nice one to do on April fool's day
ps auxf | grep httpd | grep -v grep | grep -v defunct | awk '{sum=sum+$6}; END {print sum/1024}'
This is sample output - yours may be different.
dpkg-query -l| grep -v "ii " | grep "rc " | awk '{print $2" "}' | tr -d "\n" | xargs aptitude purge -y
This is sample output - yours may be different.
This will, for an application that has already been removed but had its configuration left behind, purge that configuration from the system. To test it out first, you can remove the last -y, and it will show you what it will purge without actually doing it. I mean it never hurts to check first, "just in case." ;)
sudo aptitude purge `dpkg --get-selections | grep deinstall | awk '{print $1}'`
This is sample output - yours may be different.
kelevra@lab:~$ sudo aptitude purge `dpkg --get-selections | grep deinstall | awk '{print $1}'`
Reading package lists... Done
Building dependency tree
Reading state information... Done
Initializing package states... Done
Writing extended state information... Done
Reading task descriptions... Done
The following packages will be REMOVED:
a2ps{p} console-tools{p} desktop-base{p} fam{p} fortune-mod{p} libdatrie0{p} libfam0{p} libfs6{p} libvte9{p}
libxfce4mcs-client3{p} libxfce4mcs-manager3{p} lpr{p} x11-apps{p} x11-session-utils{p} xfonts-100dpi{p} xfonts-75dpi{p}
xfonts-scalable{p} xinit{p} xli{p}
0 packages upgraded, 0 newly installed, 19 to remove and 0 not upgraded.
Need to get 0B of archives. After unpacking 0B will be used.
Do you want to continue? [Y/n/?
Purge all configuration files of removed packages
sudo apt-get remove --purge `dpkg -l | awk '{print $2}' | grep gnome` && apt-get autoremove
This is sample output - yours may be different.
kelevra@lab:~$ sudo apt-get remove --purge `dpkg -l | awk '{print $2}' | grep gnome` && apt-get autoremove
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
python-renderpm libgtop2-common xsane libmagick++10 libplot2c2 libcamel1.2-14 python-qt4-common python-reportlab-accel
libcairomm-1.0-1 libglibmm-2.4-1c2a pstoedit python-qt4 libpangomm-1.4-1 python-qt4-dbus python-sip4 libedataserver1.2-11
libgtkmm-2.4-1c2a libwmf-bin xsane-common libgtop2-7 imagemagick python-elementtree perlmagick python-reportlab
libpstoedit0c2a
Use 'apt-get autoremove' to remove them.
The following packages will be REMOVED:
cheese* gksu* gnome-keyring* gnome-mime-data* gnome-mount* hplip-gui* inkscape* libbonoboui2-0* libebook1.2-9*
libeel2-2.20* libgksu2-0* libgnome-keyring0* libgnome-menu2* libgnome2-0* libgnome2-common* libgnomecanvas2-0*
libgnomecanvas2-common* libgnomeui-0* libgnomeui-common* libgnomevfs2-0* libgnomevfs2-common* libgnomevfs2-extra*
libgtkhtml3.14-19* libgtkhtml3.16-cil* libpam-gnome-keyring* monodoc-browser*
0 upgraded, 0 newly installed, 26 to remove and 0 not upgraded.
After this operation, 100MB disk space will be freed.
Do you want to continue [Y/n]?
Useful for removes a package and its depends, for example to remove the gnome desktop environment, also configuration files will be removed, you should be carefully and sure that you want to do this.
ifconfig en1 | awk '/inet / {print $2}' | mail -s "hello world" email@email.com
This is sample output - yours may be different.
This is useful if you have need to do port forwarding and your router doesn't assign static IPs, you can add it to a script in a cron job that checks if you IP as recently changed or with a trigger script.
This was tested on Mac OSX.