Commands using awk (1,418)
-
Outputs unique error messages from the apache log, sorted by frequency. Tested on log format: [Wed Aug 26 00:00:00 2015] [error] [client xx.xx.xx.xx] Error message here Show Sample Output
0cat /var/log/apache2/error.log | awk '{out=$9;for(i=9;i<=NF;i++){out=out" "$i}; print out}' | sed s/,\ referer.*// | sort | uniq -c | sort -nr -
tested on solaris 10
0ps -e | awk '$4~/<process name>/{print $1}' |xargs gcore -o ~/troubleshooting -
sample.csv: 79.36,94.93,10.92,27.33,95.90 3.57, 20.80,67.06,2.16, 79.23 48.45,27.95,7.66, 56.71,59.97 69.02,89.59,33.88,42.73,22.60 10.15,44.86,70.86,98.45,22.23 Show Sample Output
0awk -F , '{for(i=1;i<=NF;i++)t[NR]+=$i;$0=t[NR]}1' sample.csv -
Some times firefox run a plugin named "plugin-containe". It sometimes take so much cpu that it is impossible to keep browsing. When this happens, one can simply kill the process. If the process does not exist, the command returns an innocuous error message.
0kill -9 `top -b -n 1 | grep plugin-containe | awk '{print $1}'` -
Two files are compared according their (third, $3) column. The command shows the lines ($0) that are in file 2 and not in file1
0awk '(ARGIND==1) {tab[$3]=1} (ARGIND==2 && ! ($3 in tab) ) {print $0}' file1 file2 -
A potential source of a full filesystem are large files left open but have been deleted. On Linux, a file may be deleted (removed/unlinked) while a process has it open. When this happens, the file is essentially invisible to other processes, but it still takes on physical space on the drive. Tools like du will not see it.
0sudo lsof -nP | awk '/deleted/ { sum+=$8 } END { print sum }' -
Time protocol serves up seconds since 1900 in a 32 bit binary number. Show Sample Output
0telnet time.nist.gov 37 2>/dev/null |tail -1 | xxd | awk '{print $2 $3}'|tr -d '\n' | perl -nwe "print scalar(hex($1)-2208988800).\"\n\"" -
I have modified the original one-liner to allow it to manage multiple PIDs. However, I haven't found the way it can work properly with the watch command. NOTE: brace expand must be activated via "set -B" in order to this script can work. Show Sample Output
0cat $(echo "ls /proc/{`pidof firefox | sed -e 's/$/,/' -e 's/ /,/g'`}/smaps 2>/dev/null" | bash) | awk '/Rss/{sum += $2; } END{print sum, "KB"}' -
0lsof / | awk '{ if($7 > 1048576) print $7/1048576 "MB "$9 }' | sort -n -u | tail
-
0awk '/VmSwap/{print $2 " " $3}' /proc/$PID/status
-
0ls -1 /proc/$(ps ax | grep <Process Name> | head -n 1 | awk '{print $1;}')/task | tail -n +2
-
0for x in {0..90}; do { echo $x $(find . -type f -mtime $x -exec du -cs {} \; |awk -F " " '{sum+=$1} END {sum/=1048576}END{print sum,"G"}'); }; done
-
0lsof -Pn +D /var/cache/ | awk '{print $1}' | sort | uniq
-
seed the random number generator, find all matches in a file put all matches from the capture group into an array return a random element from the array Show Sample Output
0awk 'BEGIN{srand()} match($0, /DELTA=([0-9]+);/, a) {w[i++]=a[1]} END {print w[int(rand()*i)]}' file.name -
0awk '{counts = 137 * $3 *$6; line = $0;}$1 == "CHROM" ||$4 < max_het && $3 > min_cov && $7 == "0" && counts > 1.1 {print line}' file.ltab |cut -f 1,2,9- > file.filtered.tab
-
get the size of S3 bucket Show Sample Output
0s3cmd du s3://bucket-name | awk '{print $0/1024/1024/1024" GB"}' -
0for f in `git status | grep new | awk '{print $3}'`; do git reset HEAD $f ; done
-
if there are multiple monitors, this command uses multiple lines Show Sample Output
0xrandr | awk '/*/ {print $1}' -
to view on another box:
nc <server address> <port> | ffplay -use -r to adjust FPS and -q to adjust compression. use on trusted network only as nc is unencrypted.0ffmpeg -f x11grab -s $(xrandr | awk '/*/ {print $1}') -r 10 -i :0 -an -q 10 -f mjpeg - | nc -lp <port> -
list of all virtual hosts which are defined in all apache configuration files
0sudo httpd -S 2>&1 | grep 'port' | awk '{print $4,$2,$5}' | sort | column -t -
use get_iplay to download all listed content from http://www.bbc.co.uk/radio4extra run every night to make sure no episodes are missed
0get_iplayer --type=radio --channel "Radio 4 Extra" | grep : | awk '{ if ( NR > 1 ) { print } }'|sed 's/:.*//' |sed '$ d' > pidlist && while read p; do get_iplayer --get --fields=pid $p; done <pidlist && rm pidlist -
this is what i needed on ubuntu 12.04 for ios 9
0lsusb -s `lsusb | grep "Apple" | cut -d ' ' -f 2`:`lsusb | grep "Apple" | cut -d ' ' -f 4 | sed 's/://'` -v | grep iSerial | awk '{print $3}' -
0pmset -g batt | awk '/-InternalBattery-/ {sub(/;/, "", $2); print $2}'
-
0for i in $(git log --format='%ae' | sort -u); do echo $i; git log --author="$i" --pretty=tformat: --numstat | awk '{ add += $1; subs += $2; loc += $1 - $2 } END { printf "added lines: %s, removed lines: %s, total lines: %s\n", add, subs, loc }' - done
-
Pros: * it's much faster then for loop * removes first line of 'eselect bashcomp list' - which contains text "Available completions:"
0eselect bashcomp enable --global $(eselect bashcomp list | tail -n +2 | awk '{print $2}' |xargs) - ‹ First < 42 43 44 45 46 > Last ›
What's this?
commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.
Check These Out
Stay in the loop…
- Follow the Tweets.
-
Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.
» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10 - Subscribe to the feeds.
-
Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):
Subscribe to the feed for:
- » all commands
- » commands with 3 up-votes (commandlinefu3)
- » commands with 10 up-votes (commandlinefu10)
- » commands using awk