Commands using awk (1,418)
-
So I use OSX and don't have the shuf command. This is what I could come up with. This command assumes /usr/share/dict/words does not surpass 137,817,948 lines and line selection is NOT uniformly random. Show Sample Output
0jot 4 | awk '{ print "wc -l /usr/share/dict/words | awk '"'"'{ print \"echo $[ $RANDOM * $RANDOM % \" $1 \"]\" }'"'"' | bash | awk '"'"'{ print \"sed -n \" $1 \"p /usr/share/dict/words\" }'"'"' | bash" }' | bash | tr -d '\n' | sed 's/$/\n/' -
You need to have Brightbox Cloud CLI tools installed Show Sample Output
0for server in `brightbox-servers list |grep active|awk '{ print $1}'`;do brightbox-servers destroy $server;done -
0for ip in `brightbox-cloudips list |grep unmapped|awk '{ print $1}'`;do brightbox-cloudips destroy $ip;done
-
This will find any regular file starting with the current directory and use /dev/urandom to overwrite that file. It will is the same size of the file (in blocks) as the file. Can't handle files with spaces or odd characters in the name (who does that anyway?)
0for F in `find ./ -type f`;do SIZE=`ls -s $F | awk -F" " '{print $1}'`; dd if=/dev/urandom of=$F bs=1024 count=$SIZE;done -
Using large wordlists is cumbersome. Using password cracking programs with rules such as Hashcat or John the ripper is much more effective. In order to do this many times we need to "clean" a wordlist removing all numbers, special characters, spaces, whitespace and other garbage. This command will covert a entire wordlist to all lowercase with no garbage.
0cat dirtyfile.txt | awk '{gsub(/[[:punct:]]/,"")}1' | tr A-Z a-z | sed 's/[0-9]*//g' | sed -e 's/ //g' | strings | tr -cs '[:alpha:]' '\ ' | sed -e 's/ /\n/g' | tr A-Z a-z | sort -u > cleanfile.txt -
My script lists all users & the number of commits they made in the period, sorted alphabetically. To sort by number of commits, append a '|sort' to the end of the command. The script depends on the output format of svn log - original command didn't work for me because the string 'user' was not appearing in my output
0svn log -r{2011-08-01}:HEAD|awk '$14 ~/line/ {print $3}'|sort|uniq -c -
LC_ALL=C is here to always grep on "differ" whatever your language env. xargs -n 2 to run gvim -d with 2 arguments gvim --nofork to use only one instance of gvim
0LC_ALL=C diff -q dir1 dir2 | grep differ | awk '{ print $2, $4 }' | xargs -n 2 gvim --nofork -d -
0awk -F: '{print "echo "$1" | passwd --stdin "$1}' passwd
-
0awk '$3==$4' /etc/passwd
-
0lynx -dump http://www.anti-abuse.org/multi-rbl-check/ | grep ']' | awk -F\] '{ print $2 }' | sed '/^\[/d' | egrep -v ^[A-Z]
-
since awk was already there one can use it instead of the 2 greps. might not be faster, but fast enough
0apt-get remove $(dpkg -l | awk "/^ii linux-(image|headers)/ && ! /`uname -r`/ {print \$2}") -
This will get all links from a given URL, remove any duplicates, and output the result.
0lynx -dump http://example.com/ | awk '/http/{print $2}' | sort -u -
This shell function uses curl(1) as it is more portable than wget(1) across Unices, to show what site a shortened URL is pointing to, even if there are many nested shortened URLs. It is a refinement to www.commandlinefu.com/commands/view/9515/expand-shortened-urls to make it better for use in scripts. Only displays final result.
expandurl http://t.co/LDWqmtDMShow Sample Output0expandurl() { curl -sIL $1 2>&1 | awk '/^Location/ {print $2}' | tail -n1; } -
Sometimes you unzip a file that has no root folder and it spews files all over the place. This will clean up all of those files by deleting them. Show Sample Output
0unzip -l filename.zip |awk '{ if($4 != "Name" && $4 != "----") print $4}'|xargs -t rm -rf {} -
0ls -1 $PATH*/* | xargs file | awk -F":" '!($2~/PDF document/){print $1}' |xargs rm -rf
-
Gets the current system user running a process with the specified pid Show Sample Output
0ps aux | grep PID | grep -v 'grep' | awk '{ print $1 }' -
if you can use this depends on which netapp you have, this netapp is an FAS2020 Show Sample Output
0/usr/bin/snmpwalk -v 1 -c public 10.0.0.22 1.3.6.1.4.1.789.1.21.1.2.1.25.1| awk -F : '{print "netapp temp:" $4 }' -
0find . -type f -exec awk '/linux/ { printf "%s %s: %s\n",FILENAME,NR,$0; }' {} \;
-
svn log -v --> takes log of all Filter1 -------- -r {from}{to} --> gives from and to revision Filter2 -------- awk of line 'r'with numbers Assign user=3rd column [ie; username] Filter3 -------- if username = George print details Filter4 -------- Print lines starts with M/U/G/C/A/D [* A Added * D Deleted * U Updated * G Merged * C Conflicted] Filter5 -------- sort all files Filter6 ------- Print only uniq file's name alone. Show Sample Output
0svn log -v -r{2009-11-1}:HEAD | awk '/^r[0-9]+ / {user=$3} /./{if (user=="george") {print}}' | grep -E "^ M|^ G|^ A|^ D|^ C|^ U" | awk '{print $2}' | sort | uniq -
0lvs | awk '{printf("/dev/%- 40s %-7s %10s %20s %6s %5s %5s %5s %5s\n", $2 "/" $1, $3, $4, $5, $6, $7, $8, $9, $10)}'
-
0sudo ps aux --sort:rss | awk '{print $2"\t"$11": "$6/1024" MB"}' | column -t | less
-
Better awk example, using only mplayer, grep, cut, and awk. Show Sample Output
0mplayer -endpos 0.1 -vo null -ao null -identify *.avi 2>&1 |grep ID_LENGTH |cut -d = -f 2|awk '{SUM += $1} END { printf "%d:%d:%d\n",SUM/3600,SUM%3600/60,SUM%60}' -
This sums up the page count of multiple pdf files without the useless use of grep and sed which other commandlinefus use. Show Sample Output
0find -iname "*.pdf" -exec pdfinfo -meta {} \;|awk '{if($1=="Pages:"){s+=$2}}END{print s}' -
0ac -p | sort -nk 2 | awk '/total/{print x};{x=$1}'
-
This command will help you to find how many number of connection are made to given mysql and what are the different hosts connected to it with number of connection they are making. Show Sample Output
0mysql -u root -p -e"show processlist;"|awk '{print $3}'|awk -F":" '{print $1}'|sort|uniq -c - ‹ First < 30 31 32 33 34 > Last ›
What's this?
commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.
Check These Out
Stay in the loop…
- Follow the Tweets.
-
Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.
» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10 - Subscribe to the feeds.
-
Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):
Subscribe to the feed for:
- » all commands
- » commands with 3 up-votes (commandlinefu3)
- » commands with 10 up-votes (commandlinefu10)
- » commands using awk