Commands using awk (1,418)
-
4lsof -iTCP -sTCP:ESTABLISHED | awk '{print $1}' | sort -u
-
The command (above) will remove any duplicate rows based on the FIRST column of data in an un-sorted file. The '$1' represents a positional parameter. You can change both instances of '$1' in the command to remove duplicates based on a different column, for instance, the third:
awk '{ if ($3 in stored_lines) x=1; else print; stored_lines[$3]=1 }' infile.txt > outfile.txtOr you can change it to '$0' to base the removal on the whole row:awk '{ if ($0 in stored_lines) x=1; else print; stored_lines[$0]=1 }' infile.txt > outfile.txt** Note: I wouldn't use this on a MASSIVE file, unless you're RAM-rich ;) **4awk '{ if ($1 in stored_lines) x=1; else print; stored_lines[$1]=1 }' infile.txt > outfile.txt -
4wget -qO- www.commandlinefu.com/commands/by/PhillipNordwall | awk -F\> '/num-votes/{S+=$2; I++}END{print S/I}'
-
A shorter version of command #3014, using awk instead of sed. Useful when scraping websites with a script.
4awk '{gsub("<[^>]*>", "")}1' file -
This one-liner will output installed packages sorted by size in Kilobytes. Show Sample Output
4paste <(pacman -Q | awk '{ print $1; }' | xargs pacman -Qi | grep 'Size' | awk '{ print $4$5; }') <(pacman -Q | awk '{print $1; }') | sort -n | column -t -
Make sure to run this command in your git toplevel directory. Modify `-j4` as you like. You can also run any arbitrary command beside `git pull` in parallel on all of your git submodules. Show Sample Output
4parallel -j4 cd {}\; pwd\; git pull :::: <(git submodule status | awk '{print $2}') -
4netstat -ntu | awk ' $5 ~ /^[0-9]/ {print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
-
4import -window `xwininfo | awk '/Window id/{print $4; exit}'` `uuidgen`.png
-
When working with jailed environments you need to copy all the shared libraries to your jail environment. This is done by running ldd on a binary which needs to run inside the jail. This command will use the output from ldd to automatically copy the shared libraries to a folder of your choice. Show Sample Output
4ldd file | grep "=> /" | awk '{print $3}' | xargs -I '{}' cp -v '{}' /destination -
4xprop | awk '/PID/ {print $3}'
-
4for line in `docker ps | awk '{print $1}' | grep -v CONTAINER`; do docker ps | grep $line | awk '{printf $NF" "}' && echo $(( `cat /sys/fs/cgroup/memory/docker/$line*/memory.usage_in_bytes` / 1024 / 1024 ))MB ; done
-
Note that the -i will not help in a script. Proper error checking is required. Show Sample Output
4mv -iv $FILENAME{,.$(stat -c %y $FILENAME | awk '{print $1}')} -
pyt 'Stairway to heaven - Led Zeppelin' pyt 'brain damage - Pink Floyd' No web browser or even X needed. Just a cli and internet connection! mplayer is pauseable and can skip ahead This may break if youtube changes their search html.
4pyt() { id=$(curl -s 'https://www.youtube.com/results?search_query='$(tr ' ' + <<<"$1") | grep -om3 '"[[:alnum:]]\{11\}"' | awk NR==3 | tr -d \"); youtube-dl -q 'https://www.youtube.com/watch?v='"$id" -o - | mplayer -vo null /dev/fd/3 3<&0 </dev/tty; } -
This command will add up RAM usage of all processes whose name contains "java" and output the sum of percentages in HRF. Also, unlike the original #15430, it wont fail on processes with a usage of >9.9%. Pleases note that this command wont work reliably in use cases where a significant portion of processes involved are using less than 0.1% of RAM, because they will be counted as "0", even though a great number of them could add up to significant amounts. Show Sample Output
4ps -eo pmem,comm | grep java | awk '{sum+=$1} END {print sum " % of RAM"}' -
4ASN=32934; for s in $(whois -H -h riswhois.ripe.net -- -F -K -i $ASN | grep -v "^$" | grep -v "^%" | awk '{ print $2 }' ); do echo " blocking $s"; sudo iptables -A INPUT -s $s -j REJECT &> /dev/null || sudo ip6tables -A INPUT -s $s -j REJECT; done
-
Read all lines using decimal marker as point, then add all them up and outputs the result. Show Sample Output
4awk '{a+=$0}END{print a}' file -
3lspci | grep Ether | awk '{ VAR=$1; split(VAR,ARR,"."); count[ARR[1]]++; LINE=$0; split(LINE,LINEARR,":"); LINECOUNT[ARR[1]]=LINEARR[3]; } END { for(i in count) { printf("PCI address: %s\nPorts: %d\nCard Type: %s\n", i, count[i], LINECOUNT[i]) } }'
-
List top 20 IP from which TCP connection is in SYN_RECV state. Useful on web servers to detect a syn flood attack. Replace SYN_ with ESTA to find established connections Show Sample Output
3netstat -pant 2> /dev/null | grep SYN_ | awk '{print $5;}' | cut -d: -f1 | sort | uniq -c | sort -n | tail -20 -
This will tell you who has the most Apache connections by IP (replace IPHERE with the actual IP you wish to check). Or if you wish, remove | grep -c IPHERE for the full list.
3netstat -anl | grep :80 | awk '{print $5}' | cut -d ":" -f 1 | uniq -c | sort -n | grep -c IPHERE -
3for i in $(svn st | grep "?" | awk '{print $2}'); do svn add $i; done;
-
3ps aux | awk '{sum+=$6} END {print sum/1024}'
-
This is a 'killall' command equivalent where it is not available. Prior to executing it, set the environment variable USERNAME to the username, whose processes you want to kill or replace the username with the $USERNAME on the command above. Side effect: If any processes from other users, are running with a parameter of $USERNAME, they will be killed as well (assuming you are running this as root user) [-9] in square brackets at the end of the command is optional and strongly suggested to be your last resort. I do not like to use it as the killed process leaves a lot of mess behind.
3ps -ef | grep $USERNAME | awk {'print $2'} | xargs kill [-9] -
3touch /tmp/$$;for N in `seq -w 0 7777|grep -v [89]`; do chmod $N /tmp/$$; P=`ls -l /tmp/$$ | awk '{print $1}'`; echo $N $P; done;rm /tmp/$$
-
Displays six rows and five columns of random numbers between 0 and 1. If you need only one column, you can dispense with the "for" loop. Show Sample Output
3seq 6 | awk '{for(x=1; x<=5; x++) {printf ("%f ", rand())}; printf ("\n")}' -
3/sbin/ifconfig -a | awk '/(cast)/ { print $2 }' | cut -d':' -f2 | head -1
- ‹ First < 7 8 9 10 11 > Last ›
What's this?
commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.
Check These Out
Stay in the loop…
- Follow the Tweets.
-
Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.
» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10 - Subscribe to the feeds.
-
Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):
Subscribe to the feed for:
- » all commands
- » commands with 3 up-votes (commandlinefu3)
- » commands with 10 up-votes (commandlinefu10)
- » commands using awk