Commands tagged awk sorted by votes

Commands tagged awk (348)

sorted by

Sum columns on one line in a csv file.

sample.csv: 79.36,94.93,10.92,27.33,95.90 3.57, 20.80,67.06,2.16, 79.23 48.45,27.95,7.66, 56.71,59.97 69.02,89.59,33.88,42.73,22.60 10.15,44.86,70.86,98.45,22.23 Show Sample Output
This is sample output - yours may be different.
```
308.44
172.82
200.74
257.82
246.55
```
0

awk -F , '{for(i=1;i<=NF;i++)t[NR]+=$i;$0=t[NR]}1' sample.csv

YardApe · 2015-09-05 13:29:18 13
Monitor memory fine-grained usage (e.g. firefox)

I have modified the original one-liner to allow it to manage multiple PIDs. However, I haven't found the way it can work properly with the watch command. NOTE: brace expand must be activated via "set -B" in order to this script can work. Show Sample Output
This is sample output - yours may be different.
```
961936 KB
```
0

cat $(echo "ls /proc/{`pidof firefox | sed -e 's/$/,/' -e 's/ /,/g'`}/smaps 2>/dev/null" | bash) | awk '/Rss/{sum += $2; } END{print sum, "KB"}'

pchalim · 2015-09-30 22:22:53 9
get a random regex match from a file

seed the random number generator, find all matches in a file put all matches from the capture group into an array return a random element from the array Show Sample Output
This is sample output - yours may be different.
```
25
```
0

awk 'BEGIN{srand()} match($0, /DELTA=([0-9]+);/, a) {w[i++]=a[1]} END {print w[int(rand()*i)]}' file.name

jkirchartz · 2015-11-13 17:56:34 11
displays the uptime in a friendly way

useful for human readable reports Show Sample Output
This is sample output - yours may be different.
```
murgatroyd restarted 4 days ago
```
0

echo "$HOSTNAME restarted $(uptime | tr , ' ' | awk '{print $3" "$4}') ago"

forestb · 2015-11-20 19:25:15 10
Get current Xorg resolution via xrandr

if there are multiple monitors, this command uses multiple lines Show Sample Output
This is sample output - yours may be different.
```
1280x800
```
0

xrandr | awk '/*/ {print $1}'

misterhat · 2015-12-21 15:29:39 11
all java -Xmx values in a system, add them up

process name , process id , xms( initial memory allocation pool) , xmx(maximum memory allocation pool) Show Sample Output
This is sample output - yours may be different.
```
solr 32603 -Xmx614 -Xmx614
root 42319 -Xmx60G -Xmx60G
solr 42320 -Xmx60G -Xmx60G
```
0

ps -ef | grep jav[a] | awk '{print substr($1,1),substr($2,1),substr($0,index($0,"-Xms"),7),substr($0,index($0,"-Xmx"),7)}'

illia · 2016-02-14 18:50:58 12
Display the standard deviation of a column of numbers with awk

I was concerned that the other scripts didn't use the correct order of operations for the calculation. Variance needs to be squared before being summed, then divided by NR and then square rooted. Not sure if this method is correct but this is what I came up with. I think it works correctly but I'm not totally sure. If anyone could critique it that would be great.
This is sample output - yours may be different.
0

awk '{delta=$1; avg+=$1/NR;} END {print "stdev = " sqrt(((delta-avg)^2)/NR);}'

123nor · 2016-04-19 08:49:49 12
Finding commands to create alias for

by determining most popular use in history using percentage . Show Sample Output
This is sample output - yours may be different.
```
     1  187  25.7931%   sudo
     2  63   8.68966%   ls
     3  43   5.93103%   ping
     4  31   4.27586%   soccer
     5  26   3.58621%   cd
     6  21   2.89655%   zypper
     7  19   2.62069%   nmcli
     8  17   2.34483%   python
     9  17   2.34483%   find
    10  17   2.34483%   cat
```
0

history | awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | grep -v "./" | column -c3 -s " " -t | sort -nr | nl | head -n10

turrtle13 · 2016-04-24 17:21:35 9
Display a random man page
This is sample output - yours may be different.
0

man $(ls /bin | shuf -n1)

jubnzv · 2016-06-28 18:34:46 12
Ten most often used commands
This is sample output - yours may be different.
0

history | awk '{a[$2]++}END{for(i in a){print a[i] " " i}}' | sort -rn | head

turrtle13 · 2016-07-01 19:38:01 12
Display the top ten running processes sorted by the memory usage:
This is sample output - yours may be different.
0

ps aux | awk '{if ($5 != 0 ) print $2,$5,$6,$11}' | sort -k2rn | head -10 | column -t

turrtle13 · 2016-07-01 19:40:56 12
Disk usage summary

Adds up the used disk space on all hard drives that are directly connected to the machine (i.e. no network mounts etc.) Assumes there are no IDE drives present. Show Sample Output
This is sample output - yours may be different.
```
18 GiB used in total
```
0

echo $(df -BG | grep sd | awk '{print $3}' | paste -sd+ | tr -d G | bc) GiB used in total

thureduehrsen · 2016-08-09 10:08:35 38

Run a job in the background and prefix it's output with some string. This is particularly useful if you are running inside a docker container in a startup script (sue me, I'll run two jobs in a docker container if I want to) and you can run something like: /usr/sbin/nginx 2>&1 | awk '{print "[NGINX] " $0}' & /opt/jws-3.1/tomcat8/bin/catalina.sh run 2>&1 | awk '{print "[TOMCAT] " $0}' & while true; do ; done it can also be combined with tee to create a file log as well as a stdout log, for example if the script above where a script called "/bin/start-container.sh" then you could run /bin/start-container.sh | tee /var/log/containerlogs Show Sample Output

nginx 2>&1 | awk '{print "[NGINX] " $0}' &

hvindin · 2017-04-25 22:18:38 20

Move lots files with AWK
This is sample output - yours may be different.
0

find . -maxdepth 1 -type f | awk 'BEGIN { c=0 } { if( c == <quantity_at_a_time> ) { print l; l=$0; c=1 } else { if( l != "") { l=l" "$0 } else { l=$0 }; c++}} END { print l }' | xargs mv -t <dir_dest>

mvrpl · 2017-04-26 16:48:14 22
easily strace all your apache processes

No need for grep or xargs
This is sample output - yours may be different.
0

ps auxw | awk '/(apache|httpd)/{print"strace -F -p " $2}' | sh

AdvancedThreat · 2017-11-26 17:34:41 20
Get Hardware UUID in Mac OS X
This is sample output - yours may be different.
0

ioreg -d2 -c IOPlatformExpertDevice | awk -F\" '/IOPlatformUUID/{print $(NF-1)}'

n8felton · 2018-08-18 21:18:20 323
Identifies the file types in a directory and adds or replaces their file extensions.

It tries to identify the file types in a directory and adds or replaces them with their appropriate extensions. Please, update the "file" tool before use it (last version: 5.37): https://github.com/file/file
This is sample output - yours may be different.
0

find "$(realpath .)" -type f -exec bash -c 'p="{}"; b="${p##*/}"; e=${b#"${b%.*}"}; e2=$(file -b -F="" --extension "$p"| awk -F/ '\''$1~/^\w+$/ {print $1} '\''); if [ "$e" != ".$e2" ] && [ ! -z $e2 ]; then mv "$p" "${p%$b}${b%$e}.$e2"; fi;' \;

mikhail · 2019-05-18 01:56:25 40
Change values from 0 to 100
This is sample output - yours may be different.
0

awk '{if ($3 =="LAN" && $5 == "0.00" ) print $1, $2, "LAN", "288", "100.00"; else print $1 ,$2, $3, $4, $5 }' sla-avail-2013-Feb > sla-avail-2013-Feb_final

shantanuo · 2020-02-15 10:37:44 76

List SAN domains for a certificate

List SAN Domains for a certificate Show Sample Output

echo | openssl s_client -connect google.com:443 2>&1 | openssl x509 -noout -text | awk -F, -v OFS="\n" '/DNS:/{x=gsub(/ *DNS:/, ""); $1=$1; print $0}'

shadyabhi · 2020-04-10 06:56:51 88

MySQL: Filter out specific tables from an existing mysqldump file with awk

This command works only if the line "DROP TABLE IF EXISTS" exists for all tables in the mysqldump file. It acts like a state machine.
This is sample output - yours may be different.
0

cat db_dump.sql | awk '/DROP TABLE IF EXISTS/ { skip = $5 ~ /table1|table2/ } !skip { print $0 }' > db_dump_filtered.sql

stf42 · 2022-10-30 16:58:57 801
Edit all files found having a specific string found by grep

needs no GNU tools, as far as I see it
This is sample output - yours may be different.
-1

find . -exec grep foobar /dev/null {} \; | awk -F: '{print $1}' | xargs vi

verboEse · 2009-09-03 16:02:18 3
Make a statistic about the lines of code

Have wc work on each file then add up the total with awk; get a 43% speed increase on RHEL over using "-exec cat|wc -l" and a 67% increase on my Ubuntu laptop (this is with 10MB of data in 767 files).
This is sample output - yours may be different.
-1

find . -type f -name '*.c' -exec wc -l {} \; | awk '{sum+=$1} END {print sum}'

arcege · 2009-09-04 15:51:30 6
Show current folder permission from /, useful for debugging ssh key permission

Handled all within awk. Takes the value from $PWD and constructs directory structures and runs commands against them. The gsub() call is not necessary, but added for better visibility. If a variable DIR is given on the awk command-line, then that directory is used instead: awk -vDIR=$HOME/.ssh 'BEGIN{dir=DIR?...}' Show Sample Output
This is sample output - yours may be different.
```
drwxr-xr-x 22 root root 4096 2009-10-21 23:30 /
drwxr-xr-x 3 root root 4096 2009-10-20 13:33 /home
drwxr-xr-x 54 luser luser 4096 2009-10-22 09:48 /home/luser
```
-1

awk 'BEGIN{dir=DIR?DIR:ENVIRON["PWD"];l=split(dir,parts,"/");last="";for(i=1;i<l+1;i++){d=last"/"parts[i];gsub("//","/",d);system("ls -ld \""d"\"");last=d}}'

arcege · 2009-10-22 16:28:07 5

print statistics about users' connect time

'ac' is included in the package 'acct', which is described as "The GNU Accounting utilities for process and login accounting". Other interesting flags are: * print statistics for a specified user ac -d username * print statistics for all the users ac -p With my command, the output is also printed in a sexagesimal, more readable, style. Show Sample Output

-1

ac -d | awk '{h=int($NF); m=($NF-h)*60; s=int((m-int(m))*60); m=int(m); print $0" = "h"h "m"m "s"s "}'

karpoke · 2010-06-24 08:08:41 7

Find default gateway

Tested on CentOS, Ubuntu, and MacOS.
This is sample output - yours may be different.
-1

netstat -rn | awk '/UG/{print $2}'

putnamhill · 2010-08-09 15:56:02 5
‹ First < 10 11 12 13 14 >

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

A function to find the newest file of a set.

Usage example: $newest Desktop/* Replace "-nt" with "-ot" for oldest. Run $shopt -s dotglob first to include dotfiles.

watch iptables counters

This will allow you to watch as matches occur in real-time. To filter out only ACCEPT, DROP, LOG..etc, then run the following command: watch 'iptables -nvL | grep -v "0 0" && grep "ACCEPT"' The -v is used to do an inverted filter. ie. NOT "0 0"

Remove \r (carriage return) in a file

Happy Days

This never gets old

Get AWS temporary credentials ready to export based on a MFA virtual appliance

You might want to secure your AWS operations requiring to use a MFA token. But then to use API or tools, you need to pass credentials generated with a MFA token. This commands asks you for the MFA code and retrieves these credentials using AWS Cli. To print the exports, you can use: `awk '{ print "export AWS_ACCESS_KEY_ID=\"" $1 "\"\n" "export AWS_SECRET_ACCESS_KEY=\"" $2 "\"\n" "export AWS_SESSION_TOKEN=\"" $3 "\"" }'` You must adapt the command line to include: * $MFA_IDis ARN of the virtual MFA or serial number of the physical one * TTL for the credentials

Prints the latest modified files in a directory tree recursively

Sorts by latest modified files by looking to current directory and all subdirectories

Colorized JSON pretty printing

Uses pygmentize and python to create indented and colorized JSON output

Create a mirror of a local folder, on a remote server

Create a exact mirror of the local folder "/root/files", on remote server 'remote_server' using SSH command (listening on port 22) (all files & folders on destination server/folder will be deleted)

suspend to ram without root

$dbus-send --system --print-reply --dest="org.freedesktop.UPower" /org/freedesktop/UPower org.freedesktop.UPower.Suspend

Batch convert PNG to JPEG

Convert all PNG images in directory to JPEG using ImageMagick, and delete the old PNG images.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged awk

Commands tagged awk (348) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged awk (348)

sorted by