Commands tagged awk sorted by votes

Commands tagged awk (348)

sorted by

Get all links of a website
This is sample output - yours may be different.
0

lynx -dump http://www.cooks4arab.com | awk '/http/{print $2}' | egrep "^https{0,1}"

khaled_eng15 · 2012-04-23 10:43:33 3

Converts the output of disklabel from bytes to Gigabytes on FreeBSD

This was done in csh. Show Sample Output

echo `disklabel mfid1s4 | sed -n '$p' | awk '{print $2}'` / 1024 / 1024 | bc -l

RageJello · 2012-05-04 12:48:58 3

Split tsv file by fifth column ... or any column

Will split the std input lines into files grouped by the 5th column content.
This is sample output - yours may be different.
0

awk -F'\t' '{print $0 >>$5.tsv}'

pykler · 2012-05-16 18:18:16 3
Revert all modified files in an SVN repo

This command allows you to revert every modified file one-by-one in a while loop, but also after "echo $file;" you can do any sort of processing you might want to add before the revert happens. Show Sample Output
This is sample output - yours may be different.
```
my/files.txt
Reverted 'my/files.txt'
```
0

svn status | grep "^M" | while read entry; do file=`echo $entry | awk '{print $2}'`; echo $file; svn revert $file; done

wsams · 2012-06-17 16:01:06 9
Number of connections per IP with range 24

cut -f1,2 - IP range 16 cut -f1,2,3 - IP range 24 cut -f1,2,3,4 - IP range 24 Show Sample Output
This is sample output - yours may be different.
```
      1 192.168.1.5
      5 192.168.1.2
      8 192.168.1.52
     15 192.168.1.3
     19 192.168.1.30
```
0

netstat -tn | grep :80 | awk '{print $5}'| grep -v ':80' | cut -f1 -d: |cut -f1,2,3 -d. | sort | uniq -c| sort -n

krishnan · 2012-06-26 08:29:37 3
Get the IP address

That's the easiest way to do it. -I (or capital i) display all network addresses of a host
This is sample output - yours may be different.
0

hostname -I

bashfan · 2012-07-18 19:43:48 3
sum and average of requests responses times or sizes in Apache2 access log

Depending on your Apache access log configuration you may have to change the sum+=$11 to previous or next awk token. Beware, usually in access log last token is time of response in microseconds, penultimate token is size of response in bytes. You may use this command line to calculate sum and average of responses sizes. You can also refine the egrep regexp to match specific HTTP requests. Show Sample Output
This is sample output - yours may be different.
```
Tot=134020202(15201)
Avg=8816.54
```
0

egrep '.*(("STATUS)|("HEAD)).*' http_access.2012.07.18.log | awk '{sum+=$11; ++n} END {print "Tot="sum"("n")";print "Avg="sum/n}'

fanchok · 2012-07-27 12:18:29 3
Capture screen and mic input using FFmpeg and ALSA

Now we can capture only a specific window (we have to chose by clicking on it) ffmpeg complains about "Frame size must be a multiple of 2" so we calculate the upper even number with (g)awk trickery. We remove the grep, we are already using (g)awk here ....why losing time with grep !!! ;) Show Sample Output
This is sample output - yours may be different.
```
...
Stream mapping:
  Stream #0.0 -> #0.0
Press [q] to stop encoding
frame=   18 fps= 11 q=0.0 Lsize=      55kB time=1.80 bitrate= 252.5kbits/s    
video:55kB audio:0kB global headers:0kB muxing overhead 1.626234%
```
0

a=$(xwininfo |gawk 'BEGIN {FS="[x+ \t]*"} /-geometry/ {print int(($3+1)/2)*2"x"int(($4+1)/2)*2"+"$5"+"$6}') ; echo ${a} ; ffmpeg -f x11grab -s ${a} -r 10 -i :0.0 -sameq -f mp4 -s wvga -y /tmp/out.mpg

dwygo · 2012-08-31 14:48:41 5
Number of open connections per ip.

Count on a specific port (80) - FreeBSD friendly. Show Sample Output
This is sample output - yours may be different.
```
 25
```
0

netstat -an | grep 80 | wc -l

mrhassell · 2012-09-05 06:17:09 5
Revert all modified files in an SVN repo
This is sample output - yours may be different.
0

for file in `svn st | awk '{print $2}'`; do svn revert $file; done

srounce · 2012-09-14 11:05:29 4
prints line numbers

Let's not forget awk! Show Sample Output
This is sample output - yours may be different.
```
1	The quick brown fox 
2	jump over the lazy dog.
```
0

awk '{print NR "\t" $0}'

tox2ik · 2012-10-21 15:45:54 4
List all the unique extensions in a folder and print their count
This is sample output - yours may be different.
0

find . -type f -print | awk -F'.' '{print $NF}' | sort | uniq -c

vishnu81 · 2012-10-31 12:03:10 5
Purge frozen messages in Exim

do 1000 at a time so that if your doodoo is deep you can avoid avoid "command-line too big" error
This is sample output - yours may be different.
0

exipick -zi | xargs --max-args=1000 exim -Mrm

jasen · 2012-12-12 20:46:22 5
Command to get list of all existing programming languages from Wikipedia article

Requirements: curl, grep, awk, internet connection with access to wikipedia Loaded page: http://en.wikipedia.org/wiki/List_of_programming_languages If you can make shorter version of this listgetter, you are welcome to paste it here :) Show Sample Output
This is sample output - yours may be different.
```
R (programming language)
R++
Racket (programming language)
RAPID
Rapira
Ratfiv
Ratfor
Rc
REBOL
Core War
REFAL
Reia (programming language)
Revolution (programming language)
Rex (language)
REXX
Rlab
ROOP (programming language)
IBM RPG
RPL (programming language)
Robot Battle
RTL/2
Ruby (programming language)
Rust (programming language)
```
0

curl http://en.wikipedia.org/wiki/List_of_programming_languages | grep "<li>" | awk -F"title=" '{ print $2 }' | awk -F\" '{ print $2 }'

sxiii · 2013-01-09 21:40:11 11
One liner to show a notification on Desktop in case the load exceeds 1.

I run this via crontab every one minute on my machine occasionally to see if a process is eating up my system's resources.
This is sample output - yours may be different.
0

load=`uptime|awk -F',' '{print $3}'|awk '{print $3}'`; if [[ $(echo "if ($load > 1.0) 1 else 0" | bc) -eq 1 ]]; then notify-send "Load $load";fi

adimania · 2013-02-06 08:30:24 5

Print data from field 9 till the end separated by a white space and new record separated by newline.

It'll print the file names preserving the spaces in their names and adding new line after every new filename. I wrote this to quickly find out how many files in any directory is owned by a particular user. This can be extended using pipe and grep to do much more. Show Sample Output

awk '{for (i=9;i<=NF;i++) {printf "%s",$i; printf "%s", " ";}; printf "\n"}'

adimania · 2013-02-12 13:57:43 4

list unique file extensions recursively for a path, include extension frequency stats

If you have GNU findutils, you can get only the file name with find /some/path -type f -printf '%f\n' instead of find /some/path -type f | gawk -F/ '{print $NF}' Show Sample Output
This is sample output - yours may be different.
```
100 txt
 25 gz
 10 vim
  3 gitignore
```
0

find /some/path -type f | gawk -F/ '{print $NF}' | gawk -F. '/\./{print $NF}' | sort | uniq -c | sort -rn

skkzsh · 2013-03-18 14:40:26 4
zip all folders separate zip files with named the folder's name.
This is sample output - yours may be different.
0

eval `ls -1d * | awk '{print "zip -r "$1".zip "$1";"}'`

javaloper · 2013-04-29 12:00:54 6
Remove duplicate lines using awk
This is sample output - yours may be different.
0

awk '!($0 in array) { array[$0]; print }' temp

techie · 2013-05-09 08:23:30 10
Get the current svn branch/tag (Good for PS1/PROMPT_COMMAND cases)
This is sample output - yours may be different.
0

svn info | grep ^URL | awk -F\/ '{print $NF}'

mplf · 2013-05-18 15:34:09 7
display bash history without line numbers

alias h="history | awk '{\$1=\"\";print substr(\$0,2)}'" # h [ 07/07/2013 10:04:53 ] alias h="history | awk '{\$1=\"\";print substr(\$0,2)}'"
This is sample output - yours may be different.
0

history | awk '{$1="";print substr($0,2)}'

Fagood · 2013-07-07 08:00:26 9
Batch JPEG rename to date using ImageMagick

Requires ImageMagick. Extracts date taken from image and renames it properly. Based on StackOverflow answer. Show Sample Output
This is sample output - yours may be different.
```
`IMG_0237.JPG' -> `2013_07_18_08_15_44.jpg'
`IMG_0238.JPG' -> `2013_07_18_16_47_24.jpg'
`IMG_0239.JPG' -> `2013_07_18_20_47_01.jpg'
`IMG_0240.JPG' -> `2013_07_18_20_47_15.jpg'
`IMG_0241.JPG' -> `2013_07_19_07_21_41.jpg'
```
0

for fil in *.JPG; do datepath="$(identify -verbose $fil | grep DateTimeOri | awk '{print $2"_"$3 }' | sed s%:%_%g)"; mv -v $fil $datepath.jpg; done

unixmonkey63215 · 2013-08-02 01:42:04 8
Find a machine's IP address and FQDN
This is sample output - yours may be different.
0

for i in `ip addr show dev eth1 | grep inet | awk '{print $2}' | cut -d/ -f1`; do echo -n $i; echo -en '\t'; host $i | awk '{print $5}'; done

BoxingOctopus · 2013-09-05 17:50:33 7
Get IP address from domain

Show Sample Output
This is sample output - yours may be different.
```
123.123.123.123
```
0

host example.com | head -1 | awk '{print $4}'

wes · 2013-09-05 22:20:49 7
Get the Nth argument of the last command in $HISTFILE

This gets the Nth argument in the last line of your history file. This is useful where history is being written after each command, and you want to use arguments from the previous command in the current command, such as when doing copies/moving directories etc. I wrote this after getting irritated with having to continually type in long paths/arguments. You could also use $_ if all you want is the last argument. Show Sample Output
This is sample output - yours may be different.
```
$ echo "This little piggy went to the market"
This little piggy went to the market
$ garg 5
went
```
0

function garg () { tail -n 1 ${HISTFILE} | awk "{ print \$$1 }" }

plasticphyte · 2013-09-10 04:07:46 10
‹ First < 8 9 10 11 12 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Fix Ubuntu's Broken Sound Server

Ever since the switch to pulseaudio, Ubuntu users including myself have found themselves with no sound intermittently. To fix this, just use this command and restarts firefox or mplayer or whatever.

converting vertical line to horizontal line

Download all Phrack .tar.gzs

Capture video of a linux desktop

This is identical to the original command, but allows you to specify an offset if you have multiple monitors. In this case you'll record a 1366x768 window that's offset by 1366 pixels in the x axis and 0 pixels in the y axis. Note also that the -sameq option has been replaced by -qscale - at least in ffmpeg version 1.2.1 on Fedora 19.

Create a mirror of a local folder, on a remote server

Create a exact mirror of the local folder "/root/files", on remote server 'remote_server' using SSH command (listening on port 22) (all files & folders on destination server/folder will be deleted)

Test network speed without wasting disk

The above command will send 4GB of data from one host to the next over the network, without consuming any unnecessary disk on either the client nor the host. This is a quick and dirty way to benchmark network speed without wasting any time or disk space. Of course, change the byte size and count as necessary. This command also doesn't rely on any extra 3rd party utilities, as dd, ssh, cat, /dev/zero and /dev/null are installed on all major Unix-like operating systems.

Recover tmp flash videos (deleted immediately by the browser plugin)

Newer versions of the flashplayer browser plugin delete the tmp flash video immediately after opening a filehandle to prevent the user from "exporting" the video by simply copying the /tmp/FlashXYZ file. This command searches such deleted flash videos and creates symbolic links to the opened filehandle with the same name as the deleted file. This allows you to play your flash-videos (from e.g. youtube) with e.g. mplayer or copy the buffered video if you want to keep it.

To create files with specific permission:

wget multiple files (or mirror) from NTLM-protected Sharepoint

If you have to deal with MS Sharepoint which is (rarely, let's hope) used in e.g. certain corporate environments). This uses Cntlm. For single files, just use cURL -- its NTLM authentication works quite well. # /etc/cntlm.conf: # Username account # Domain domain # Password ############ # Proxy 10.20.30.40 (IP of the sharepoint site) # NoProxy * # Listen 3128

kill all processes using a directory/file/etc

This command will kill all processes using a directory. It's quick and dirty. One may also use a -9 with kill in case regular kill doesn't work. This is useful if one needs to umount a directory.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged awk

Commands tagged awk (348) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged awk (348)

sorted by