Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

2011-03-12 - Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands
To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner
Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed
Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.
Hide

Tags

Hide

Functions

Commands using read from sorted by
Terminal - Commands using read - 282 results
lsof -n -P|grep FlashXX|awk '{ print "/proc/" $2 "/fd/" substr($4, 1, length($4)-1) }'|while read f;do newname=$(exiftool -FileModifyDate -FileType -t -d %Y%m%d%H%M%S $f|cut -f2|tr '\n' '.'|sed 's/\.$//');echo "$f -> $newname";cp $f ~/Vids/$newname;done
2012-02-25 01:49:45
User: mhs
Functions: awk cp cut echo grep read sed tr
8

Certain Flash video players (e.g. Youtube) write their video streams to disk in /tmp/ , but the files are unlinked. i.e. the player creates the file and then immediately deletes the filename (unlinking files in this way makes it hard to find them, and/or ensures their cleanup if the browser or plugin should crash etc.) But as long as the flash plugin's process runs, a file descriptor remains in its /proc/ hierarchy, from which we (and the player) still have access to the file. The method above worked nicely for me when I had 50 tabs open with Youtube videos and didn't want to have to re-download them all with some tool.

while read ; do python <script> ; done
2012-02-23 22:29:09
User: Zulu
Functions: python read
-2

Very useful for test a script. After launch this command, you only have to press ENTER for launch your script again. I work with screen and tape ENTER instead of '!!'+ENTER

If you break your script with CTRL-C, it will wait for press ENTER and will re-launch

You can write like it : while read -p "Press ENTER" ; do python ; done

ls|grep .mp3 >list.txt; while read line; do newname=`echo $line|sed 's/\ /-/g'|sort`; newname=`echo $newname|tr -s '-' `; echo $newname; echo $newname>> tracklist.txt;mv "$line" "$newname"; done <list.txt; rm list.txt
echo -n "IP Address or Machine Name: "; read IP; ping -c 1 -q $IP >/dev/null 2>&1 && echo -e "\e[00;32mOnline\e[00m" || echo -e "\e[00;31mOffline\e[00m"
2012-02-09 07:00:03
User: crlf
Functions: echo ping read
Tags: bash echo IP ping
1

I have used single packet, and in a silent mode with no display of ping stats. This is with color and UI improvement to the http://www.commandlinefu.com/commands/view/10220/check-if-a-machine-is-online. It is as per the enhancements suggested.

fdupes -R -1 path | while read -r line; do (echo $line | xargs -n 1 | (first="true"; firstfile=""; while read file; do if [ "$first" == "true" ]; then first="false"; firstfile=$file; else ln --force "$firstfile" "$file"; fi; done)); done
2012-02-01 15:08:18
User: eclewis
Functions: echo ln read xargs
1

This variation can handle file paths containing spaces.

fdupes -r -1 Neu | while read line; do j="0"; buf=""; for file in ${line[*]}; do if [ "$j" == "0" ]; then j="1"; buf=$file; else ln -f $buf $file; fi; done; done
for fn in xkcd*.png xkcd*.jpg; do echo $fn; read xw xh <<<$(identify -format '%w %h' $fn); nn="$(echo $fn | sed 's/xkcd-\([^-]\+\)-.*/\1/')"; wget -q -O xkcd-${nn}.json http://xkcd.com/$nn/info.0.json; tt="$(sed 's/.*"title": "\([^"]\+\)",.*/\1/' ...
2012-01-06 20:26:11
User: fpunktk
Functions: echo read wget
-2

full command:

for fn in xkcd*.png xkcd*.jpg; do; echo $fn; read xw xh <<<$(identify -format '%w %h' $fn); nn="$(echo $fn | sed 's/xkcd-\([0-9]\+\)-.*/\1/')"; wget -q -O xkcd-${nn}.json http://xkcd.com/$nn/info.0.json; tt="$(sed 's/.*"title": "\([^"]*\)", .*/\1/' xkcd-${nn}.json)"; at="$(sed 's/.*alt": "\(.*\)", .*/\1/' xkcd-${nn}.json)"; convert -background white -fill black -font /usr/share/fonts/truetype/freefont/FreeSansBold.ttf -pointsize 26 -size ${xw}x -gravity Center caption:"$tt" tt.png; convert -background '#FFF9BD' -border 1x1 -bordercolor black -fill black -font /usr/share/fonts/truetype/freefont/FreeSans.ttf -pointsize 16 -size $(($xw - 2))x -gravity Center caption:"$at" at.png; th=$(identify -format '%h' tt.png); ah=$(identify -format '%h' at.png); convert -size ${xw}x$(($xh+$th+$ah+5)) "xc:white" tt.png -geometry +0+0 -composite $fn -geometry +0+$th -composite at.png -geometry +0+$(($th+$xh+5)) -composite ${fn%\.*}_cmp.png; echo -e "$fn $nn $xw $xh $th $ah \n$tt \n$at\n"; done

this assumes that all comics are saved as xkcd-[number]-[title].{png|jpg}.

it will then download the title and alt-text, create pictures from them, and put everything together in a new png-file.

it's not perfect, but it worked for nearly all my comics.

it uses the xkcd-json-interface.

though it's poorly written, it doesn't completely break on http://xkcd.com/859/

vgdisplay -v 2>/dev/null | grep "^ LV Name" | while read A B LVDEV; do echo $LVDEV; done
while read l; do echo -e "$RANDOM\t$l"; done | sort -n | cut -f 2
c="cp -a";e="~";echo -e "\npaste\n";i=0;k="1"; while [[ "$k" != "" ]]; do read -a k;r[i]=$k;((i++));done;i=0;while :;do t=${r[i]};[ "$t" == "" ] && break; g=$(echo $c ${r[i]} $e);echo -e $g "\ny/n?";read y;[ "$y" != "n" ] && eval $g;((i++));done
2011-12-04 12:45:44
User: knoppix5
Functions: echo eval read
-1

Schematics:

command [options] [paste your variable here] parameter

command [options] [paste entire column of variables here] parameter

...

(hard-code command "c" and parameter "e" according to your wishes: in example shown command = "cp -a" and parameter = "~")

Features:

- Quick exchange only variable part of a long command line

- Make variable part to be an entire column of data (i.e. file list)

- Full control while processing every single item

Hints:

Paste column of data from anywhere. I.e. utilize the Block Select Mode to drag, select and copy columns (In KDE Konsole with Ctrl+Alt pressed, or only Ctrl pressed in GNOME Terminal respectively).

Disadvantages:

You can paste only one single variable in a row. If there are more space separated variables in a row only first one will be processed, but you can arrange your variables in a column instead. To transpose rows to columns or vice versa look at Linux manual pages for 'cut' and 'paste'.

TODO:

- add edit mode to vary command "c" and parameter "e" on the fly

- add one edit mode more to handle every list item different

- add y/n/a (=All) instead of only y(=default)/n to allowed answers

Disclaimer:

The code is not optimized, only the basic idea is presented here. It's up to you to shorten code or extend the functionality.

read -s PASS; echo $PASS | convert sensitive.jpg -encipher - -depth 8 png24:hidden.png
2011-11-25 18:15:23
User: kev
Functions: echo read
Tags: ImageMagick
0

Do not use JPEG, GIF, or any other 'lossy' image encoding with Encryption

read -s pass; echo $pass | md5sum | base64 | cut -c -16
2011-11-24 20:23:47
User: bugmenot
Functions: cut echo md5sum read
21

Why remember? Generate!

Up to 48 chars, works on any unix-like system (NB: BSD use md5 instead of md5sum)

read c; while [ -n "$c" ]; do clear; echo -e "$c = "$(echo "$c" |bc -l)"\n"; read c; done
read -d ""
2011-10-25 18:44:11
User: totti
Functions: read
1

Handle any bad named file which contains ",',\n,\b,\t,` etc

Store the file name as null character separated list

find . -print0 >name.lst

and retrieve it using

read -r -d ""

Eg:

find . -print0 >name.lst; cat name.lst| while IFS="" read -r -d "" file; do ls -l "$file"; done
while read line; do echo $line; done <<< "$var"
2011-09-22 16:53:32
User: totti
Functions: echo read
-5

Consider the following simple situation [ reading something using while and read ]

[See script 1 in sample output]

---------------------------------------------------

The variable var is assigned with "nullll" at first. Inside the while loop [piped while] it is assigned with "whillleeee". [Onlly 2 assignments stmts]. Outside the loop the last assigned value for "var" [and no variable] inside the while can't be accessed [Due to pipe, var is executed in a sub shell].

In these type of situation variables can be accessed by modifying as follows.

[See script 2 in sample output]

___________________________

Vary helpful when reading a set of items, say file names, stored on a file [or variable] to an array an use it later.

Is there any other way 2 access variables inside and outside the loop ??

find . -printf "%T@ %p\n" | sed -e 1d | while read ts fn; do ts=${ts%.*}; if [ $ts -ge ${gts:-0} ]; then gts=$ts; echo `date -d @$gts` $fn; fi; done
read -ra words <<< "<sentence>" && echo "${words[@]^}"
param=${param:-$(read -p "Enter parameter: "; echo "$REPLY")}
2011-09-08 20:48:31
User: frans
Functions: echo read
8

Can be used for command line parameters too.

If you have a more complicated way of entering values (validation, GUI, ...), then write a function i.e. EnterValue() that echoes the value and then you can write:

param=${param:-$(EnterValue)}
ping HOSTNAME | while read pong; do echo "$(date): $pong"; done
get_duration () { IFS=.: read -r _ h m s _ < <(ffmpeg -i "$1" 2>&1 | grep Duration);echo $(( h * 3600 + m * 60 + s )); }
SCALE=3; WIDTHL=10; WIDTHR=60; BAR="12345678"; BAR="${BAR//?/==========}"; while read LEFT RIGHT rest ; do RIGHT=$((RIGHT/SCALE)); printf "%${WIDTHL}s: %-${WIDTHR}s\n" "${LEFT:0:$WIDTHL}" "|${BAR:0:$RIGHT}*"; done < dataset.dat
2011-08-22 19:35:21
User: andreasS
Functions: printf read
0

WIDTHL=10 and WIDTHR=60 are setting the widths of the left and the right column/bar. BAR="12345678" etc. is used to create a 80 char long string of "="s. I didn't know any shorter way.

If you want to pipe results into it, wrap the whole thing in ( ... )

I know that printing bar graphs can be done rather easily by other means. Here, I was looking for a Bash only variant.

while IFS= read -r -u3 -d $'\0' file; do file "$file" | egrep -q 'executable|ELF' && chmod +x "$file"; done 3< <(find . -type f -print0)
2011-08-18 15:37:23
User: keymon
Functions: chmod egrep file find read
0

If you make a mess (like I did) and you removed all the executable permissions of a directory (or you set executable permissions to everything) this can help.

It supports spaces and other special characters in the file paths, but it will work only in bash, GNU find and GNU egrep.

You can complement it with these two commands:

1. add executable permission to directories:

find . type d -print0 | xargs -0 chmod +x

2. and remove to files:

find . type d -print0 | xargs -0 chmod -x

Or, in the same loop:

while IFS= read -r -u3 -d $'\0' file; do case $(file "$file" | cut -f 2- -d :) in :*executable*|*ELF*|*directory*) chmod +x "$file" ;; *) chmod -x "$file" ;; esac || break done 3< <(find . -print0)

Ideas stolen from Greg's wiki: http://mywiki.wooledge.org/BashFAQ/020

git branch | cut -c3- | grep -v "^master$" | while read line; do git branch -d $line; done | grep 'Deleted branch' | awk '{print $3;}' | while read line; do git push <target_remote> :$line; done
2011-08-13 16:58:34
User: gocoogs
Functions: awk cut grep read
0

attempts to delete all local branches. git will fail on any branches not fully merged into local master, so don't worry about losing work. git will return the names of any successfully deleted branches. Find those in the output with grep, then push null repositories to the corresponding names to your target remote.

assumes:

- your local and remote branches are identically named, and there's nothing extra in the remote branch that you still want

- EDIT: you want to keep your local master branch

find . -type l | (while read FN ; do test -e "$FN" || ls -ld "$FN"; done)
read day month year < <(date +'%d %m %y')
2011-07-30 06:06:29
User: frans
Functions: date read
Tags: bash read
10

No command substitution but subshell redirection