Commands using sed (1,319)

  • Get Lorum Ipsum random text from lorumipsum.com

    This will generate 3 paragraphs with random text. Change the 3 to any number. Show Sample Output


    5
    lynx -source http://www.lipsum.com/feed/xml?amount=3|perl -p -i -e 's/\n/\n\n/g'|sed -n '/<lipsum>/,/<\/lipsum>/p'|sed -e 's/<[^>]*>//g'
    houghi · 2010-04-26 17:26:44 5
  • Prepare a commandlinefu command.

    This command will format your alias or function to a single line, trimming duplicate white space and newlines and inserting delimiter semi-colons, so it continues to work on a single line. Show Sample Output


    5
    goclf() { type "$1" | sed '1d' | tr -d "\n" | tr -s '[:space:]'; echo }
    meathive · 2010-06-26 21:44:17 17
  • Get all shellcode on binary file from objdump

    Tired copy paste to get opcode from objdump huh ? Get more @ http://gunslingerc0de.wordpress.com Show Sample Output


    5
    objdump -d ./PROGRAM|grep '[0-9a-f]:'|grep -v 'file'|cut -f2 -d:|cut -f1-6 -d' '|tr -s ' '|tr '\t' ' '|sed 's/ $//g'|sed 's/ /\\x/g'|paste -d '' -s |sed 's/^/"/'|sed 's/$/"/g'
    gunslinger_ · 2010-07-11 15:44:48 112
  • Convert ascii string to hex

    If you're going to use od, here's how to suppress the labels at the beginning. Also, it doesn't output the \x, hence the sed command at the end. Remove it for space separated hex values instead Show Sample Output


    5
    echo -n "text" | od -A n -t x1 |sed 's/ /\\x/g'
    camocrazed · 2010-07-14 15:31:36 8
  • Remove color codes (special characters) with sed


    5
    sed -r "s:\x1B\[[0-9;]*[mK]::g"'
    roylez · 2010-07-22 05:37:25 6
  • livehttpheaders (firefox addon) replacement

    Default output-file is "liveh.txt". This uses only BRE, in case you're using an older version of sed(1) that doesn't have support for ERE added. With a modern sed(1), to reduce false positive matches, you might do something like: liveh(){ tcpdump -lnnAs512 -i ${1-} tcp |sed 's/.*GET /GET /;s/.*Host: /Host: /;s/.*POST /POST /;/GET |Host: |POST /!d;/[\"'"'"]/d;/\.\./d;w '"${2-liveh.txt}"'' >/dev/null ;} Anyway, it's easy to clean up the output file with sed(1) later.


    5
    liveh(){ tcpdump -lnAs512 ${1-} tcp |sed ' s/.*GET /GET /;s/.*Host: /Host: /;s/.*POST /POST /;/[GPH][EOo][TSs]/!d;w '"${2-liveh.txt}"' ' >/dev/null ;} # usage: liveh [-i interface] [output-file] && firefox &
    argv · 2010-10-11 01:01:11 3
  • Google Translate

    substitute "example" with desired string; tl = target language (en, fr, de, hu, ...); you can leave sl parameter as-is (autodetection works fine) Show Sample Output


    5
    wget -U "Mozilla/5.0" -qO - "http://translate.google.com/translate_a/t?client=t&text=translation+example&sl=auto&tl=fr" | sed 's/\[\[\[\"//' | cut -d \" -f 1
    sairon · 2011-03-06 13:46:16 109
  • remove empty lines in place with backup

    Modify file in place to remove empty lines and create a backup of the original with the extension .bak Show Sample Output


    5
    sed -e '/^$/d' -i .bak filewithempty.lines
    donnoman · 2011-03-24 00:03:25 6
  • Get a BOFH excuse

    Gets a BOFH excuse from the BOFH excuse server (towel.blinkenlights.nl port 666), and passes it through sed and tr to get rid of telnet connection stuff. Show Sample Output


    5
    telnet towel.blinkenlights.nl 666 | sed "s/=== The BOFH Excuse Server ===//" | tr -d '\n' && echo
    hintss · 2011-03-31 05:50:57 3
  • Get International Space Station sighting information for your city

    This command outputs a table of sighting opportunities for the International Space Station. Find the URL for your city here: http://spaceflight.nasa.gov/realdata/sightings/ Show Sample Output


    5
    links -dump "http://spaceflight.nasa.gov/realdata/sightings/cities/view.cgi?country=United_States&region=Wisconsin&city=Portage" | sed -n '/--/,/--/p'
    eightmillion · 2011-05-03 12:15:56 7
  • full memcache client in under 255 chars (uses dd, sed and nc)

    usage: mem memcache-command [arguments] where memcache-command might be: set add get[s] append prepend replace delete incr decr cas stats verbosity version notes: exptime argument is set to 0 (no expire) flags argument is set to 1 (arbitrary)


    5
    mem(){ { case $1 in st*|[vgid]*) printf "%s " "$@";; *) dd if=$3 2>&1|sed '$!d;/^0/d;s/ .*//;s/^/'"$1"' '"$2"' 1 0 /; r '"$3"'' 2>/dev/null;;esac;printf "\r\nquit\r\n";}|nc -n 127.0.0.1 11211; }
    argv · 2011-06-17 06:39:07 6
  • Prepend a text to a file.

    Using the sed -i (inline), you can replace the beginning of the first line of a file without redirecting the output to a temporary location.


    5
    sed -i '1s/^/text to prepend\n/' file1
    xeor · 2011-06-25 12:02:11 14
  • Search and Replace across multiple files

    - grep for the word in a files, use recursion (to find files in sub directories), and list only file matches -| xargs passes the results from the grep command to sed -sed -i uses a regular expression (regex) to evaluate the change: s (search) / search word / target word / g (global replace)


    5
    grep -lr -e '<oldword>' * | xargs sed -i 's/<oldword>/<newword>/g'
    o0110o · 2011-08-27 02:42:51 7
  • regex for turning a URL into a real hyperlink (i.e. for posting somewhere that accepts basic html)

    This should work with anything://url.whatever etc etc ;)


    5
    sed "s/\([a-zA-Z]*\:\/\/[^ ]*\)\(.*\)/\<a href=\"\1\"\>\1\<\/a\>\2/"
    c3w · 2012-01-04 02:08:29 7
  • Use top to monitor only all processes with the same name fragment 'foo'

    pgrep foo may return several pids for process foobar footy01 etc. like this: 11427 12576 12577 sed puts "-p " in front and we pass a list to top: top -p 11427 -p 12576 -p 12577


    5
    top $(pgrep foo | sed 's|^|-p |g')
    michelsberg · 2012-06-14 15:13:00 3
  • run shell with your commandlinefu.com's favourites as bash_history

    This makes your commandlinefu.com's favorites appear as most recent commands in your history.


    5
    (cat ~/.bash_history;U='curl -s www.commandlinefu.com';$U/users/signin -c/tmp/.c -d'username=<USER>&password=<PASS>&submit=1'|$U/commands/favourites/json -b/tmp/.c|grep -Po 'nd":.*?[^\\]",'|sed -re 's/.*":"(.*)",/\1/g')>~/.h;HISTFILE=~/.h bash --login
    xenomuta · 2012-08-17 12:31:51 34
  • Encrypt and password-protect execution of any bash script

    (Please see sample output for usage) script.bash is your script, which will be crypted to script.secure script.bash --> script.secure You can execute script.secure only if you know the password. If you die, your script dies with you. If you modify the startup line, be careful with the offset calculation of the crypted block (the XX string). Not difficult to make script editable (an offset-dd piped to a gpg -d piped to a vim - piped to a gpg -c directed to script.new ), but not enough space to do it on a one liner. Show Sample Output


    5
    echo "eval \"\$(dd if=\$0 bs=1 skip=XX 2>/dev/null|gpg -d 2>/dev/null)\"; exit" > script.secure; sed -i s:XX:$(stat -c%s script.secure): script.secure; gpg -c < script.bash >> script.secure; chmod +x script.secure
    rodolfoap · 2013-03-09 11:16:48 20
  • Graphically show percent of mount space used

    Automatically drops mount points that have non-numeric sizes (e.g. /proc). Tested in bash on Linux and AIX. Show Sample Output


    5
    for m in `df -P | awk -F ' ' '{print $NF}' | sed -e "1d"`;do n=`df -P | grep "$m$" | awk -F ' ' '{print $5}' | cut -d% -f1`;i=0;if [[ $n =~ ^-?[0-9]+$ ]];then printf '%-25s' $m;while [ $i -lt $n ];do echo -n '=';let "i=$i+1";done;echo " $n";fi;done
    drockney · 2013-07-29 20:12:39 6
  • Convert a script to one-liner

    Opposite: Convert an one-liner to script: foo() { <one-liner> ; } ... typeset -f foo ... unset -f foo


    5
    (sed 's/#.*//g'|sed '/^ *$/d'|tr '\n' ';'|xargs echo) < script.sh
    knoppix5 · 2013-10-26 23:23:51 16
  • Shows cpu load in percent

    Show the current load of the CPU as a percentage. Read the load from /proc/loadavg and convert it using sed: Strip everything after the first whitespace: sed -e 's/ .*//' Delete the decimal point: sed -e 's/\.//' Remove leading zeroes: sed -e 's/^0*//' Show Sample Output


    5
    sed -e 's/ .*//' -e 's/\.//' -e 's/^0*//' /proc/loadavg
    flatcap · 2014-04-18 19:12:05 10
  • Get simple weather info from a zip code

    weather 97405 Show Sample Output


    5
    weather() { curl -s "http://www.wunderground.com/q/zmw:$1.1.99999" | grep "og:title" | cut -d\" -f4 | sed 's/&deg;/ degrees F/'; }
    cadejscroggins · 2015-09-19 07:57:47 12
  • Change every instance of OLD to NEW in file FILE

    Very quick way to change a word in a file. I use it all the time to change variable names in my PHP scripts (sed -i 's/$oldvar/$newvar/g' index.php)


    4
    sed -i 's/OLD/NEW/g' FILE
    nanexcool · 2009-02-05 18:07:41 26
  • Delete line number 10 from file

    Very useful when the ssh key of a host has changed and ssh refuses to connect to the machine, while giving you the line number that has changed in ~/.ssh/known_hosts.


    4
    sed -i '10d' <somefile>
    raphink · 2009-02-16 15:05:29 160
  • remove leading blank lines


    4
    sed '/./,$!d'
    grep · 2009-02-16 20:39:40 9
  • Convert Unix newlines to DOS newlines

    The ctrl+v,ctrl+m portion represents key presses that you should do. If you do it successfully you should see a ^M character appear.


    4
    sed 's/$/<ctrl+v><ctrl+m>/'
    SiegeX · 2009-02-16 20:53:05 12
  • ‹ First  < 4 5 6 7 8 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

list all files in a directory, sorted in reverse order by modification time, use file descriptors.
It's both silly, and infinitely useful. Especially useful in logfile directories where you want to know what file is being updated while troubleshooting.

Convert seconds to [DD:][HH:]MM:SS
Converts any number of seconds into days, hours, minutes and seconds. sec2dhms() { declare -i SS="$1" D=$(( SS / 86400 )) H=$(( SS % 86400 / 3600 )) M=$(( SS % 3600 / 60 )) S=$(( SS % 60 )) [ "$D" -gt 0 ] && echo -n "${D}:" [ "$H" -gt 0 ] && printf "%02g:" "$H" printf "%02g:%02g\n" "$M" "$S" }

BASH: Print shell variable into AWK

See how many more processes are allowed, awesome!
There is a limit to how many processes you can run at the same time for each user, especially with web hosts. If the maximum # of processes for your user is 200, then the following sets OPTIMUM_P to 100. $ OPTIMUM_P=$(( (`ulimit -u` - `find /proc -maxdepth 1 \( -user $USER -o -group $GROUPNAME \) -type d|wc -l`) / 2 )) This is very useful in scripts because this is such a fast low-resource-intensive (compared to ps, who, lsof, etc) way to determine how many processes are currently running for whichever user. The number of currently running processes is subtracted from the high limit setup for the account (see limits.conf, pam, initscript). An easy to understand example- this searches the current directory for shell scripts, and runs up to 100 'file' commands at the same time, greatly speeding up the command. $ find . -type f | xargs -P $OPTIMUM_P -iFNAME file FNAME | sed -n '/shell script text/p' I am using it in my http://www.askapache.com/linux-unix/bash_profile-functions-advanced-shell.html especially for the xargs command. Xargs has a -P option that lets you specify how many processes to run at the same time. For instance if you have 1000 urls in a text file and wanted to download all of them fast with curl, you could download 100 at a time (check ps output on a separate [pt]ty for proof) like this: $ cat url-list.txt | xargs -I '{}' -P $OPTIMUM_P curl -O '{}' I like to do things as fast as possible on my servers. I have several types of servers and hosting environments, some with very restrictive jail shells with 20processes limit, some with 200, some with 8000, so for the jailed shells my xargs -P10 would kill my shell or dump core. Using the above I can set the -P value dynamically, so xargs always works, like this. $ cat url-list.txt | xargs -I '{}' -P $OPTIMUM_P curl -O '{}' If you were building a process-killer (very common for cheap hosting) this would also be handy. Note that if you are only allowed 20 or so processes, you should just use -P1 with xargs.

Count the total number of files in each immediate subdirectory
counts the total (recursive) number of files in the immediate (depth 1) subdirectories as well as the current one and displays them sorted. Fixed, as per ashawley's comment

for too many arguments by *
$ grep ERROR *.log -bash: /bin/grep: Argument list too long $ echo *.log | xargs grep ERROR /dev/null 20090119.00011.log:DANGEROUS ERROR

get total of inodes of root partition

A signal trap that logs when your script was killed and what other processes were running at that time
trap is the bash builtin that allows you to execute commands when the current script receives a particular signal. Uses $0 for the script name, $$ for the script PID, tee to output to STDOUT as well as a log file and ps to log other running processes.

take a look to command before action
add |sh when you agree the list, I often use that method to prevent typos in dangerous or long operations

Get gzip compressed web page using wget.
Like the original command, but the -f allows this one to succeed even if the website returns uncompressed data. From gzip(1) on the -f flag: If the input data is not in a format recognized by gzip, and if the --stdout is also given, copy the input data without change to the standard output: let zcat behave as cat.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: