Commands using dd (167)

  • This invokes tar on the remote machine and pipes the resulting tarfile over the network using ssh and is saved on the local machine. This is useful for making a one-off backup of a directory tree with zero storage overhead on the source. Variations on this include using compression on the source by using 'tar cfvp' or compression at the destination via ssh user@host "cd dir; tar cfp - *" | gzip - > file.tar.gz


    6
    ssh user@host "cd targetdir; tar cfp - *" | dd of=file.tar
    bwoodacre · 2009-03-18 07:43:22 8
  • The above command will send 4GB of data from one host to the next over the network, without consuming any unnecessary disk on either the client nor the host. This is a quick and dirty way to benchmark network speed without wasting any time or disk space. Of course, change the byte size and count as necessary. This command also doesn't rely on any extra 3rd party utilities, as dd, ssh, cat, /dev/zero and /dev/null are installed on all major Unix-like operating systems. Show Sample Output


    6
    dd if=/dev/zero bs=4096 count=1048576 | ssh user@host.tld 'cat > /dev/null'
    atoponce · 2010-06-08 18:49:51 10
  • An easy method to generate ISOs from CD/DVD media.


    6
    dd if=/dev/cdrom of=~/cdrom_image.iso
    o0110o · 2012-07-10 06:03:25 10
  • This example will close the pipe after transferring 100MB at a speed of 3MB per second.


    6
    cat /dev/urandom | pv -L 3m | dd bs=1M count=100 iflag=fullblock > /dev/null
    bugmenot · 2012-07-29 00:42:16 10
  • This is just a proof of concept: A FILE WHICH CAN AUTOMOUNT ITSELF through a SIMPLY ENCODED script. It takes advantage of the OFFSET option of mount, and uses it as a password (see that 9191? just change it to something similar, around 9k). It works fine, mounts, gets modified, updated, and can be moved by just copying it. USAGE: SEE SAMPLE OUTPUT The file is composed of three parts: a) The legible script (about 242 bytes) b) A random text fill to reach the OFFSET size (equals PASSWORD minus 242) c) The actual filesystem Logically, (a)+(b) = PASSWORD, that means OFFSET, and mount uses that option. PLEASE NOTE: THIS IS NOT AN ENCRYPTED FILESYSTEM. To improve it, it can be mounted with a better encryption script and used with encfs or cryptfs. The idea was just to test the concept... with one line :) It applies the original idea of http://www.commandlinefu.com/commands/view/7382/command-for-john-cons for encrypting the file. The embedded bash script can be grown, of course, and the offset recalculation goes fine. I have my own version with bash --init-file to startup a bashrc with a well-defined environment, aliases, variables. Show Sample Output


    6
    dd if=/dev/zero of=T bs=1024 count=10240;mkfs.ext3 -q T;E=$(echo 'read O;mount -o loop,offset=$O F /mnt;'|base64|tr -d '\n');echo "E=\$(echo $E|base64 -d);eval \$E;exit;">F;cat <(dd if=/dev/zero bs=$(echo 9191-$(stat -c%s F)|bc) count=1) <(cat T;rm T)>>F
    rodolfoap · 2013-01-31 01:38:30 13
  • This shell snippet reads a single keypress from stdin and stores it in the $KEY variable. You do NOT have to press the enter key! The key is NOT echoed to stdout! This is useful for implementing simple text menus in scripts and similar things.


    5
    stty cbreak -echo; KEY=$(dd bs=1 count=1 2>/dev/null); stty -cbreak echo
    inof · 2009-06-09 13:15:49 9

  • 5
    dd bs=1 seek=2TB if=/dev/null of=ext3.test
    Superhuman · 2009-09-03 08:35:20 3
  • This command clone the first partition of the primary master IDE drive to the second partition of the primary slave IDE drive (!!! back up all data before trying anything like this !!!)


    5
    sudo dd if=/dev/hda1 of=/dev/hdb2
    0disse0 · 2009-09-05 09:16:52 4
  • You can use this to directly dump from machine A (with dvd drive) to machine B (without dvd drive) . I used this to copy dvd using my friend's machine to my netbook. Above command is to be issued on machine B. Advantages : 1) No wasting time dumping first to machine A and then copying to Machine B. 2) You dont need to use space on Machine A. In fact, this will work even when Machine A doesnt have enough hdd space to dump the DVD. Use -C ssh option on slow networks (enables compression). you can replace "dd if=/dev/dvd" with any ripping command as long as it spews the iso to stdout.


    5
    ssh user@machine_A dd if=/dev/dvd0 > dvddump.iso
    kamathln · 2009-09-11 18:08:36 12
  • A bit different from some of the other submissions. Has bold and uses all c printable characters. Change the bs=value to speed up and increase the sizes of the bold and non-bold strings.


    5
    echo -ne "\e[32m" ; while true ; do echo -ne "\e[$(($RANDOM % 2 + 1))m" ; tr -c "[:print:]" " " < /dev/urandom | dd count=1 bs=50 2> /dev/null ; done
    psykotron · 2009-12-19 19:05:04 4

  • 5
    dd if=/dev/zero of=foo.txt bs=1M count=1
    EBAH · 2010-10-20 09:36:34 3
  • Install a basic FreeBSD system on a distant server. I use this to install FreeBSD on servers that can only boot a Linux rescue system. This sytem loads on ram when booted, so it is possible to install freely. You can even install on ZFS root !


    5
    dd if=mfsbsd.iso | ssh distant.server dd of=/dev/sda
    gormux · 2010-12-23 09:34:33 10
  • usage: mem memcache-command [arguments] where memcache-command might be: set add get[s] append prepend replace delete incr decr cas stats verbosity version notes: exptime argument is set to 0 (no expire) flags argument is set to 1 (arbitrary)


    5
    mem(){ { case $1 in st*|[vgid]*) printf "%s " "$@";; *) dd if=$3 2>&1|sed '$!d;/^0/d;s/ .*//;s/^/'"$1"' '"$2"' 1 0 /; r '"$3"'' 2>/dev/null;;esac;printf "\r\nquit\r\n";}|nc -n 127.0.0.1 11211; }
    argv · 2011-06-17 06:39:07 6

  • 5
    sudo dd if=/dev/sdc bs=4096 | pv -s `sudo mount /dev/sdc /media/sdc && du -sb /media/sdc/ |awk '{print $1}' && sudo umount /media/sdc`| sudo dd bs=4096 of=~/USB_BLACK_BACKUP.IMG
    juanmi · 2013-02-06 09:29:10 7
  • This is similar to how you would generate a file with all zeros dd if=/dev/zero of=allzeros bs=1024 count=2k


    4
    tr '\000' '\377' < /dev/zero | dd of=allones bs=1024 count=2k
    azeey · 2009-12-08 16:05:28 4
  • This is a more accurate way to watch the progress of a dd process. The $DDPID=$! is needed so that you don't get the PID of the sleep. The sleep 1 is needed because in my testing at least, if you run kill -USR1 against dd too quickly, it will kill it off instead of display the status. So you need to wait a second, probably so that it can configure itself to trap the USR1 signal. Show Sample Output


    4
    dd if=fromfile of=tofile & DDPID=$! ; sleep 1 ; while kill -USR1 $DDPID ; do sleep 5; done
    deltaray · 2010-01-12 15:01:44 4
  • Useful for testing purposes


    4
    dd if=/dev/zero of=testfile bs=1024 count=5000
    mpathi · 2011-03-25 09:08:50 3
  • the speed is about 500MB/s on my machine. i think it's fast enough to output not too many bytes. while a C program may output 1GB per sencond on my machine. if the size is not the power of 512,you may change the bs and count in dd. Show Sample Output


    4
    tr '\0' '\377' < /dev/zero|dd count=$((<bytes>/512))
    cfy · 2011-04-05 14:26:02 3
  • create an iso from your cd/dvd-rom device . You need to umount /dev/cdrom before using the cli Show Sample Output


    4
    dd if=/dev/cdrom of=~/cd_image.iso
    eastwind · 2011-09-24 15:27:04 6
  • Your platform may not have pv by default. If you are using Homebew on OSX, simply 'brew install pv'. Show Sample Output


    4
    pv -tpreb /dev/sda | dd of=/dev/sdb bs=1M
    sc0ttyd · 2013-08-19 23:04:15 7

  • 3
    dd if=/dev/random of=bigfile bs=1024 count=102400
    amiga500 · 2009-02-17 05:41:20 8
  • See: http://imgur.com/JgjK2.png for example. Do some serious benchmarking from the commandline. This will write to a file with the time it took to compress n bytes to the file (increasing by 1). Run: gnuplot -persist <(echo "plot 'lzma' with lines, 'gzip' with lines, 'bzip2' with lines") To see it in graph form.


    3
    for a in bzip2 lzma gzip;do echo -n>$a;for b in $(seq 0 256);do dd if=/dev/zero of=$b.zero bs=$b count=1;c=$(date +%s%N);$a $b.zero;d=$(date +%s%N);total=$(echo $d-$c|bc);echo $total>>$a;rm $b.zero *.bz2 *.lzma *.gz;done;done
    matthewbauer · 2009-10-20 01:00:51 5
  • The following command will clone usb stick inside /dev/sdc to /dev/sdd Double check you got the correct usb sticks (origional-clone)with fdisk -l.


    3
    dd if=/dev/sdc of=/dev/sdd conv=notrunc & while killall -USR1 dd; do sleep 5; done
    bw · 2010-01-12 14:09:40 134
  • Note: Replace 200000 with drive bytes/512, and /dev/sdx with the destination drive/partition. ;) Note: You may need to install pipebench, this is easy with "sudo apt-get install pipebench" on Ubuntu. The reason I hunted around for the pieces to make up this command is that I wanted to specifically flip all of the bits on a new HDD, before running an Extended SMART Self-Test (actually, the second pass, as I've already done one while factory-zeroed) to ensure there are no physical faults waiting to compromise my valuable data. There were several sites that came up in a Google search which had a zero-fill command with progress indicator, and one or two with a fill-with-ones command, but none that I could find with these two things combined (I had to shuffle around the dd command(s) to get this to happen without wasting speed on an md5sum as well). For reference, these are the other useful-looking commands I found in my search: Zero-fill drive "/dev/sdx", with progress indicator and md5 verification (run sudo fdisk -l to get total disk bytes, then divide by 512 and enter the resulting value into this command for a full wipe) dd if=/dev/zero bs=512 count=<size/512> | pipebench | sudo tee /dev/sdx | md5sum And this command for creating a file filled with ones is my other main source (besides the above command and man pages, that is - I may be a Linux newbie but I do read!): tr '\000' '\377' < /dev/zero | dd of=allones bs=1024 count=2k Hope someone finds this useful! :) Cheers, - Gliktch Show Sample Output


    3
    tr '\000' '\377' < /dev/zero | dd bs=512 count=200000 status=noxfer | pipebench | sudo dd of=/dev/sdx
    Gliktch · 2010-08-31 15:38:27 6
  • Adjust "sleep X" to your needs. *NOTE: First sleep is required because bash doesn't have a "post-test" syntax (do XXX while). Show Sample Output


    3
    dd if=/path/to/inputfile of=/path/to/outputfile & pid=$! && sleep X && while kill -USR1 $pid; do sleep X; done
    cyrusza · 2010-12-02 15:07:18 7
  •  < 1 2 3 4 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Change user within ssh session retaining the current MIT cookie for X-forwarding
When you remotely log in like "ssh -X userA:host" and become a different user with "su UserB", X-forwarding will not work anymore since /home/UserB/.Xauthority does not exist. This will use UserA's information stored in .Xauthority for UserB to enable X-forwarding. Watch http://prefetch.net/blog/index.php/2008/04/05/respect-my-xauthority/ for details.

Open Remote Desktop (RDP) from command line and connect local resources
The above command will open a Remote Desktop connection from command line, authenticate using default username and password (great for virtual machines; in the exampe above it's administrator:password), create a shared folder between your machine and the other machine and configure resolution to best fit your desktop (I don't like full screen because it make the desktop panels to disappear). The command will run in the background, and expect to receive parameters. You should enter hostname or IP address as a parameter to the command, and can also override the defaults parameters with your own.

High resolution video screen recording
$ gorecord foo.mp4 I've tried all of the screen recorders available for Linux and this is easily the best. xvidcap segfaults; VNC is too much hassle. There are alternatives of this command already here that I am just too lazy to reply to. Messing with the frames per second option, -r, 25 seems to be the best. Any lower and the video will look like a flipbook, if it records at all - -r 10 won't - any faster is the same, oddly enough. Edit: CLF doesn't like my long command to add audio, so here it is in the description. $ goaddaudio() ${ $if [ $# != 3 ]; then $ echo 'goaddaudio < audio > < src video > < dst video >' $ return $ fi $ $ f=goaddaudio$RANDOM $ ffmpeg -i "$2" &> $f $ d=$( grep Duration $f | awk '{print $2}' | tr -d ',' ) && $ rm $f && $ ffmpeg -i "$1" -i "$2" -r 25 -ab 192k -ar 44100 -sameq -t $d "$3" $}

Remove job from crontab by commandline
The "-u USER" is optional if root user is used

Do quick arithmetic on numbers from STDIN with any formatting using a perl one liner.
Good for summing the numbers embedded in text - a food journal entry for example with calories listed per food where you want the total calories. Use this to monitor and keep a total on anything that ouputs numbers.

Create the oauth token required for a Twitter stream feed
This is the THIRD in a set of five commands. See my other commands for the previous two. This step creates the oauth 1.0 token as explained in http://oauth.net/core/1.0/ The token is required for a Twitter filtered stream feed (and almost all Twitter API calls) This token is simply an encrypted version of your base string. The encryption key used is your hmac. The last part of the command scans the Base64 token string for '+', '/', and '=' characters and converts them to percentage-hex escape codes. (URI-escapeing). This is also a good example of where the $() syntax of Bash command substitution fails, while the backtick form ` works - the right parenthesis in the case statement causes a syntax error if you try to use the $() syntax here. See my previous two commands step1 and step2 to see how the base string variable $b and hmac variable $hmac are generated.

count how many times a string appears in a (source code) tree
grep -o puts each occurrence in a separate line

Mount directories in different locations
Like symlinked directories, you can mount a directory at a different location. For example mounting a directory from one location in to the http root without having to make your program follow symlinks or change permissions when reading.

Lookup hostname for IP address

Find the package that installed a command


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: