commandlinefu.com is the place to record those command-line gems that you return to again and again.
Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.
If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/
You can sign-in using OpenID credentials, or register a traditional username and password.
First-time OpenID users will be automatically assigned a username which can be changed after signing in.
Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.
Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):
Subscribe to the feed for:
If you're trying to create a sparse file, you can use dd by 'skip'ing to the last block instance.
ls -ls shows the actual size vs. the reported size
Change /dev/sda with the Disk or partition you want to back-up
To restore use:
dd if=~/backup-disk-YY-MM-DD.img of=/dev/sda
Once again change /dev/sda accordingly
Note: Replace 200000 with drive bytes/512, and /dev/sdx with the destination drive/partition. ;)
Note: You may need to install pipebench, this is easy with "sudo apt-get install pipebench" on Ubuntu.
The reason I hunted around for the pieces to make up this command is that I wanted to specifically flip all of the bits on a new HDD, before running an Extended SMART Self-Test (actually, the second pass, as I've already done one while factory-zeroed) to ensure there are no physical faults waiting to compromise my valuable data. There were several sites that came up in a Google search which had a zero-fill command with progress indicator, and one or two with a fill-with-ones command, but none that I could find with these two things combined (I had to shuffle around the dd command(s) to get this to happen without wasting speed on an md5sum as well).
For reference, these are the other useful-looking commands I found in my search:
Zero-fill drive "/dev/sdx", with progress indicator and md5 verification (run sudo fdisk -l to get total disk bytes, then divide by 512 and enter the resulting value into this command for a full wipe)
dd if=/dev/zero bs=512 count=<size/512> | pipebench | sudo tee /dev/sdx | md5sum
And this command for creating a file filled with ones is my other main source (besides the above command and man pages, that is - I may be a Linux newbie but I do read!):
tr '\000' '\377' < /dev/zero | dd of=allones bs=1024 count=2k
Hope someone finds this useful! :)
This command utilizes 'pv' to show dd's progress.
Notes on use with dd:
-- dd block size (bs=...) is a widely debated command-line switch and should usually be between 1024 and 4096. You won't see much performance improvements beyond 4096, but regardless of the block size, dd will transfer every bit of data.
-- pv's switch, '-s' should be as close to the size of the data source as possible.
-- dd's out file, 'of=...' can be anything as the data within that file are the same regardless of the filename / extension.
Use this if you're using vi editing mode.
Example use :
sudo vim /root/bin/ ##uh... autocomplete doesn't work... dd sudo ls /root/bin
##ah! that's the name of the file!
<p> sudo vim /root/bin/ ##resume here! Thanks readline!
read the memory from C:0000 to F:FFFF
without the need auf dmidecode
The above command will send 4GB of data from one host to the next over the network, without consuming any unnecessary disk on either the client nor the host. This is a quick and dirty way to benchmark network speed without wasting any time or disk space.
Of course, change the byte size and count as necessary.
This command also doesn't rely on any extra 3rd party utilities, as dd, ssh, cat, /dev/zero and /dev/null are installed on all major Unix-like operating systems.
Mac OS X needs some clean up. First command works in Linux, Solaris just needs the "0000000"'s removed
piping through 'pv' shows a simple progress/speed bar for dd. This is a replacement for my otherwise favorite 'while :;do killall -USR1 dd;sleep 1;done'
This line removes the 300k header from a Nero image file converting it to ISO format
bs = buffer size (basically defined the size of a "unit" used by count and skip)
count = the number of buffers to copy (16m * 32 = 1/2 gig)
skip = (32 * 2) we are grabbing piece 3...which means 2 have already been written so skip (2 * count)
i will edit this later if i can to make this all more understandable
This is a more accurate way to watch the progress of a dd process. The $DDPID=$! is needed so that you don't get the PID of the sleep. The sleep 1 is needed because in my testing at least, if you run kill -USR1 against dd too quickly, it will kill it off instead of display the status. So you need to wait a second, probably so that it can configure itself to trap the USR1 signal.
The following command will clone usb stick inside /dev/sdc to /dev/sdd
Double check you got the correct usb sticks (origional-clone)with fdisk -l.
A bit different from some of the other submissions. Has bold and uses all c printable characters. Change the bs=value to speed up and increase the sizes of the bold and non-bold strings.
I wanted to create a copy of my whole laptop disk on an lvm disk of the same size.
First I created the logical volume: lvcreate -L120G -nlaptop mylvms
SOURCE: dd if=/dev/sda bs=16065b | netcat ip-target 1234
TARGET: nc -l -p 1234 | dd of=/dev/mapper/mylvms-laptop bs=16065b
to follow its process you issue the following command in a different terminal
STATS: on target in a different terminal: watch -n60 -- kill -USR1 $(pgrep dd)
If you have some drive imaging to do, you can boot into any liveCD and use a commodity machine. The drives will be written in parallel.
To improve efficiency, specify a larger block size in dd:
dd if=/dev/sda bs=64k | tee >(dd of=/dev/sdb bs=64k) | dd of=/dev/sdc bs=64k
To image more drives , insert them as additional arguments to tee:
dd if=/dev/sda | tee >(dd of=/dev/sdb) >(dd of=/dev/sdc) >(dd of=/dev/sdd) | dd of=/dev/sde
This is similar to how you would generate a file with all zeros
dd if=/dev/zero of=allzeros bs=1024 count=2k
For disk space constraint testing. Leaves a little space available for creating temp files, etc. Easily free up the used disk space again by deleting the dummy00 file. Can tailor the testing by building smaller 'blocks' to suit the needs of the testing.
WARNING: do not do this to the '/' (root) filesystem unless you know what you are doing... on some systems it could crash the OS.