Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Universal configuration monitoring and system of record for IT.
Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

May 19, 2015 - A Look At The New Commandlinefu
I've put together a short writeup on what kind of newness you can expect from the next iteration of clfu. Check it out here.
March 2, 2015 - New Management
I'm Jon, I'll be maintaining and improving clfu. Thanks to David for building such a great resource!
Hide

Top Tags

Hide

Functions

Commands using rsync from sorted by
Terminal - Commands using rsync - 76 results
rsync -a --delete empty-dir/ target-dir/
2016-06-07 16:56:55
User: malathion
Functions: rsync
Tags: delete rsync
7

This command works by rsyncing the target directory (containing the files you want to delete) with an empty directory. The '--delete' switch instructs rsync to remove files that are not present in the source directory. Since there are no files there, all the files will be deleted.

I'm not clear on why it's faster than 'find -delete', but it is.

Benchmarks here: https://web.archive.org/web/20130929001850/http://linuxnote.net/jianingy/en/linux/a-fast-way-to-remove-huge-number-of-files.html

rsync -avxHAXW --info=progress2 /old-disk /new-disk/
blank=$(mktemp -d); rsync --delete "$blank/" "bigdir/"; rmdir "$blank"
2016-02-15 11:23:15
User: Natureshadow
Functions: mktemp rmdir rsync
Tags: rsync rm
4

rsync'ing an empty directory over a directory to be deleted recursively is much faster than using rm -rf, for various reasons. Relevant only for directories with really a lot of files.

rsync -a --progress -e 'ssh -p 2200 -i .vagrant/machines/default/virtualbox/private_key' [email protected]:/vagrant/vm/old_timecapsule_backup /Volumes/2TB/
2015-12-17 15:28:12
User: brainstorm
Functions: rsync
1

That is, after running `vagrant ssh-config` to determine ports and ip's:

vagrant ssh-config

Host default

HostName 127.0.0.1

User vagrant

Port 2200

UserKnownHostsFile /dev/null

StrictHostKeyChecking no

PasswordAuthentication no

IdentityFile /Users/romanvg/tmp/.vagrant/machines/default/virtualbox/private_key

IdentitiesOnly yes

LogLevel FATAL

rsync -v --ignore-existing `ls | head -n 40` [email protected]:/location
rsync -ahhmz --progress --stats [[user@]host:]/source/path/ [[user@]host:]/destination/path/
2014-11-13 18:52:45
User: juangmorales
Functions: rsync
0

Functionally the same as the Microsoft Robocopy (https://en.wikipedia.org/wiki/Robocopy) command below but with the benefits of compression and optionally specifying a user.

robocopy /e [//host]/source/path [//host]/destination/path

Options:

-a: archive mode - rescursive, copy symlinks as symlinks, preserve permissions, preserve modification times, preserve group, preserve owner, preserve device files and special files

-hh: Numbers in human-readable K=1024 format. Single "h" will produce human-readable K=1000 format

-m: don't copy empty directories

-z: use compression (if both source and destination are local it's faster to omit this)

--progress: Shows progress during the transfer and implies --verbose (verbose output)

--stats: Summary after the transfer stops

rsync --recursive --info=progress2 <src> <dst>
2014-10-21 22:19:44
User: koter84
Functions: rsync
Tags: rsync progress
0

update the output of rsync after completing a file but don't create newlines, just overwrite the last line, this looks a lot better in scripts where you do want to see a progress-indicator, but not the lengthy logs

this option is available since rsync 3.1.0

rsync -arvz -e 'ssh -p 2233' --progress --delete [email protected]:/path/to/folder /path/to/local/folder
2014-09-26 10:42:26
User: nadavkav
Functions: rsync
1

Useful, when you need to backup/copy/sync a folder over ssh with a non standard port number

rsync -av -f"+ */" -f"- *" [source] [dest]
rsync -avz --dry-run /somewhere/source_directory /somewhereelse/target_directory
2014-04-01 20:55:59
User: tsener
Functions: rsync
Tags: diff rsync
-1

--dry-run will only show you which files would be otherwise synced with rsync.

-z is for compressio

-v vervose

-a "as is" - permissions, ownership etc.

rsync -a -v --delete sending_directory target_directory
rsync -av -e "ssh -o MACs=hmac-ripemd160" --progress --partial [email protected]://path/to/remote/stuff .
2014-02-01 00:46:38
User: RAKK
Functions: rsync
Tags: ssh rsync hmac
0

This command allows you to mirror folders or files with rsync using a secure SSH channel with a forced HMAC integrity algorithm. Use this if you are absolutely adamant about preserving data integrity while mirroring a set of files. --partial is for resumability.

while true; do rsync -vR $(git ls-files | inotifywait -q -e modify -e attrib -e close_write --fromfile - --format '%w') [email protected]:dest/dir/; done
2014-01-21 10:31:41
User: leucos
Functions: rsync
0

While edtiing a project under git, it is sometimes nice to sync changes immediately to a test machine.

This command will take care of this if you have inotifywait installed on the developement machine.

Note the -R (relative) in rsync. with

rsync foo/bar/baz [email protected]:dest/dir/

it will put 'baz' in dest/dir/foo/bar/ which is what we want.

this can be turned into a function for additionnal flexibility :

function gitwatch() {

if [ -z $1 ]; then

echo "You must provide a rsync destination"

return

fi

while true; do

rsync -vR $(git ls-files | inotifywait -q -e modify -e attrib -e close_write --fromfile -

--format '%w') $1

done

}

rsync -arvx --numeric-ids --stats --progress --bwlimit=1000 file server:destination_directory
2013-10-01 13:00:59
Functions: file rsync
Tags: Linux rsync
0

Useful for transferring large file over a network during operational hours

rsync -v -r --size-only --compare-dest=../A/ B/ C/
2013-09-10 21:41:16
User: knoppix5
Functions: rsync
9

Assumed dir A, B, C are subdirs of the current dir

Exact syntax of the command is:

rsync -v -r --size-only --compare-dest=/path_to_A/A/ /path_to_B/B/ /path_to_C/C/

(do not omit end-slashes, since that would copy only the names and not the contents of subdirs of dir B to dir C)

You can replace --size-only with --checksum for more thorough file differences validation

Useful switch:

-n, --dry-run perform a trial run with no changes made

rsync -a --append source-file destination
BEGIN=`date`; rsync -avxW /home/ /backups/home ; echo "Begin time: $BEGIN" ; echo "End time..: `date`"
2013-07-06 08:24:45
User: ryanchapman
Functions: echo rsync
Tags: rsync
1

Copies a directory structure from /home/ to /backups/home (notice that the destination does not have a trailing slash)

rsync -avzhP <[[user@]host1:]directory1> <[[user@]host2:]directory2>
rsync --progress [email protected]:/path/to/source /path/to/target/ | stdbuf -oL tr '\r' '\n' >> rsyncprogress.txt
2013-03-26 11:06:45
User: MessedUpHare
Functions: rsync tr
0

This line unbuffers the interactive output of rsync's --progress flag

creating a new line for every update.

This output can now be used within a script to make actions (or possibly piped into a GUI generator for a progress bar)

rsync
2013-03-01 14:47:19
User: techie
Functions: rsync
-9

rsync is the best command ever and I am interested what the rest of you think is the best command

rsync -arl --rsh=ssh --progress --exclude-from=/etc/localbin/exclude_files.txt /var/www/html/source/* <user>@<server>:/var/www/html/source/
rsync -a -v --stats -e ssh /home root@<newserver>:/root/
mkdir r1 && sshfs remote1:/home/user r1 && rsync r1/stuff remote2:~/backups/
2013-01-11 14:12:22
User: unhammer
Functions: mkdir rsync
Tags: rsync sshfs
-1

rsync by itself doesn't support copying between two remote hosts, but if you use sshfs you can pretend one of them is local. If you have a passphrase-less ssh-key, you can even put this script into a cron job.

A faster alternative is to run ssh-keygen on remote1 and put the pubkey into remote2:~/.ssh/authorized_keys, running rsync on remote1 (or vice versa), but the problem with that is that now a hacker on remote1 can access remote2 at any time. The above method ensures your local computer stays the weak link.

rsync -aHAXxv --numeric-ids --delete --progress -e "ssh -T -c arcfour -o Compression=no -x" user@<source>:<source_dir> <dest_dir>
2012-12-26 13:46:23
User: somaddict
Functions: rsync
Tags: ssh rsync
10

This creates an archive that does the following:

rsync::

(Everyone seems to like -z, but it is much slower for me)

-a: archive mode - rescursive, preserves owner, preserves permissions, preserves modification times, preserves group, copies symlinks as symlinks, preserves device files.

-H: preserves hard-links

-A: preserves ACLs

-X: preserves extended attributes

-x: don't cross file-system boundaries

-v: increase verbosity

--numeric-ds: don't map uid/gid values by user/group name

--delete: delete extraneous files from dest dirs (differential clean-up during sync)

--progress: show progress during transfer

ssh::

-T: turn off pseudo-tty to decrease cpu load on destination.

-c arcfour: use the weakest but fastest SSH encryption. Must specify "Ciphers arcfour" in sshd_config on destination.

-o Compression=no: Turn off SSH compression.

-x: turn off X forwarding if it is on by default.

Flip: rsync -aHAXxv --numeric-ids --delete --progress -e "ssh -T -c arcfour -o Compression=no -x" [source_dir] [dest_host:/dest_dir]

rsync -ayz -e ssh --exclude=/proc --exclude=/sys --exclude=/dev / [email protected]:/MNTDIR
2012-11-06 09:43:42
User: bones
Functions: rsync ssh
7

Copies the complete root-dir of a linux server to another one, where the new harddisks formated and mountet. Very useful to migrate a root-server to another one.