commandlinefu.com is the place to record those command-line gems that you return to again and again.
Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.
If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/
You can sign-in using OpenID credentials, or register a traditional username and password.
First-time OpenID users will be automatically assigned a username which can be changed after signing in.
Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.
Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):
Subscribe to the feed for:
use find with rsync
copy files to a ssh server with gzip compression
resume a partial scp-filetransfer with rsync
dname is a directory named something like 20090803 for Aug 3, 2009. lastbackup is a soft link to the last backup made - say 20090802. $folder is the folder being backed up. Because this uses hard linking, files that already exist and haven't changed take up almost no space yet each date directory has a kind of "snapshot" of that day's files. Naturally, lastbackup needs to be updated after this operation. I must say that I can't take credit for this gem; I picked it up from somewhere on the net so long ago I don't remember where from anymore. Ah, well...
Systems that are only somewhat slicker than this costs hundreds or even thousands of dollars - but we're HACKERS! We don't need no steenkin' commercial software... :)
This will backup the _contents_ of /media/SOURCE to /media/TARGET where TARGET is formatted with ntfs. The --modify-window lets rsync ignore the less accurate timestamps of NTFS.
tested on cygwin and Fedora 9 .
good to remember for those jobs where you cannot set a site-specific connect option in your ~/.ssh/config file.
It can resume a failed secure copy ( usefull when you transfer big files like db dumps through vpn ) using rsync.
It requires rsync installed in both hosts.
rsync --partial --progress --rsh=ssh $file_source $user@$host:$destination_file local -> remote
rsync --partial --progress --rsh=ssh $user@$host:$remote_file $destination_file remote -> local
connect to a remote server using ftp protocol over FUSE file system, then rsync the remote folder to a local one and then unmount the remote ftp server (FUSE FS)
it can be divided to 3 different commands and you should have curlftpfs and rsync installed
If you have lots of remote hosts sitting "behind" an ssh proxy host, then there is a special-case use of "rsynch" that allows one to easily copy directories and files across the ssh proxy host, without having to do two explicit copies: the '-e' option allows for a replacement "rsh" command. We use this option to specify an "ssh" tunnel command, with the '-A' option that causes authentication agent requests to be forwarded back to the local host. If you have ssh set up correctly, the above command can be done without any passwords being entered.
If your user has sudo on the remote box, you can rsync data as root without needing to login as root. This is very helpful if the remote box does not allow root to login over SSH (which is a common security restriction).
rsync from source to dest all between >30
rsyncs files to a server excluding listed files
also a file can be used to exclude common exclude rules and/or to exclude a ton of files, like so:
rsync --exclude-from '~/.scripts/exclude.txt'
where exclude.txt has one rule per line:
-P displays a progress meter
-z tells rsync to use compression
Put it into your sh startup script (I use
alias scpresume='rsync --partial --progress --rsh=ssh'
in bash). When a file transfer via scp has aborted, just use scpresume instead of scp and rsync will copy only the parts of the file that haven't yet been transmitted.
Create a exact mirror of the local folder "/root/files", on remote server 'remote_server' using SSH command (listening on port 22)
(all files & folders on destination server/folder will be deleted)
copying files from one server to another using rysnc. Root access need to be allowed on the destination.
transfer files from localhost to a remotehost.
You will be prompted for a password unless you have your public keys set-up.