Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

OpenID
or
Sign in Register

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

2011-03-12 - Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands
To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner
Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed
Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.
Hide

Tags

Hide

Functions

Hide

Credits

Site by David Winterbottom (user root).

Commands using tar from sorted by
Terminal - Commands using tar - 194 results
tar pcf - home | pv -s $(du -sb home | awk '{print $1}') --rate-limit 500k | gzip > /mnt/c/home.tar.gz
2010-04-02 15:29:03
User: Sail
Functions: awk du gzip tar
1
Up
Down
tar directory and compress it with showing progress and Disk IO limits

tar directory and compress it with showing progress and Disk IO limits. Pipe Viewer can be used to view the progress of the task, Besides, he can limit the disk IO, especially useful for running Servers.

Show sample output | Comments (2) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
ls | while read filename; do tar -czvf "$filename".tar.gz "$filename"; rm "$filename"; done
2010-03-29 08:10:38
User: Thingymebob
Functions: ls read rm tar
-2
Up
Down
Individually compress each file in a directory

Compresses each file individually, creating a $fileneame.tar.gz and removes the uncompressed version, usefull if you have lots of files and don't want 1 huge archive containing them all. you could replace ls with ls *.pdf to just perform the action on pdfs for example.

Comments (2) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
ssh user@host "cat /path/to/backup/backupfile.tar.bz2" |tar jpxf -
2010-03-24 01:35:28
User: mack
Functions: ssh tar
Tags: ssh tar
8
Up
Down
Recover remote tar backup with ssh

Here how to recover the remote backup over ssh

Show sample output | Comments (2) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
tar jcpf - [sourceDirs] |ssh user@host "cat > /path/to/backup/backupfile.tar.bz2"
2010-03-24 01:29:25
User: mack
Functions: ssh tar
Tags: ssh tar
13
Up
Down
Remote backups with tar over ssh

Execute it from the source host, where the source files you wish backup resides. With the minus '-' the tar command deliver the compressed output to the standar output and, trough over the ssh session to the remote host. On the other hand the backup host will be receive the stream and read it from the standar input sending it to the /path/to/backup/backupfile.tar.bz2

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
Server: nc -l 1234 |tar xvfpz - ;Client: tar zcfp - /path/to/dir | nc localhost 1234
2010-03-02 14:24:04
User: unixmonkey8504
Functions: tar
2
Up
Down
Netcat & Tar

Create a tarball on the client and send it across the network with netcat on port 1234 where its extracted on the server in the current directory.

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
ssh -c 'tar cvzf - -C /path/to/src/*' | tar xzf -
2010-03-02 14:15:17
User: unixmonkey8504
Functions: ssh tar
0
Up
Down
Create & transfer tarball over ssh

Create tarball on stdout which is piped to tar reading from stdin all over ssh

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
wget -qO - http://example.com/path/to/blah.tar.gz | tar xzf -
2010-02-15 04:00:29
User: TuxOtaku
Functions: tar wget
17
Up
Down
download and unpack tarball without leaving it sitting on your hard drive
Comments (1) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
tar -zcvpf backup_`date +"%Y%m%d_%H%M%S"`.tar.gz `find <target> -atime +5 -type f` 2> /dev/null | parallel -X rm -f
2010-01-28 12:41:41
User: unixmonkey8046
Functions: rm tar
Tags: backup xargs parallel
-3
Up
Down
backup and remove files with access time older than 5 days.

This deals nicely with files having special characters in the file name (space ' or ").

Parallel is from https://savannah.nongnu.org/projects/parallel/

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
svn st | cut -c 9- | parallel -X tar -czvf ../backup.tgz
2010-01-28 11:43:16
User: unixmonkey8046
Functions: cut tar
Tags: svn xargs tar parallel
-2
Up
Down
archive all files containing local changes (svn)

xargs deals badly with special characters (such as space, ' and "). In this case if you have a file called '12" record'.

Parallel https://savannah.nongnu.org/projects/parallel/ does not have this problem.

Both solutions work bad if the number of files is more than the allowed line length of the shell.

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
tar -tf <file.tar.gz> | parallel rm
2010-01-28 08:28:16
User: unixmonkey8046
Functions: tar
Tags: xargs tar parallel
-2
Up
Down
Remove all files previously extracted from a tar(.gz) file.

xargs deals badly with special characters (such as space, ' and "). To see the problem try this:

touch important_file

touch 'not important_file'

ls not* | xargs rm

Parallel https://savannah.nongnu.org/projects/parallel/ does not have this problem.

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
tar --exclude-vcs -cf src.tar src/
2010-01-20 10:16:17
User: hendry
Functions: tar
28
Up
Down
Exclude .svn, .git and other VCS junk for a pristine tarball

~$ tar --version

tar (GNU tar) 1.20

Comments (3) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
dir='path to file'; tar cpf - "$dir" | pv -s $(du -sb "$dir" | awk '{print $1}') | tar xpf - -C /other/path
2010-01-19 19:05:45
User: starchox
Functions: awk dir du tar
Tags: copy tar cp
-2
Up
Down
transfer files locally to be sure that file permissions are kept correctly showing progress

This may seem like a long command, but it is great for making sure all file permissions are kept in tact. What it is doing is streaming the files in a sub-shell and then untarring them in the target directory. Please note that the -z command should not be used for local files and no perfomance increase will be visible as overhead processing (CPU) will be evident, and will slow down the copy.

You also may keep simple with, but you don't have the progress info:

cp -rpf /some/directory /other/path
Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
file='path to file'; tar -cf - "$file" | pv -s $(du -sb "$file" | awk '{print $1}') | gzip -c | ssh -c blowfish user@host tar -zxf - -C /opt/games
2010-01-19 16:02:45
User: starchox
Functions: awk du file gzip ssh tar
Tags: ssh awk tar directory compress cipher
3
Up
Down
improve copy file over ssh showing progress

You set the file/dirname transfer variable, in the end point you set the path destination, this command uses pipe view to show progress, compress the file outut and takes account to change the ssh cipher. Support dirnames with spaces.

Merged ideas and comments by http://www.commandlinefu.com/commands/view/4379/copy-working-directory-and-compress-it-on-the-fly-while-showing-progress and http://www.commandlinefu.com/commands/view/3177/move-a-lot-of-files-over-ssh

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
wtzc () { wget "$@"; foo=`echo "$@" | sed 's:.*/::'`; tar xzvf $foo; blah=`echo $foo | sed 's:,*/::'`; bar=`echo $blah | sed -e 's/\(.*\)\..*/\1/' -e 's/\(.*\)\..*/\1/'`; cd $bar; ls; }
2010-01-17 11:25:47
User: oshazard
Functions: cd sed tar wget
-1
Up
Down
wget, tar xzvf, cd, ls

Combines a few repetitive tasks when compiling source code. Especially useful when a hypen in a file-name breaks tab completion.

1.) wget source.tar.gz

2.) tar xzvf source.tar.gz

3.) cd source

4.) ls

From there you can run ./configure, make and etc.

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
curl http://example.com/a.gz | tar xz
2009-12-20 18:47:49
User: psykotron
Functions: tar
8
Up
Down
Extract tarball from internet without local saving
Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
find . \! -type d | rev | sort | rev | tar c --files-from=- --format=ustar | bzip2 --best > a.tar.bz2
2009-12-20 14:04:39
User: pornel
Functions: bzip2 c++ find rev sort tar
Tags: tar rev solid archive
2
Up
Down
Slightly better compressed archives

Avoids creating useless directory entries in archive, and sorts files by (roughly) extension, which is likely to group similar files together for better compression. 1%-5% improvement.

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
wget -O - http://example.com/a.gz | tar xz
2009-12-20 02:38:12
User: pornel
Functions: tar wget
4
Up
Down
Extract tarball from internet without local saving

Use `tar xj` for bzip2 archives.

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
tar -cf - . | pv -s $(du -sb . | awk '{print $1}') | gzip > out.tgz
2009-12-18 17:09:08
User: opertinicy
Functions: awk du gzip tar
Tags: awk du tar gzip pv progress
25
Up
Down
copy working directory and compress it on-the-fly while showing progress

What happens here is we tell tar to create "-c" an archive of all files in current dir "." (recursively) and output the data to stdout "-f -". Next we specify the size "-s" to pv of all files in current dir. The "du -sb . | awk ?{print $1}?" returns number of bytes in current dir, and it gets fed as "-s" parameter to pv. Next we gzip the whole content and output the result to out.tgz file. This way "pv" knows how much data is still left to be processed and shows us that it will take yet another 4 mins 49 secs to finish.

Credit: Peteris Krumins http://www.catonmat.net/blog/unix-utilities-pipe-viewer/

Show sample output | Comments (2) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
tar -cvzf arch.tgz $(find /path/dir -not -type d)
2009-12-15 13:46:54
User: pysquared
Functions: find tar
3
Up
Down
Tar - Compress by excluding folders

If you give tar a list of filenames, it will not add the directories, so if you don't care about directory ownership or permissions, you can save some space.

Tar will create directories as necessary when extracting.

This command is limited by the maximum supported size of the argument list, so if you are trying to tar up the whole OS for instance, you may just get "Argument list too long".

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
tar -cvf /path/dir.tar /path/dir* --exclude "/path/dir/name" --exclude "/path/dir/opt"
2009-12-15 09:48:41
User: sandeepverma
Functions: tar
-1
Up
Down
Tar - Compress by excluding folders

Tar - Compress by excluding folders

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
tar -tf /path/to/file.tar
2009-12-01 20:58:07
User: imsaar
Functions: tar
0
Up
Down
lists contents of a tar file
Comments (1) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
tar cf - <dir>|split -b<max_size>M - <name>.tar.
2009-11-11 01:53:33
User: dinomite
Functions: split tar
Tags: tar split tarball archive
17
Up
Down
Split a tarball into multiple parts

Create a tar file in multiple parts if it's to large for a single disk, your filesystem, etc.

Rejoin later with `cat .tar.*|tar xf -`

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
for F in $(find ./ -name "*.tgz") ; do tar -tvzf $F ; done
2009-11-11 00:50:52
User: alchandia
Functions: find tar
Tags: tar
-2
Up
Down
List files in tarballs

The magic is performed by the parameter -t

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
find <path> -name "*.tgz" -or -name "*.tar.gz" | while read file; do echo "$file: "; tar -tzf $file; done
2009-11-10 20:39:04
User: polaco
Functions: echo find read tar
Tags: find tar list
-2
Up
Down
List files in tarballs

This script will list all the files in the tarballs present on any folder or subfolder of the provided path. The while loop is for echoing the file name of the tarball before listing the files, so the tarball can be identified

Comments (3) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
ssh <host> 'tar -cz /<folder>/<subfolder>' | tar -xvz
2009-11-10 20:06:47
User: polaco
Functions: ssh tar
Tags: ssh copy tar compression
9
Up
Down
Copy a folder tree through ssh using compression (no temporary files)

This command will copy a folder tree (keeping the parent folders) through ssh. It will:

- compress the data

- stream the compressed data through ssh

- decompress the data on the local folder

This command will take no additional space on the host machine (no need to create compressed tar files, transfer it and then delete it on the host).

There is some situations (like mirroring a remote machine) where you simply cant wait for a huge time taking scp command or cant compress the data to a tarball on the host because of file system space limitation, so this command can do the job quite well.

This command performs very well mainly when a lot of data is involved in the process. If you copying a low amount of data, use scp instead (easier to type)

Show sample output | Comments (4) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate