Commands using tar (225)


  • 70
    wget -qO - "http://www.tarball.com/tarball.gz" | tar zxvf -
    jianingy · 2009-02-08 12:22:54 7

  • 41
    tar -tf <file.tar.gz> | xargs rm -r
    prayer · 2009-07-06 22:23:11 3
  • ~$ tar --version tar (GNU tar) 1.20


    29
    tar --exclude-vcs -cf src.tar src/
    hendry · 2010-01-20 10:16:17 3
  • What happens here is we tell tar to create "-c" an archive of all files in current dir "." (recursively) and output the data to stdout "-f -". Next we specify the size "-s" to pv of all files in current dir. The "du -sb . | awk ?{print $1}?" returns number of bytes in current dir, and it gets fed as "-s" parameter to pv. Next we gzip the whole content and output the result to out.tgz file. This way "pv" knows how much data is still left to be processed and shows us that it will take yet another 4 mins 49 secs to finish. Credit: Peteris Krumins http://www.catonmat.net/blog/unix-utilities-pipe-viewer/ Show Sample Output


    26
    tar -cf - . | pv -s $(du -sb . | awk '{print $1}') | gzip > out.tgz
    opertinicy · 2009-12-18 17:09:08 3
  • Create an AES256 encrypted and compressed tar archive. User is prompted to enter the password. Decrypt with: openssl enc -d -aes256 -in <file> | tar --extract --file - --gzip


    25
    tar --create --file - --posix --gzip -- <dir> | openssl enc -e -aes256 -out <file>
    seb1245 · 2012-11-27 15:33:45 1
  • this bzips a folder and transfers it over the network to "host" at 777k bit/s. cstream can do a lot more, have a look http://www.cons.org/cracauer/cstream.html#usage for example: echo w00t, i'm 733+ | cstream -b1 -t2 hehe :)


    24
    tar -cj /backup | cstream -t 777k | ssh host 'tar -xj -C /backup'
    wires · 2009-07-02 10:05:53 6
  • At client side: tar c myfile | nc localhost 7000 ##Send file myfile to server tar c mydir | nc localhost 7000 ## Send directory mydir to server


    19
    while true; do nc -l 7000 | tar -xvf -; done
    anhpht · 2011-10-26 23:43:51 4
  • Create a tar file in multiple parts if it's to large for a single disk, your filesystem, etc. Rejoin later with `cat .tar.*|tar xf -` Show Sample Output


    17
    tar cf - <dir>|split -b<max_size>M - <name>.tar.
    dinomite · 2009-11-11 01:53:33 0

  • 17
    wget -qO - http://example.com/path/to/blah.tar.gz | tar xzf -
    TuxOtaku · 2010-02-15 04:00:29 1
  • This will uncompress the file while it's being downloaded which makes it much faster


    16
    wget http://URL/FILE.tar.gz -O - | tar xfz -
    theturingmachine · 2011-01-18 12:17:16 1
  • it compresses the files and folders to stdout, secure copies it to the server's stdin and runs tar there to extract the input and output to whatever destination using -C. if you emit "-C /destination", it will extract it to the home folder of the user, much like `scp file [email protected]:`. the "v" in the tar command can be removed for no verbosity.


    13
    tar czv file1 file2 folder1 | ssh [email protected] tar zxv -C /destination
    xsawyerx · 2009-01-29 10:38:26 4
  • Execute it from the source host, where the source files you wish backup resides. With the minus '-' the tar command deliver the compressed output to the standar output and, trough over the ssh session to the remote host. On the other hand the backup host will be receive the stream and read it from the standar input sending it to the /path/to/backup/backupfile.tar.bz2 Show Sample Output


    13
    tar jcpf - [sourceDirs] |ssh [email protected] "cat > /path/to/backup/backupfile.tar.bz2"
    mack · 2010-03-24 01:29:25 0
  • Create a tgz archive of all the files containing local changes relative to a subversion repository. Add the '-q' option to only include files under version control: svn st -q | cut -c 8- | sed 's/^/\"/;s/$/\"/' | xargs tar -czvf ../backup.tgz Useful if you are not able to commit yet but want to create a quick backup of your work. Of course if you find yourself needing this it's probably a sign you should be using a branch, patches or distributed version control (git, mercurial, etc..)


    12
    svn st | cut -c 8- | sed 's/^/\"/;s/$/\"/' | xargs tar -czvf ../backup.tgz
    chrisdrew · 2009-02-09 11:24:31 3
  • This command will copy a folder tree (keeping the parent folders) through ssh. It will: - compress the data - stream the compressed data through ssh - decompress the data on the local folder This command will take no additional space on the host machine (no need to create compressed tar files, transfer it and then delete it on the host). There is some situations (like mirroring a remote machine) where you simply cant wait for a huge time taking scp command or cant compress the data to a tarball on the host because of file system space limitation, so this command can do the job quite well. This command performs very well mainly when a lot of data is involved in the process. If you copying a low amount of data, use scp instead (easier to type) Show Sample Output


    12
    ssh <host> 'tar -cz /<folder>/<subfolder>' | tar -xvz
    polaco · 2009-11-10 20:06:47 4
  • If archive has leading directory level same as archive name and you want to strip it, this command is for you.


    12
    tar -xaf archive.tar.gz --strip-components=1
    sirex · 2011-11-29 07:38:19 0
  • Leave it to a proprietary software vendor to turn a cheap and easy parlor trick into a selling point. "Hey guys, why don't we turn our _collection of multiple files_ into a *collection of multiple files*!!" Extract the ^above with this: cat pics.tar.gz.??? | tar xzv ^extract on any Unix - no need to install junkware! (If you must make proprietary software, at least make it do something *new*) if [ -e windows ]; then use 7-Zip


    11
    tar czv Pictures | split -d -a 3 -b 16M - pics.tar.gz.
    asmoore82 · 2009-06-09 19:48:01 3
  • Useful to move many files (thousands or millions files) over ssh. Faster than scp because this way you save a lot of tcp connection establishments (syn/ack packets). If using a fast lan (I have just tested gigabyte ethernet) it is faster to not compress the data so the command would be: tar -cf - /home/user/test | ssh [email protected] 'cd /tmp; tar xf -'


    11
    tar -cf - /home/user/test | gzip -c | ssh [email protected] 'cd /tmp; tar xfz -'
    esplinter · 2009-08-24 18:35:38 6
  • The command extracting the tar contents into particular directory ...


    11
    tar xfz filename.tar.gz -C PathToDirectory
    Dhinesh · 2011-11-17 12:43:56 2
  • This Anti-TarBomb function makes it easy to unpack a .tar.gz without worrying about the possibility that it will "explode" in your current directory. I've usually always created a temporary folder in which I extracted the tarball first, but I got tired of having to reorganize the files afterwards. Just add this function to your .zshrc / .bashrc and use it like this; atb arch1.tar.gz and it will create a folder for the extracted files, if they aren't already in a single folder. This only works for .tar.gz, but it's very easy to edit the function to suit your needs, if you want to extract .tgz, .tar.bz2 or just .tar. More info about tarbombs at http://www.linfo.org/tarbomb.html Tested in zsh and bash. UPDATE: This function works for .tar.gz, .tar.bz2, .tgz, .tbz and .tar in zsh (not working in bash): atb() { l=$(tar tf $1); if [ $(echo "$l" | wc -l) -eq $(echo "$l" | grep $(echo "$l" | head -n1) | wc -l) ]; then tar xf $1; else mkdir ${1%.t(ar.gz||ar.bz2||gz||bz||ar)} && tar xf $1 -C ${1%.t(ar.gz||ar.bz2||gz||bz||ar)}; fi ;} UPDATE2: From the comments; bepaald came with a variant that works for .tar.gz, .tar.bz2, .tgz, .tbz and .tar in bash: atb() {shopt -s extglob ; l=$(tar tf $1); if [ $(echo "$l" | wc -l) -eq $(echo "$l" | grep $(echo "$l" | head -n1) | wc -l) ]; then tar xf $1; else mkdir ${1%[email protected](ar.gz|ar.bz2|gz|bz|ar)} && tar xf $1 -C ${1%[email protected](ar.gz|ar.bz2|gz|bz|ar)}; fi ; shopt -u extglob} Show Sample Output


    10
    atb() { l=$(tar tf $1); if [ $(echo "$l" | wc -l) -eq $(echo "$l" | grep $(echo "$l" | head -n1) | wc -l) ]; then tar xf $1; else mkdir ${1%.tar.gz} && tar xf $1 -C ${1%.tar.gz}; fi ;}
    elfreak · 2010-10-16 05:50:32 5

  • 10
    tar -czvf - /src/dir | ssh remotehost "(cd /dst/dir ; tar -xzvf -)"
    AllyUnion · 2010-12-18 00:17:34 1
  • Using 7z to create archives is OK, but when you use tar, you preserve all file-specific information such as ownership, perms, etc. If that's important to you, this is a better way to do it.


    8
    tar cf - /path/to/data | 7z a -si archivename.tar.7z
    slashdot · 2009-07-14 14:21:30 2

  • 8
    curl http://example.com/a.gz | tar xz
    psykotron · 2009-12-20 18:47:49 0
  • Here how to recover the remote backup over ssh Show Sample Output


    8
    ssh [email protected] "cat /path/to/backup/backupfile.tar.bz2" |tar jpxf -
    mack · 2010-03-24 01:35:28 2
  • This is freaking sweet!!! Here is the full alias, (I didn't want to cause display problems on commandlinefu.com's homepage): alias tarred='( ( D=`builtin pwd`; F=$(date +$HOME/`sed "s,[/ ],#,g" <<< ${D/${HOME}/}`#-%F.tgz); S=$SECONDS; tar --ignore-failed-read --transform "s,^${D%/*},`date +${D%/*}.%F`,S" -czPf "$"F "$D" && logger -s "Tarred $D to $F in $(($SECONDS-$S)) seconds" ) & )' Creates a .tgz archive of whatever directory it is run from, in the background, detached from current shell so if you logout it will still complete. Also, you can run this as many times as you want, if the archive .tgz already exists, it just moves it to a numbered backup '--backup=numbered'. The coolest part of this is the transformation performed by tar and sed so that the archive file names are automatically created, and when you extract the archive file it is completely safe thanks to the transform command. If you archive lets say /home/tombdigger/new-stuff-to-backup/ it will create the archive /home/#home#tombdigger#new-stuff-to-backup#-2010-11-18.tgz Then when you extract it, like tar -xvzf #home#tombdigger#new-stuff-to-backup#-2010-11-18.tgz instead of overwriting an existing /home/tombdigger/new-stuff-to-backup/ directory, it will extract to /home/tombdigger/new-stuff-to-backup.2010-11-18/ Basically, the tar archive filename is the PWD with all '/' replaced with '#', and the date is appended to the name so that multiple archives are easily managed. This example saves all archives to your $HOME/archive-name.tgz, but I have a $BKDIR variable with my backup location for each shell user, so I just replaced HOME with BKDIR in the alias. So when I ran this in /opt/askapache/SOURCE/lockfile-progs-0.1.11/ the archive was created at /askapache-bk/#opt#askapache#SOURCE#lockfile-progs-0.1.11#-2010-11-18.tgz Upon completion, uses the universal logger tool to output its completion to syslog and stderr (printed to your terminal), just remove that part if you don't want it, or just remove the '-s ' option from logger to keep the logs only in syslog and not on your terminal. Here's how my syslog server recorded this.. 2010-11-18T00:44:13-05:00 gravedigger.askapache.com (127.0.0.5) [user] [notice] (logger:) Tarred /opt/askapache/SOURCE/lockfile-progs-0.1.11 to /askapache-bk/tarred/#opt#SOURCE#lockfile-progs-0.1.11#-2010-11-18.tgz in 4 seconds Caveats Really this is very robust and foolproof, the only issues I ever have with it (I've been using this for years on my web servers) is if you run it in a directory and then a file changes in that directory, you get a warning message and your archive might have a problem for the changed file. This happens when running this in a logs directory, a temp dir, etc.. That's the only issue I've ever had, really nothing more than a heads up. Advanced: This is a simple alias, and very useful as it works on basically every linux box with semi-current tar and GNU coreutils, bash, and sed.. But if you want to customize it or pass parameters (like a dir to backup instead of pwd), check out this function I use.. this is what I created the alias from BTW, replacing my aa_status function with logger, and adding $SECONDS runtime instead of using tar's --totals function tarred () { local GZIP='--fast' PWD=${1:-`pwd`} F=$(date +${BKDIR}/%m-%d-%g-%H%M-`sed -u 's/[\/\ ]/#/g' [[ ! -r "$PWD" ]] && echo "Bad permissions for $PWD" 1>&2 && return 2; ( ( tar --totals --ignore-failed-read --transform "[email protected]^${PWD%/*}@`date +${PWD%/*}.%m-%d-%g`@S" -czPf $F $PWD && aa_status "Completed Tarp of $PWD to $F" ) & ) } #From my .bash_profile http://www.askapache.com/linux-unix/bash_profile-functions-advanced-shell.html Show Sample Output


    8
    alias tarred='( ( D=`builtin pwd`; F=$(date +$HOME/`sed "s,[/ ],#,g" <<< ${D/${HOME}/}`#-%F.tgz); tar --ignore-failed-read --transform "s,^${D%/*},`date +${D%/*}.%F`,S" -czPf "$F" "$D" &>/dev/null ) & )'
    AskApache · 2010-11-18 06:24:34 0

  • 7
    curl http://example.com/foo.tar.gz | tar zxvf -
    anarchivist · 2009-02-18 13:02:05 0
  •  1 2 3 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Display usb power mode on all devices

cpu and memory usage top 10 under Linux
The original version gives an error, here is the correct output

check open ports without netstat or lsof

Unlock more space form your hard drive
This command changes the reserved space for privileged process on '/dev/sda' to 1 per cent.

Check which modules your PHP is using

convert hex to decimal ; decimal to hex

Download file with multiple simultaneous connections
`aria2c` (from the aria2 project) allows. Change -s 4 to an arbitrary number of segments to control the number of concurrent connections. It is also possible to provide multiple URLs to the same content (potentially over multiple protocols) to download the file concurrently from multiple hosts.

ttyS0 - terminal on serial connection
I actually planned to do this for quite a long time, but since I haven't had any suitable client hardware, I procrastinated this. Now, the old laptop I've got from my dad, features an RS-232 port. So, now that I had technically a client, that I could test my RS-232 connection with, I ordered a null modem cable. There is no RS-232 outlet on my desktop computer directly on the mainboard, but theres a connector on the mainbord, where a RS-232 outlet can be attached to. The outlet will then cover up a PCI slot. # Activating RS-232 Ok, once all cables were in place, I tried to access both RS-232 ports by reading from them directly. They're usually ttyS0 or ttyS1, depending what COM-Port it is. From the file /proc/tty/driver/serial, information about the serial setup can be obtained. A setserial -q /dev/ttyS0 might be usefull as well. Usually, the UART Type is 16550A, on a standard PC. Mine wasn't working though. At leas not right from the start, when I tried to read the interface with cat /dev/ttyS0 I got the following error: # cat /dev/ttyS0 cat: /dev/ttyS0: Input/output error Obviously, the driver couldn't activate the hardware. Reason was, it was deactivated in BIOS. After activating RS-232 there, it worked well. As a last action, I added myself to the uucp group, so I have user permission to the serial lines. It is not necessary for the terminal setup, but a good idea to do so, just for future projects, maybe... # Setting up a terminal Once the Serial line is configured and working properly, it's time to let a terminal run on that port. This is what I added to my /etc/inittab : s0:2345:respawn:/sbin/agetty -L 38400 ttyS0 I added it quite on the top of that file, right below the 'si' statement, mingetty cannot be used for serial connections, it cannot be run in a console, too. I tried it for testing purposes, but the cosole - along with your login program - will log you out, as soon as you log in over your serial line. '-L' means this is a local line, with no carrier signal. 38400 is the standard speed of a Linux console, it might be a bit high, I was told, but it works well. I tested that with some higher values as well (115200) and it worked too, I guess it tepends on things like cable length, etc. Last parameter, is the serial tty to listen on. The terminal type can be specified as an additional parameter at the end of the parameter list, vt102, for instance. This is sometimes required, depending on the client. After finishing editing /etc/inittab, an init q will make the system re-read /etc/inittab and apply changes. The agetty should now be listening on ttyS0. #Setting up a client It's time to establish a connection and test the serial line. I use a laptop, that has an RS-232 port, so some preliminary setup is required. I tried minicom as terminal initially, but it turned out, not to be the best client. It initializes the modem, this lasts quite long, and it doesn't convey ANSI colors. So the better option is cu, it's part of the UUCP-Package. Oh, and the serial port of that computer, has to be accessible as well, of course. Once everything was set up, I established the connection: $ cu -l ttyS0 -38400 --nostop Pretty self explanatory, I think. The --nostop option disables XON/XOFF handling. # root access over ttyS0 In order to become root over the serial terminal, the tty needs to be added to /etc/securetty I appended ttyS0 to the end of the file. It is now possible, to gain root access over the serial terminal. The agetty process needs to be restarted to apply changes. # Accessing GRUB over ttyS0 To make bootloader access possible over ttyS0, some changes to /boot/grub/menu.lst need to be done. (GRUB is the bootloader I use, I suppose LiLo has similar capabilities.) Those are the lines, I appended to the top of my menu.lst : serial --unit=0 --speed=38400 --word=8 --parity=no --stop=1 terminal --timeout=3 serial console The serial command initiates the serial terminal option, --unit=0 defines our first serial connector, I my case, it's the only one I have on my machine. I used the standard Linux-Console speed, as well as the "8N1" connection strategy. terminal defines the terminal priorities, first terminal (serial) is the standard one, the last one is the secondary terminal (console). --timeout=3 enables a delay on both consoles, with a prompt for a keystroke. Depending on which terminal, the key is pressed, this terminal, will be used. If no key is pressed after the timeout, the standard console (in my case serial) will be used. # Relaying Kernel output on boot The Kernel accepts multiple console options, of which the last one, is the standard console, and the one that will be used in Single User mode. These are my Kernel options: title Fedora Core (2.6.20-1.2316.fc5) root (hd0,0) kernel /vmlinuz-2.6.20-1.2316.fc5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet vga=795 console=tty0 console=ttyS0,38400 initrd /initrd-2.6.20-1.2316.fc5.img console=tty0 is the standard console, located on the machine, i.e. monitor and keyboard.

cpu info

Run netcat to server files of current folder


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: