Commands tagged tar

The following command finds all the files not modified in the last 5 days under /protocollo/paflow directory and creates an archive files under /var/dump-protocollo in the format of ddmmyyyy_archive.tar

This is a shortcut to tar up all files matching a wildcard. Tar doesn't have the --include (apparently).

Here how to recover the remote backup over ssh

Execute it from the source host, where the source files you wish backup resides. With the minus '-' the tar command deliver the compressed output to the standar output and, trough over the ssh session to the remote host. On the other hand the backup host will be receive the stream and read it from the standar input sending it to the /path/to/backup/backupfile.tar.bz2

xargs deals badly with special characters (such as space, ' and "). In this case if you have a file called '12" record'.

Parallel https://savannah.nongnu.org/projects/parallel/ does not have this problem.

Both solutions work bad if the number of files is more than the allowed line length of the shell.

xargs deals badly with special characters (such as space, ' and "). To see the problem try this:

touch important_file

touch 'not important_file'

ls not* | xargs rm

Parallel https://savannah.nongnu.org/projects/parallel/ does not have this problem.

This may seem like a long command, but it is great for making sure all file permissions are kept in tact. What it is doing is streaming the files in a sub-shell and then untarring them in the target directory. Please note that the -z command should not be used for local files and no perfomance increase will be visible as overhead processing (CPU) will be evident, and will slow down the copy.

You also may keep simple with, but you don't have the progress info:

You set the file/dirname transfer variable, in the end point you set the path destination, this command uses pipe view to show progress, compress the file outut and takes account to change the ssh cipher. Support dirnames with spaces.

Merged ideas and comments by http://www.commandlinefu.com/commands/view/4379/copy-working-directory-and-compress-it-on-the-fly-while-showing-progress and http://www.commandlinefu.com/commands/view/3177/move-a-lot-of-files-over-ssh

Avoids creating useless directory entries in archive, and sorts files by (roughly) extension, which is likely to group similar files together for better compression. 1%-5% improvement.

What happens here is we tell tar to create "-c" an archive of all files in current dir "." (recursively) and output the data to stdout "-f -". Next we specify the size "-s" to pv of all files in current dir. The "du -sb . | awk ?{print $1}?" returns number of bytes in current dir, and it gets fed as "-s" parameter to pv. Next we gzip the whole content and output the result to out.tgz file. This way "pv" knows how much data is still left to be processed and shows us that it will take yet another 4 mins 49 secs to finish.

Credit: Peteris Krumins http://www.catonmat.net/blog/unix-utilities-pipe-viewer/

Create a tar file in multiple parts if it's to large for a single disk, your filesystem, etc.

Rejoin later with `cat .tar.*|tar xf -`

The magic is performed by the parameter -t

This script will list all the files in the tarballs present on any folder or subfolder of the provided path. The while loop is for echoing the file name of the tarball before listing the files, so the tarball can be identified

This command will copy a folder tree (keeping the parent folders) through ssh. It will:

- compress the data

- stream the compressed data through ssh

- decompress the data on the local folder

This command will take no additional space on the host machine (no need to create compressed tar files, transfer it and then delete it on the host).

There is some situations (like mirroring a remote machine) where you simply cant wait for a huge time taking scp command or cant compress the data to a tarball on the host because of file system space limitation, so this command can do the job quite well.

This command performs very well mainly when a lot of data is involved in the process. If you copying a low amount of data, use scp instead (easier to type)

The command as given would create the file "/result_path/result.tar.gz" with the contents of the target folder including permissions and sub- folder structure.

The lifehacker way: http://lifehacker.com/software/top/geek-to-live--encrypt-your-data-178005.php#Alternate%20Method:%20OpenSSL

"That command will encrypt the unencrypted-data.tar file with the password you choose and output the result to encrypted-data.tar.des3. To unlock the encrypted file, use the following command:"

This command creates tar zip of a directory and its sub-directories.

creates a compressed tar archive of files in /path/foo and writes to a timestamped filename in /path.

Use tar command for a backup info with a date of creation

If you want certain files out of a directory hierarchy, this will copy just the listed files, but will create the directory hierarchy in the new location ($DIR/)

command to decrypt:

Of course, don't forget to rm the original files ;) You may also want to look at the openssl docs for more options.

the f is for file and - stdout, This way little shorter.

I Like copy-directory function It does the job but looks like SH**, and this doesn't understand folders with whitespaces and can only handle full path, but otherwise fine,

function copy-directory () { ; FrDir="$(echo $1 | sed 's:/: :g' | awk '/ / {print $NF}')" ; SiZe="$(du -sb $1 | awk '{print $1}')" ; (cd $1 ; cd .. ; tar c $FrDir/ )|pv -s $SiZe|(cd $2 ; tar x ) ; }

Create a single tar.gz archive

I know it's a very basic one, but it's one I keep forgetting.

Sometimes it is handy to be able to list contents of a tar file within a compressed archive, such as 7Zip in this instance, without having to extract the archive first. This is especially helpful when dealing with larger sized files.

Using 7z to create archives is OK, but when you use tar, you preserve all file-specific information such as ownership, perms, etc. If that's important to you, this is a better way to do it.