Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Universal configuration monitoring and system of record for IT.
Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

May 19, 2015 - A Look At The New Commandlinefu
I've put together a short writeup on what kind of newness you can expect from the next iteration of clfu. Check it out here.
March 2, 2015 - New Management
I'm Jon, I'll be maintaining and improving clfu. Thanks to David for building such a great resource!
Hide

Top Tags

Hide

Functions

Hide

Credits

Commands using gzip from sorted by
Terminal - Commands using gzip - 51 results
file='path to file'; tar -cf - "$file" | pv -s $(du -sb "$file" | awk '{print $1}') | gzip -c | ssh -c blowfish user@host tar -zxf - -C /opt/games
2010-01-19 16:02:45
User: starchox
Functions: awk du file gzip ssh tar
3

You set the file/dirname transfer variable, in the end point you set the path destination, this command uses pipe view to show progress, compress the file outut and takes account to change the ssh cipher. Support dirnames with spaces.

Merged ideas and comments by http://www.commandlinefu.com/commands/view/4379/copy-working-directory-and-compress-it-on-the-fly-while-showing-progress and http://www.commandlinefu.com/commands/view/3177/move-a-lot-of-files-over-ssh

tar -cf - . | pv -s $(du -sb . | awk '{print $1}') | gzip > out.tgz
2009-12-18 17:09:08
User: opertinicy
Functions: awk du gzip tar
26

What happens here is we tell tar to create "-c" an archive of all files in current dir "." (recursively) and output the data to stdout "-f -". Next we specify the size "-s" to pv of all files in current dir. The "du -sb . | awk ?{print $1}?" returns number of bytes in current dir, and it gets fed as "-s" parameter to pv. Next we gzip the whole content and output the result to out.tgz file. This way "pv" knows how much data is still left to be processed and shows us that it will take yet another 4 mins 49 secs to finish.

Credit: Peteris Krumins http://www.catonmat.net/blog/unix-utilities-pipe-viewer/

sudo ls -RFal / | gzip > all_files_list.txt.gz
2009-12-14 21:40:56
User: roryokane
Functions: gzip ls sudo
2

This command is meant to be used to make a lightweight backup, for when you want to know which files might be missing or changed, but you don't care about their contents (because you have some way to recover them).

Explanation of parts:

"ls -RFal /" lists all files in and below the root directory, along with their permissions and some other metadata.

I think sudo is necessary to allow ls to read the metadata of certain files.

"| gzip" compresses the result, from 177 MB to 16 MB in my case.

"> all_files_list.txt.gz" saves the result to a file in the current directory called all_files_list.txt.gz. This name can be changed, of course.

dd if=/dev/<device location> | gzip -c /<path to backup location>/<disk image name>.img.gz
mysqldump -uUSERNAME -pPASSWORD database | gzip > /path/to/db/files/db-backup-`date +%Y-%m-%d`.sql.gz ;find /path/to/db/files/* -mtime +5 -exec rm {} \;
gzip *
for file in *.foo; do gzip "$file"; done
2009-10-15 01:27:05
User: dfrios
Functions: file gzip
1

It gzip each file in a directory separately

sitepass() { echo -n "$@" | md5sum | sha1sum | sha224sum | sha256sum | sha384sum | sha512sum | gzip - | strings -n 1 | tr -d "[:space:]" | tr -s '[:print:]' | tr '!-~' 'P-~!-O' | rev | cut -b 2-11; history -d $(($HISTCMD-1)); }
2009-10-01 20:14:57
User: syssyphus
Tags: Security
14

usage: sitepass MaStErPaSsWoRd example.com

description: An admittedly excessive amount of hashing, but this will give you a pretty secure password, It also eliminates repeated characters and deletes itself from your command history.

tr '!-~' 'P-~!-O' # this bit is rot47, kinda like rot13 but more nerdy

rev # this avoids the first few bytes of gzip payload, and the magic bytes.

find /home/dir -mtime +1 -print -exec gzip -9 {} \; -exec mv {}.gz {}_`date +%F`.gz \;
for db in $(mysql -e 'show databases' -s --skip-column-names); do mysqldump $db | gzip > "/backups/mysqldump-$(hostname)-$db-$(date +%Y-%m-%d-%H.%M.%S).gz"; done
tar -cf - /home/user/test | gzip -c | ssh user@sshServer 'cd /tmp; tar xfz -'
2009-08-24 18:35:38
User: esplinter
Functions: gzip ssh tar
Tags: ssh file move
11

Useful to move many files (thousands or millions files) over ssh. Faster than scp because this way you save a lot of tcp connection establishments (syn/ack packets).

If using a fast lan (I have just tested gigabyte ethernet) it is faster to not compress the data so the command would be:

tar -cf - /home/user/test | ssh user@sshServer 'cd /tmp; tar xf -'

find $MAILDIR/ -type f -printf '%T@ %p\n' | sort --reverse | sed -e '{ 1,100d; s/[0-9]*\.[0-9]* \(.*\)/\1/g }' | xargs -i sh -c "cat {}&&rm -f {}" | gzip -c >>ARCHIVE.gz
curl http://www.commandlinefu.com/commands/by/<your username>/rss|gzip ->commandlinefu-contribs-backup-$(date +%Y-%m-%d-%H.%M.%S).rss.gz
2009-08-10 12:43:33
Functions: date gzip
10

Use `zless` to read the content of your *rss.gz file:

zless commandlinefu-contribs-backup-2009-08-10-07.40.39.rss.gz
FECHA=$(date +"%F") FINAL="$FECHA.sql.gz" mysqldump -h localhost -u user --password="pass" --opt jdiaz61_lupajuridica | gzip > /home/jdiaz61/www/backup/$FINAL
2009-08-09 14:51:46
User: juancabo
Functions: date gzip
-2

Tres lineas en un shell script para copiar la base de datos diaramente

for I in $(mysql -e 'show databases' -s --skip-column-names); do mysqldump $I | gzip > "$I.sql.gz"; done
gzip -dc /tmp/pavanlimo.gz | psql -U user db
# dd if=/dev/sda | gzip -c | ssh user@ip 'dd of=/mnt/backups/sda.dd'
pg_dump otrs2 | gzip > dump.gz
mysqldump [options] |gzip ->mysqldump-$(date +%Y-%m-%d-%H.%M.%S).gz
2009-06-12 12:42:59
Functions: date gzip
1

Performs a mysqldump and gzip-compresses the output file with a timestamp in the resulting dump file. Inspect the file for integrity or fun with this command afterward, if you desire:

zcat mysqldump-2009-06-12-07.41.01.tgz | less
dd if=/dev/sda5 bs=2048 conv=noerror,sync | gzip -fc | lftp -u user,passwd domain.tld -e "put /dev/stdin -o backup-$(date +%Y%m%d%H%M).gz; quit"
tar cvf - foodir | gzip > foo.tar.gz
2009-05-19 11:33:51
User: sharfah
Functions: gzip tar
Tags: tar gzip
-8

Where foodir is the directory you want to zip up.

< /dev/urandom tr -dc A-Za-z0-9_ | head -c $((1024 * 1024)) | tee >(gzip -c > out.gz) >(bzip2 -c > out.bz) > /dev/null
for x in `psql -e\l | awk '{print $1}'| egrep -v "(^List|^Name|\-\-\-\-\-|^\()"`; do pg_dump -C $x | gzip > /var/lib/pgsql/backups/$x-nightly.dmp.gz; done
2009-02-21 15:21:09
User: f4nt
Functions: awk egrep gzip
1

Ran as the postgres user, dumps each database individually. It dumps with the create statements as well, so you can just 'zcat $x-nightly.dmp.gz | psql' to reimport/recreate a database from a backup.

find . -type f -name "*" -mtime +10 -print -exec gzip {} \;
2009-02-16 14:33:32
User: migueldiab
Functions: find gzip
3

Useful for a cron job that runs nightly, gzipping or alternatively deleting files from a specific directory that are older than 10 days (in this case)

cat /dev/urandom | gzip -9 > /dev/null &
2009-02-16 14:12:38
Functions: cat gzip
3

check your load with top... Start more of these jobs to get an multi-core cpu busy...