Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

2011-03-12 - Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands
To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner
Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed
Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.
Hide

Tags

Hide

Functions

Commands using gzip from sorted by
Terminal - Commands using gzip - 50 results
tar -cf - . | pv -s $(du -sb . | awk '{print $1}') | gzip > out.tgz
2009-12-18 17:09:08
User: opertinicy
Functions: awk du gzip tar
25

What happens here is we tell tar to create "-c" an archive of all files in current dir "." (recursively) and output the data to stdout "-f -". Next we specify the size "-s" to pv of all files in current dir. The "du -sb . | awk ?{print $1}?" returns number of bytes in current dir, and it gets fed as "-s" parameter to pv. Next we gzip the whole content and output the result to out.tgz file. This way "pv" knows how much data is still left to be processed and shows us that it will take yet another 4 mins 49 secs to finish.

Credit: Peteris Krumins http://www.catonmat.net/blog/unix-utilities-pipe-viewer/

sudo ls -RFal / | gzip > all_files_list.txt.gz
2009-12-14 21:40:56
User: roryokane
Functions: gzip ls sudo
2

This command is meant to be used to make a lightweight backup, for when you want to know which files might be missing or changed, but you don't care about their contents (because you have some way to recover them).

Explanation of parts:

"ls -RFal /" lists all files in and below the root directory, along with their permissions and some other metadata.

I think sudo is necessary to allow ls to read the metadata of certain files.

"| gzip" compresses the result, from 177 MB to 16 MB in my case.

"> all_files_list.txt.gz" saves the result to a file in the current directory called all_files_list.txt.gz. This name can be changed, of course.

dd if=/dev/<device location> | gzip -c /<path to backup location>/<disk image name>.img.gz
mysqldump -uUSERNAME -pPASSWORD database | gzip > /path/to/db/files/db-backup-`date +%Y-%m-%d`.sql.gz ;find /path/to/db/files/* -mtime +5 -exec rm {} \;
gzip *
for file in *.foo; do gzip "$file"; done
2009-10-15 01:27:05
User: dfrios
Functions: file gzip
1

It gzip each file in a directory separately

sitepass() { echo -n "$@" | md5sum | sha1sum | sha224sum | sha256sum | sha384sum | sha512sum | gzip - | strings -n 1 | tr -d "[:space:]" | tr -s '[:print:]' | tr '!-~' 'P-~!-O' | rev | cut -b 2-11; history -d $(($HISTCMD-1)); }
2009-10-01 20:14:57
User: syssyphus
Tags: Security
14

usage: sitepass MaStErPaSsWoRd example.com

description: An admittedly excessive amount of hashing, but this will give you a pretty secure password, It also eliminates repeated characters and deletes itself from your command history.

tr '!-~' 'P-~!-O' # this bit is rot47, kinda like rot13 but more nerdy

rev # this avoids the first few bytes of gzip payload, and the magic bytes.

find /home/dir -mtime +1 -print -exec gzip -9 {} \; -exec mv {}.gz {}_`date +%F`.gz \;
for db in $(mysql -e 'show databases' -s --skip-column-names); do mysqldump $db | gzip > "/backups/mysqldump-$(hostname)-$db-$(date +%Y-%m-%d-%H.%M.%S).gz"; done
tar -cf - /home/user/test | gzip -c | ssh user@sshServer 'cd /tmp; tar xfz -'
2009-08-24 18:35:38
User: esplinter
Functions: gzip ssh tar
Tags: ssh file move
11

Useful to move many files (thousands or millions files) over ssh. Faster than scp because this way you save a lot of tcp connection establishments (syn/ack packets).

If using a fast lan (I have just tested gigabyte ethernet) it is faster to not compress the data so the command would be:

tar -cf - /home/user/test | ssh user@sshServer 'cd /tmp; tar xf -'

find $MAILDIR/ -type f -printf '%T@ %p\n' | sort --reverse | sed -e '{ 1,100d; s/[0-9]*\.[0-9]* \(.*\)/\1/g }' | xargs -i sh -c "cat {}&&rm -f {}" | gzip -c >>ARCHIVE.gz
curl http://www.commandlinefu.com/commands/by/<your username>/rss|gzip ->commandlinefu-contribs-backup-$(date +%Y-%m-%d-%H.%M.%S).rss.gz
2009-08-10 12:43:33
Functions: date gzip
10

Use `zless` to read the content of your *rss.gz file:

zless commandlinefu-contribs-backup-2009-08-10-07.40.39.rss.gz
FECHA=$(date +"%F") FINAL="$FECHA.sql.gz" mysqldump -h localhost -u user --password="pass" --opt jdiaz61_lupajuridica | gzip > /home/jdiaz61/www/backup/$FINAL
2009-08-09 14:51:46
User: juancabo
Functions: date gzip
-2

Tres lineas en un shell script para copiar la base de datos diaramente

for I in $(mysql -e 'show databases' -s --skip-column-names); do mysqldump $I | gzip > "$I.sql.gz"; done
gzip -dc /tmp/pavanlimo.gz | psql -U user db
# dd if=/dev/sda | gzip -c | ssh user@ip 'dd of=/mnt/backups/sda.dd'
pg_dump otrs2 | gzip > dump.gz
mysqldump [options] |gzip ->mysqldump-$(date +%Y-%m-%d-%H.%M.%S).gz
2009-06-12 12:42:59
Functions: date gzip
1

Performs a mysqldump and gzip-compresses the output file with a timestamp in the resulting dump file. Inspect the file for integrity or fun with this command afterward, if you desire:

zcat mysqldump-2009-06-12-07.41.01.tgz | less
dd if=/dev/sda5 bs=2048 conv=noerror,sync | gzip -fc | lftp -u user,passwd domain.tld -e "put /dev/stdin -o backup-$(date +%Y%m%d%H%M).gz; quit"
tar cvf - foodir | gzip > foo.tar.gz
2009-05-19 11:33:51
User: sharfah
Functions: gzip tar
Tags: tar gzip
-9

Where foodir is the directory you want to zip up.

< /dev/urandom tr -dc A-Za-z0-9_ | head -c $((1024 * 1024)) | tee >(gzip -c > out.gz) >(bzip2 -c > out.bz) > /dev/null
for x in `psql -e\l | awk '{print $1}'| egrep -v "(^List|^Name|\-\-\-\-\-|^\()"`; do pg_dump -C $x | gzip > /var/lib/pgsql/backups/$x-nightly.dmp.gz; done
2009-02-21 15:21:09
User: f4nt
Functions: awk egrep gzip
1

Ran as the postgres user, dumps each database individually. It dumps with the create statements as well, so you can just 'zcat $x-nightly.dmp.gz | psql' to reimport/recreate a database from a backup.

find . -type f -name "*" -mtime +10 -print -exec gzip {} \;
2009-02-16 14:33:32
User: migueldiab
Functions: find gzip
2

Useful for a cron job that runs nightly, gzipping or alternatively deleting files from a specific directory that are older than 10 days (in this case)

cat /dev/urandom | gzip -9 > /dev/null &
2009-02-16 14:12:38
Functions: cat gzip
3

check your load with top... Start more of these jobs to get an multi-core cpu busy...

pv access.log | gzip > access.log.gz
2009-02-06 08:50:40
User: p3k
Functions: gzip
48

Pipe viewer is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion. Source: http://www.catonmat.net/blog/unix-utilities-pipe-viewer/