Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

2011-03-12 - Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands
To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner
Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed
Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.
Hide

Tags

Hide

Functions

Commands by dopeman from sorted by
Terminal - Commands by dopeman - 11 results
awk '{ if ($1 in stored_lines) x=1; else print; stored_lines[$1]=1 }' infile.txt > outfile.txt
2010-12-15 17:08:47
User: dopeman
Functions: awk
4

The command (above) will remove any duplicate rows based on the FIRST column of data in an un-sorted file.

The '$1' represents a positional parameter. You can change both instances of '$1' in the command to remove duplicates based on a different column, for instance, the third:

awk '{ if ($3 in stored_lines) x=1; else print; stored_lines[$3]=1 }' infile.txt > outfile.txt

Or you can change it to '$0' to base the removal on the whole row:

awk '{ if ($0 in stored_lines) x=1; else print; stored_lines[$0]=1 }' infile.txt > outfile.txt

** Note: I wouldn't use this on a MASSIVE file, unless you're RAM-rich ;) **

buf () { filename=$1; filetime=$(date +%Y%m%d_%H%M%S); cp ${filename} ${filename}_${filetime}; }
2010-12-14 13:19:52
User: dopeman
Functions: cp date
1
buf myfile.txt

This is useful when you are making small but frequent changes to a file. It keeps things organised and clear for another administrator to see what changed and at what time. An overview of changes can be deduced using a simple:

ls -ltr
alias dush="du -xsm * | sort -n | awk '{ printf(\"%4s MB ./\",\$1) ; for (i=1;i<=NF;i++) { if (i>1) printf(\"%s \",\$i) } ; printf(\"\n\") }' | tail"
2010-07-15 10:38:27
User: dopeman
Functions: alias
-1

Essentially the same as funky's alias, but will not traverse filesystems and has nicer formatting.

lsof +L1
2010-07-14 17:21:01
User: dopeman
Tags: Linux
14

I have come across a situation in the past where someone has unlinked a file by running an 'rm' command against it while it was still being written to by a running process.

The problem manifested itself when a 'df' command showed a filesystem at 100%, but this did not match the total value of a 'du -sk *'.

When this happens, the process continues to write to the file but you can no longer see the file on the filesystem. Stopping and starting the process will, more often than not, get rid of the unlinked file, however this is not always possible on a live server.

When you are in this situation you can use the 'lsof' command above to get the PID of the process that owns the file (in the sample output this is 23521).

Run the following command to see a sym-link to the file (marked as deleted):

cd /proc/23521/fd && ls -l

Truncate the sym-link to regain your disk space:

> /proc/23521/fd/3

I should point out that this is pretty brutal and *could* potentially destabilise your system depending on what process the file belongs to that you are truncating.

lgrep() { string=$1; file=$2; awk -v String=${string} '$0 ~ String' ${file}; }
2010-01-19 09:42:19
User: dopeman
Functions: awk
1

This is a handy way to circumvent the "Maximum line length of 2048 exceeded" grep error.

Once you have run the above command (or put it in your .bashrc), files can be searched using:

lgrep search-string /file/to/search
ps -ef | awk '/process-name/ && !/awk/ {print}'
2009-08-19 11:22:09
User: dopeman
Functions: awk ps
1

This does the same thing as many of the 'grep' based alternatives but allows a more finite control over the output. For example if you only wanted the process ID you could change the command:

ps -ef | awk '/mingetty/ && !/awk/ {print $2}'

If you wanted to kill the returned PID's:

ps -ef | awk '/mingetty/ && !/awk/ {print $2}' | xargs -i kill {}
ssh user@host "(cd /path/to/remote/top/dir ; tar cvf - ./*)" | tar xvf -
2009-03-31 13:08:45
User: dopeman
Functions: ssh tar
Tags: copy files
1

This command will copy files and directories from a remote machine to the local one.

Ensure you are in the local directory you want to populate with the remote files before running the command.

To copy a directory and it's contents, you could:

ssh user@host "(cd /path/to/a/directory ; tar cvf - ./targetdir)" | tar xvf -

This is especially useful on *nix'es that don't have 'scp' installed by default.

ps aux | sort +2n | tail -20
2009-03-31 12:03:34
User: dopeman
Functions: ps sort tail
3

This command will show the 20 processes using the most CPU time (hungriest at the bottom).

You can see the 20 most memory intensive processes (hungriest at the bottom) by running:

ps aux | sort +3n | tail -20

Or, run both:

echo "CPU:" && ps aux | sort +2n | tail -20 && echo "Memory:" && ps aux | sort +3n | tail -20
du -xk | sort -n | tail -20
2009-03-30 11:37:43
User: dopeman
Functions: du sort tail
7

This command will tell you the 20 biggest directories starting from your working directory and skips directories on other filesystems. Useful for resolving disk space issues.

perl -pi -e 's/foo/bar/g' $(grep -rl foo ./*)
2009-03-27 17:21:35
User: dopeman
Functions: grep perl
Tags: perl
-1

This command will replace all instances of 'foo' with 'bar' in all files in the current working directory and any sub-directories.

perl -pi -e 's/foo/bar/g' $(grep -l foo ./*)
2009-03-27 17:18:08
User: dopeman
Functions: grep perl
-1

This command will replace all instances of 'foo' with 'bar' in all files in the current working directory.