Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

OpenID
or
Sign in Register

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

2011-03-12 - Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands
To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner
Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed
Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.
Hide

Tags

Hide

Functions

Hide

Credits

Site by David Winterbottom (user root).

Commands using perl

Commands using perl from sorted by
Terminal - Commands using perl - 316 results
cat ~/.bash_history | perl -lane 'if($F[0] eq "sudo"){$hash{$F[1]}++}else{$hash{$F[0]}++};$all++;END {@top = map {[$_, $hash{$_}]} sort {$hash{$b}<=>$hash{$a}} keys %hash;printf("%10s%10d%10.2f%%\n", $_->[0],$_->[1],$_->[1]/$all*100) for @top[0..9]}'
2012-07-02 09:38:06
User: vinian
Functions: cat perl sort
0
Up
Down
top ten shell command in bash shell
Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
perl -p -e 's/\\n/\n/g'
2012-06-06 15:37:34
User: icorbett
Functions: perl
0
Up
Down
You can't do that on sed

This also works with -i, just like one might do with sed.

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
cat /var/log/nginx/access.log | grep -oe '^[0-9.]\+' | perl -ne 'system("geoiplookup $_")' | grep -v found | grep -oe ', [A-Za-z ]\+$' | sort | uniq -c | sort -n
2012-05-08 13:28:25
User: theist
Functions: cat grep perl sort uniq
Tags: sort uniq geoip
-1
Up
Down
Per country GET report, based on access log. Easy to transform to unique IP

Per country GET report, based on access log. Easy to transform to unique IP

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
find . -printf '%p\n' | perl -ne 'if( m/(.*)\/(.*)/ ) { $p = $1; $f = $2; $p =~ s/[^\/]/ /g; $p =~ s/\//|/g; print "$p/$f\n"; } elsif( m/(.*)/ ) { print "$1\n"; } else { print "error interpreting: \"$_\"\n"; }'
2012-04-24 19:51:00
User: cbetti
Functions: find perl
0
Up
Down
Display directory hierarchy listing as a tree

Provides a much cleaner, easier to read output than the closest alternative, ls -1R. This alternative makes it easier to differentiate directories from files: find . -printf '%y %p\n' | perl -ne 'if( m/(\w) (.*)\/(.*)/ ) { $t = $1; $p = $2; $f = $3; $t =~ s/[^d]/ /; $p =~ s/[^\/]/ /g; $p =~ s/\//|/g; print "$t $p/$f\n"; } elsif( m/(\w) (.*)/ ) { print "$1 $2\n"; } else { print "error interpreting: \"$_\"\n"; }'

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
ldapsearch -xLLL -b "cn=schema" "(objectclass=*)" \ \* objectclasses attributetypes | perl -p0e 's/\n //g'
2012-04-04 13:31:31
User: ioggstream
Functions: perl
-2
Up
Down
Dump 389ds schema

Dump 389ds schema, putting "\n " on one line with perl.

You have to specify the objectclasses, attributetypes operational attributes too, otherwise they won't be dumped!

Comments (1) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
ldapsearch -xLLL -b "cn=schema" "(objectclass=*)" \ \* objectclasses attributetypes | perl -p0e 's/\n //g'
2012-04-03 16:21:43
User: ioggstream
Functions: perl
-2
Up
Down
Dump 389ds schema

Dump 389ds schema, putting "\n " on one line with perl.

You have to specify the objectclasses, attributetypes operational attributes too, otherwise they won't be dumped!

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
cat z.log | perl -ne 'BEGIN{ print "DATE;RATE\n"; } /\[(\d.*)\]/ && print $1; /CURRENT RATE: +(\S+) msg.*/ && print ";" .$1 . "\n"; '
2012-04-03 13:52:20
User: hute37
Functions: cat perl
1
Up
Down
multiline data block parse and CSV data extraction with perl

extract data in multiline blocks of data with perl pattern matching loop

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
ldapsearch -xLLL -b "cn=schema" "(objectclass=*)" \ \* objectclasses attributetypes | perl -p0e 's/\n //g'
2012-04-03 11:25:55
User: ioggstream
Functions: perl
-2
Up
Down
Dump 389ds schema

Dump 389ds schema, putting "\n " on one line with perl.

You have to specify the objectclasses, attributetypes operational attributes too, otherwise they won't be dumped!

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
ldapsearch -xLLL -b "cn=schema" "(objectclass=*)" \ \* objectclasses attributetypes | perl -p0e 's/\n //g'
2012-04-03 09:56:19
User: ioggstream
Functions: perl
-2
Up
Down
Dump 389ds schema

Dump 389ds schema, putting "\n " on one line with perl.

You have to specify the objectclasses, attributetypes operational attributes too, otherwise they won't be dumped!

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
ldapsearch -xLLL -b "cn=schema" "(objectclass=*)" \ \* objectclasses attributetypes | perl -p0e 's/\n //g'
2012-04-03 09:44:23
User: ioggstream
Functions: perl
0
Up
Down
Dump 389ds schema

Dump 389ds schema, putting "\n " on one line with perl.

You have to specify the objectclasses, attributetypes operational attributes too, otherwise they won't be dumped!

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
ldapsearch -xLLL -b "cn=schema" "(objectclass=*)" \ \* objectclasses attributetypes | perl -p0e 's/\n //g'
2012-04-03 09:44:11
User: ioggstream
Functions: perl
-2
Up
Down
Dump 389ds schema

Dump 389ds schema, putting "\n " on one line with perl.

You have to specify the objectclasses, attributetypes operational attributes too, otherwise they won't be dumped!

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
ldapsearch -xLLL -b "cn=schema" "(objectclass=*)" \ \* objectclasses attributetypes | perl -p0e 's/\n //g'
2012-04-03 09:44:00
User: ioggstream
Functions: perl
-2
Up
Down
Dump 389ds schema

Dump 389ds schema, putting "\n " on one line with perl.

You have to specify the objectclasses, attributetypes operational attributes too, otherwise they won't be dumped!

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
perl -e 'printf "%04o\n", (stat shift)[2] & 0777;' file
2012-03-22 15:05:04
User: zlemini
Functions: perl stat
1
Up
Down
Perl oneliner to print access rights in octal format

This prints file access rights in octal - useful when "stat" is unavailable.

Show sample output | Comments (3) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
for f in `find . -name "*.php"`; do perl -p -i.bak -e 's/<\?php \/\*\*\/ eval\(base64_decode\(\"[^\"]+"\)\);\?>//' $f; done
2012-03-12 10:44:33
User: lizuka
Functions: perl
Tags: perl hack dreamhost
0
Up
Down
clean php files from base64_decode() hack

loop through files in .php extension

make a .bak of each of them

find the base64_decode function and remove it

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
perl -e "print 'yes' if `exim -bt $s_email_here | grep -c malformed`;"
2012-02-28 04:42:41
User: DewiMorgan
Functions: perl
Tags: perl regex email PHP exim
0
Up
Down
Validate an email address

People are *going* to post the wrong ways to do this. It's one of the most common form-validation tasks, and also one of the most commonly messed up. Using a third party tool or library like exim means that you are future-proofing yourself against changes to the email standard, and protecting yourself against the fact that actually checking whether an email address is valid is *not possible*.

Still, perhaps your boss is insisting you really do need to check them internally. OK.

Read the RFCs. The bet before the @ is specified by RFC2821 and RFC2822. The domain name part is specified by RFC1035, RFC1101, RFC1123 and RFC2181.

Generally, when people say "email address", they mean that part of the address that the RFC terms the "addr-spec": the "blah@domain.tld" address, with no display names, comments, quotes, etc. Also "root@localhost" and "root" should be invalid, as should arbitrary addressing schemes specified by a protocol indicator, like "jimbo@myprotocol:foo^bar^baz".

So... With the smallest poetic license for readability (allowing underscores in domain names so we can use "\w" instead of "[a-z0-9]"), the RFCs give us:

^(?:"(?:[^"\\]|\\.)+"|[-^!#\$%&'*+\/=?`{|}~.\w]+)@(?=.{3,255}$)(?:[\w][\w-]{0,62}\.){1,128}[\w][\w-]{0,62}$

Not perfect, but the best I can come up with, and most compliant I've found. I'd be interested to see other people's ideas, though. It's still not going to verify you an address fersure, properly, 100% guaranteed legit, though. What else can you do? Well, you could also:

* verify that the address is either a correct dotted-decimal IP, or contains letters.

* remove reserved domains (.localhost, .example, .test, .invalid), reserved IP ranges, and so forth from the address.

* check for banned domains (whitehouse.gov, example.com...)

* check for known TLDs including alt tlds.

* see if the domain has an MX record set up: if so, connect to that host, else connect to the domain.

* see if the given address is accepted by the server as a recipient or sender (this fails for yahoo.*, which blocks after a few attempts, assuming you are a spammer, and for other domains like rediffmail.com, home.com).

But these are moving well out of the realm of generic regex checks and into the realm of application-specific stuff that should be done in code instead - especially the latter two. Hopefully, this is all you needed to point out to your boss "hey, email validation this is a dark pit with no bottom, we really just want to do a basic check, then send them an email with a link in it: it's the industry standard solution."

Of course, if you want to go nuts, here's an idea that you could do. Wouldn't like to do it myself, though: I'd rather just trust them until their mail bounces too many times. But if you want it, this (untested) code checks to see if the mail domain works. It's based on a script by John Coggeshall and Jesse Houwing that also asked the server if the specific email address existed, but I disliked that idea for several reasons. I suspect: it will get you blocked as a spambot address harvester pretty quick; a lot of servers would lie to you; it would take too much time; this way you can cache domains marked as "OK"; and I suspect it would add little to the reliability test.

// Based on work by: John Coggeshall and Jesse Houwing.

// http://www.zend.com/zend/spotlight/ev12apr.php

mailRegex = '^(?:"(?:[^"\\\\]|\\\\.)+"|[-^!#\$%&\'*+\/=?`{|}~.\w]+)';

mailRegex .= '@(?=.{3,255}$)(?:[\w][\w-]{0,62}\.){1,128}[\w][\w-]{0,62}$';

function ValidateMail($address) {

  global $mailRegex; // Yes, globals are evil. Put it inline if you want.

  if (!preg_match($mailRegex)) {

    return false;

  }

  list ( $localPart, $Domain ) = split ("@",$Email);

  // connect to the first available MX record, or to domain if no MX record.

  $ConnectAddress = new Array();

  if (getmxrr($Domain, $MXHost)) {

    $ConnectAddress = $MXHost;

  } else {

    $ConnectAddress[0] = $Domain;

  }

  // check all MX records in case main server is down - may take time!

  for ($i=0; $i < count($ConnectAddress); $i++ ) {

    $Connect = fsockopen ( $ConnectAddress[$i], 25 );

    if ($Connect){

      break;

    }

  }

  if ($Connect) {

    socket_set_blocking($Connect,0);

    // Only works if socket_blocking is off.

    if (ereg("^220", $Out = fgets($Connect, 1024))) {

      fclose($Connect); // Unneeded, but let's help the gc.

      return true;

    }

    fclose($Connect); // Help the gc.

  }

  return false;

}

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
svn st | awk ' {if ( $1 == "?" ){print $1="",$0}} ' | sed -e 's/^[ \t]*//' | sed 's/ /\\ /g' | perl -ne '`svn add ${1}@` if /(.*)(@*)(.*)/'
2012-02-23 18:42:02
User: JulieCaroline
Functions: awk perl sed
Tags: svn perl
0
Up
Down
add all files not under version control to repository

handels @, ?, whitespaces in names.

replace "?" and "add" by "!" and "rm" for svn mass remove.

---> I m looking for a nicer way to write it, perhaps with something using " perl -ne '`blahblah` if /\?(.*)/' "

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
tail -f production.log | perl -ne 'if (/^Completed.in.(\d+)/){$d = int($1/1000);print "\n";$f{$d}++;for $t (sort(keys(%f))){print $t."s: ".$f{$t}."\n"}}'
2012-02-23 14:37:33
User: theist
Functions: perl tail
Tags: perl log rails ruby filter tail -f
0
Up
Down
filter rails log to count requests sliced in seconds

Commandline perl filter for, using a production.log from a rails app, display on realtime the count of requests grouped by "seconds to complete" (gross round, but fair enough for an oneliner) :)

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
perl -le 'print ~~ map {-s} <*>'
2012-02-21 21:09:48
User: MarxBro
Functions: perl
Tags: perl find wc
6
Up
Down
Count number of files in a directory

Just want to post a Perl alternative.

Does not count hidden files ('.' ones).

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
perl -e 'rename $_, s/ /-/gr for <*.mp3>'
2012-02-21 17:07:18
User: ironcamel
Functions: perl
1
Up
Down
Tracklist reaplace backspace to '-'

Requires perl 5.14 or greater

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
perl -e 'for (<*.mp3>) { $old = $_; s/ /-/g; rename $old, $_ }'
2012-02-16 15:53:41
User: ironcamel
Functions: perl rename
0
Up
Down
Tracklist reaplace backspace to '-'
Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
#(see sample) $ cat x | perl -pe 'BEGIN{ print "TIME;...\n"; } s!(\S+) - (\S+) - \[(\d\d)/(\S\S\S)/(\S+):(\d\d):(\d\d:\d\d) \S+\] "(\S+) (.*/)(\S+)(?:\.([^?]*)(\?\S*)?) HTTP/\S+" (\d+) (\S+)!$3-$4-$5 $6:$7;$6;$2;$1;$8;$13;1;$14;$11;$10;$9;$12;!' > x.csv
2012-02-10 16:58:50
User: hute37
Functions: cat perl
2
Up
Down
Apache CLF access log format to CSV converter

- excel date compatible with a separate hour field

- added a fixed 1 for easier request counter aggregation

- split URL in directory, filename, fileext, query

- used with tomcat valve with response bytes replaced by elapsed time

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
du -k | sort -n | perl -ne 'if ( /^(\d+)\s+(.*$)/){$l=log($1+.1);$m=int($l/log(1024)); printf ("%6.1f\t%s\t%25s %s\n",($1/(2**(10*$m))),(("K","M","G","T","P")[$m]),"*"x (1.5*$l),$2);}' | more
2012-02-07 15:49:19
User: Q_Element
Functions: du perl printf sort
Tags: perl, file sizes one line
0
Up
Down
One line Perl Script to determine the largest file sizes on a Linux Server

This one line Perl script will display the smallest to the largest files sizes in all directories on a server.

Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
useradd -m -p $(perl -e'print crypt("passwordscelta", "stigghiola")') user
2012-02-06 19:53:01
User: 0disse0
Functions: perl useradd
0
Up
Down
Create new user with home dir and given password

The crypt function takes a password, key, as a string, and a salt character array which is described below, and returns a printable ASCII string which starts with another salt. It is believed that, given the output of the function, the best way to find a key that will produce that output is to guess values of key until the original value of key is found.

from http://en.wikipedia.org/wiki/Crypt_(Unix)

Comments (2) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
perl -i -ne 'print if $. == 3..5' <filename>
2012-01-27 23:15:10
User: bashrc
Functions: perl
Tags: bash vi
3
Up
Down
Remove a range of lines from a file
Show sample output | Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate
genRandomText() { perl -e '$n=shift; print chr(int(rand(26)) + 97) for 1..$n; print "\n"' $1;}
2012-01-21 00:21:20
User: putnamhill
Functions: perl
Tags: perl random for loop
-1
Up
Down
Generate Random Text based on Length

If you don't have seq, you can use perl.

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate