Commands using perl (369)

  • This shell function grabs the weather forecast for the next 24 to 48 hours from weatherunderground.com. Replace <YOURZIPORLOCATION> with your zip code or your "city, state" or "city, country", then calling the function without any arguments returns the weather for that location. Calling the function with a zip code or place name as an argument returns the weather for that location instead of your default. To add a bit of color formatting to the output, use the following instead: weather(){ curl -s "http://api.wunderground.com/auto/wui/geo/ForecastXML/index.xml?query=${@:-<YOURZIPORLOCATION>}"|perl -ne '/<title>([^<]+)/&&printf "\x1B[0;34m%s\x1B[0m: ",$1;/<fcttext>([^<]+)/&&print $1,"\n"';} Requires: perl, curl Show Sample Output


    7
    weather(){ curl -s "http://api.wunderground.com/auto/wui/geo/ForecastXML/index.xml?query=${@:-<YOURZIPORLOCATION>}"|perl -ne '/<title>([^<]+)/&&printf "%s: ",$1;/<fcttext>([^<]+)/&&print $1,"\n"';}
    eightmillion · 2010-02-10 01:23:39 16
  • Converts reserved characters in a URI to their percent encoded counterparts. Alternate python version: echo "$url" | python -c 'import sys,urllib;print urllib.quote(sys.stdin.read().strip())' Show Sample Output


    7
    echo "$url" | perl -MURI::Escape -ne 'chomp;print uri_escape($_),"\n"'
    eightmillion · 2010-02-13 00:44:48 32
  • Using perl you can search for patterns spanning several lines, a thing that grep can't do. Append the list of files to above command or pipe a file through it, just as with regular grep. If you add the 's' modifier to the regex, the dot '.' also matches line endings, useful if you don't known how many lines you need are between parts of your pattern. Change '*' to '*?' to make it greedy, that is match only as few characters as possible. See also http://www.commandlinefu.com/commands/view/1764/display-a-block-of-text-with-awk to do a similar thing with awk. Edit: The undef has to be put in a begin-block, or a match in the first line would not be found.


    7
    perl -ne 'BEGIN{undef $/}; print "$ARGV\t$.\t$1\n" if m/(first line.*\n.*second line)/mg'
    hfs · 2010-03-18 15:46:10 33
  • Works only if modules are installed "the right way"


    7
    perl -MExtUtils::Installed -e '$inst = ExtUtils::Installed->new(); @modules = $inst->modules(); print join("\n", @modules);'
    braak · 2010-07-20 15:47:32 9
  • syntax follows regular command line expression. example: let's say you have a directory (with subdirs) that has say 4000 .php files. All of these files were made via script, but uh-oh, there was a typo! if the typo is "let's go jome!" but you meant it to say "let's go home!" find . -name "*.php" | xargs perl -pi -e "s/let\'s\ go\ jome\!/let\'s\ go\ home\!/g" all better :) multiline: find . -name "*.php" | xargs perl -p0777i -e 's/knownline1\nknownline2/replaced/m' indescriminate line replace: find ./ -name '*.php' | xargs perl -pi -e 's/\".*$\"/\new\ line\ content/g' Show Sample Output


    6
    find . -name "*.txt" | xargs perl -pi -e 's/old/new/g'
    neztach · 2009-02-06 00:28:03 28
  • Find all files that contain string XXX in them, change the string from XXX to YYY, make a backup copy of the file and save a list of files changed in /tmp/fileschanged.


    6
    find . -type f -exec grep -l XXX {} \;|tee /tmp/fileschanged|xargs perl -pi.bak -e 's/XXX/YYY/g'
    drossman · 2009-02-16 02:55:23 59

  • 6
    nmap -sS -O -oX /tmp/nmap.xml 10.1.1.0/24 -v -v && perl nmap2nagios.pl -v -r /tmp/10net.xml -o /etc/nagios/10net.cfg
    scubacuda · 2009-02-19 18:42:37 8
  • Finds all directories containing more than 99MB of files, and prints them in human readable format. The directories sizes do not include their subdirectories, so it is very useful for finding any single directory with a lot of large files. Show Sample Output


    6
    du -hS / | perl -ne '(m/\d{3,}M\s+\S/ || m/G\s+\S/) && print'
    Alioth · 2009-03-25 18:06:53 5

  • 6
    perl -lne '$l{$_}=length;END{for(sort{$l{$a}<=>$l{$b}}keys %l){print}}' < /usr/share/dict/words | tail
    grokskookum · 2009-09-10 14:49:03 3
  • Cool but useless. Show Sample Output


    6
    perl -nle 'printf "%0*v8b\n"," ",$_;'
    forcefsck · 2011-01-21 09:20:54 6
  • I use this in my bashrc to expand hosts defined in ~/.ssh/config: function _ssh_completion() { perl -ne 'print "$1 " if /^Host (.+)$/' ~/.ssh/config } complete -W "$(_ssh_completion)" ssh Here's a great article on how to setup your own ~/.ssh/config: http://blogs.perl.org/users/smylers/2011/08/ssh-productivity-tips.html


    6
    perl -ne 'print "$1 " if /^Host (.+)$/' ~/.ssh/config
    bashrc · 2011-08-21 14:51:20 7

  • 5
    u=`curl -d 'dl.start=Free' $(curl $1|perl -wpi -e 's/^.*"(http:\/\/rs.*)" method.*$/$1/'|egrep '^http'|head -n1)|grep "Level(3) \#2"|perl -wpi -e 's/^.*(http:\/\/rs[^\\\\]*).*$/$1/'`;sleep 60;wget $u
    fel1x · 2009-04-01 20:14:41 6
  • Mouse around the title of this item, and note that your cookies are being logged to the console. If I were evil, I could instead send everyone's cookies to my site, and then post up-votes on all my submissions using their cookies, and try to delete every other submission, until clfu was completely pwned by me, redirecting people to malware and porn sites, and so on. Update - now fixed.


    5
    perl -pi -e 's/<a href="#" onmouseover="console.log('xss! '+document.cookie)" style="position:absolute;height:0;width:0;background:transparent;font-weight:normal;">xss</a>/<\/a>/g'
    isaacs · 2009-07-08 22:26:15 30

  • 5
    perl -ne 'split /,/ ; $a+= $_[3]; END {print $a."\n";}' -f ./file.csv
    ioggstream · 2009-08-07 09:35:52 3
  • 'jot' does not come with most *nix distros, so we need to use seq to make it work. This version tested good on Fedora 11.


    5
    for x in `seq 0 25 $(curl "http://www.commandlinefu.com/commands/browse"|grep "Terminal - All commands" |perl -pe 's/.+(\d+),(\d+).+/$1$2/'|head -n1)`; do curl "http://www.commandlinefu.com/commands/browse/sort-by-votes/plaintext/$x" ; done > a.txt
    SuperFly · 2009-08-27 11:02:53 7
  • In this example the command "somecommand" will be executed and sent a SIGALARM signal if it runs for more than 10 seconds. It uses the perl alarm function. It's not 100% accurate on timing, but close enough. I found this really useful when executing scripts and commands that I knew might hang E.g. ones that connect to services that might not be running. Importantly this can be used within a sequential script. The command will not release control until either the command completes or the timeout is hit. Show Sample Output


    5
    perl -e "alarm 10; exec @ARGV" "somecommand"
    jgc · 2009-09-23 12:03:55 9

  • 5
    perl -MCPAN -e 'CPAN::Shell->install(CPAN::Shell->r)'
    grokskookum · 2009-09-24 03:27:12 4
  • Place the regular expression you want to validate between the forward slashes in the eval block. Show Sample Output


    5
    perl -we 'my $regex = eval {qr/.*/}; die "$@" if $@;'
    tlacuache · 2009-10-13 21:50:47 10

  • 5
    perl -pe 's/%([0-9a-f]{2})/sprintf("%s", pack("H2",$1))/eig'
    putnamhill · 2009-11-25 14:32:50 38
  • First we accept a socket and fork the server. Then we overload the new socket as a code ref. This code ref takes one argument, another code ref, which is used as a callback. The callback is called once for every line read on the socket. The line is put into $_ and the socket itself is passed in to the callback. Our callback is scanning the line in $_ for an HTTP GET request. If one is found it parses the file name into $1. Then we use $1 to create an new IO::All file object... with a twist. If the file is executable("-x"), then we create a piped command as our IO::All object. This somewhat approximates CGI support. Whatever the resulting object is, we direct the contents back at our socket which is in $_[0].


    5
    perl -MIO::All -e 'io(":8080")->fork->accept->(sub { $_[0] < io(-x $1 ? "./$1 |" : $1) if /^GET \/(.*) / })'
    Neo23x0 · 2010-03-31 15:03:55 8
  • This will generate 3 paragraphs with random text. Change the 3 to any number. Show Sample Output


    5
    lynx -source http://www.lipsum.com/feed/xml?amount=3|perl -p -i -e 's/\n/\n\n/g'|sed -n '/<lipsum>/,/<\/lipsum>/p'|sed -e 's/<[^>]*>//g'
    houghi · 2010-04-26 17:26:44 5
  • Safe for whitespaces in names.


    5
    dpkg -L iptables | perl -lne 'print if -f && -x'
    depesz · 2010-10-30 15:47:51 7
  • Using tail to follow and standard perl to count and print the lps when lines are written to the logfile.


    5
    tail -f /var/log/logfile|perl -e 'while (<>) {$l++;if (time > $e) {$e=time;print "$l\n";$l=0}}'
    madsen · 2011-06-21 10:28:26 7

  • 4
    find . -type d | perl -nle 'print s,/,/,g," $_"' | sort -n | tail
    sirhc · 2009-02-23 22:52:07 5
  • When Ldapsearch queries an Active directory server, all the dates are shown using a timestamp of 18 digits. This perl regexp decodes them in a more human friendly notation. 11644473600 corresponds to some microsoft epoch. Show Sample Output


    4
    ldapsearch -v -H ldap://<server> -x -D cn=<johndoe>,cn=<users>,dc=<ourdomain>,dc=<tld> -w<secret> -b ou=<lazystaff>,dc=<ourdomain>,dc=<tld> -s sub sAMAccountName=* '*' | perl -pne 's/(\d{11})\d{7}/"DATE-AD(".scalar(localtime($1-11644473600)).")"/e'
    flux · 2009-04-22 00:57:34 5
  •  < 1 2 3 4 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

find all non-html files

Show a prettified list of nearby wireless APs

Copy a file to a new directory created on the fly
You need to cp, mv, scp, ..., some files around from one place to another, and after having laboriously typed out the source path, you remember that the destination directory doesn't yet exist, and so the command will fail. So rather than killing the command line and starting over, just interpolate the results of creating the directory and echo its name. You could DRY this with a for; do; done, but that may be more trouble than it's worth.

Repeat a command until stopped
In this case it runs the command 'curl localhost:3000/site/sha' waiting the amount of time in sleep, ie: 1 second between runs, appending each run to the console. This works well for any command where the output is less than your line width This is unlike watch, because watch always clears the display.

static compilation

list files recursively by size

Draw a Sierpinski triangle
OK, not the most useful but a good way to impress friends. Requires the "display" command from ImageMagick.

Numeric zero padding file rename
This uses Perl's rename utility (you may have to call it as prename on your box) and won't choke on spaces or other characters in filenames. It will also zero pad a number even in filenames like "vacation-4.jpg".

Paste the contents of OS X clipboard into a new text file

Printout a list of field numbers (awk index) from a CSV file with headers as first line.
Useful to identify the field number in big CSV files with large number of fields. The index is the reference to use in processing with commands like 'cut' or 'awk' involved.


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: