Commands by drewk (12)

  • This produces a parseable output of the last day of the month in future or past. Change the '-v-0m' to be a month plus or minus from the current system time. Show Sample Output


    -4
    date -j -v1d -v-0m -v-1d +'%m %d %Y'
    drewk · 2010-03-04 17:47:51 6
  • PDF files are simultaneously wonderful and heinous. They are wonderful in being ubiquitous and mostly being cross platform. They are heinous in being very difficult to work with from the command line, search, grep, use only the text inside the PDF, or use outside of proprietary products. xpdf is a wonderful set of PDF tools. It is on many linux distros and can be installed on OS X. While primarily an open PDF viewer for X, xpdf has the tool "pdftotext" that can extract formated or unformatted text from inside a PDF that has text. This text stream can then be further processed by grep or other tool. The '-' after the file name directs output to stdout rather than to a text file the same name as the PDF. Make sure you use version 3.02 of pdftotext or later; earlier versions clipped lines. The lines extracted from a PDF without the "-layout" option are very long. More paragraphs. Use just to test that a pattern exists in the file. With "-layout" the output resembles the lines, but it is not perfect. xpdf is available open source at http://www.foolabs.com/xpdf/


    27
    pdftotext [file] - | grep 'YourPattern'
    drewk · 2010-02-14 21:42:35 16
  • Bash has a great history system of its commands accessed by the ! built-in history expansion operator (documented elsewhere on this site or on the web). You can combine the ! operator inside the process redirection Very handy. Show Sample Output


    7
    <(!!)
    drewk · 2010-02-06 18:35:10 8
  • The format is JJJJJ YR-MO-DA HH:MM:SS TT L DUT1 msADV UTC(NIST) OTM and is explained more fully here: http://tf.nist.gov/service/acts.htm Show Sample Output


    8
    cat </dev/tcp/time.nist.gov/13
    drewk · 2009-12-03 21:40:14 11

  • -1
    grep -c '^From ' mbox_file
    drewk · 2009-11-07 03:31:05 4
  • Uses curl to download page of membership of US Congress. Use sed to strip HTML then perl to print a line starting with two tabs (a line with a representative) Show Sample Output


    -1
    curl "http://www.house.gov/house/MemberWWW.shtml" 2>/dev/null | sed -e :a -e 's/<[^>]*>//g;/</N;//ba' | perl -nle 's/^\t\t(.*$)/ $1/ and print;'
    drewk · 2009-09-24 23:37:36 15
  • How often do you make a directory (or series of directories) and then change into it to do whatever? 99% of the time that is what I do. This BASH function 'md' will make the directory path then immediately change to the new directory. By using the 'mkdir -p' switch, the intermediate directories are created as well if they do not exist. Show Sample Output


    32
    md () { mkdir -p "$@" && cd "$@"; }
    drewk · 2009-09-24 16:09:19 20
  • The sort utility is well used, but sometimes you want a little chaos. This will randomize the lines of a text file. BTW, on OS X there is no | sort -R option! There is also no | shuf These are only in the newer GNU core... This is also faster than the alternate of: | awk 'BEGIN { srand() } { print rand() "\t" $0 }' | sort -n | cut -f2- Show Sample Output


    0
    cat ~/SortedFile.txt | perl -wnl -e '@f=<>; END{ foreach $i (reverse 0 .. $#f) { $r=int rand ($i+1); @f[$i, $r]=@f[$r,$i] unless ($i==$r); } chomp @f; foreach $line (@f){ print $line; }}'
    drewk · 2009-09-24 15:42:43 5
  • The backtick operator, in general, will execute the text inside the backticks. On OS X, the pbpaste command will put the contents of the OS X clipboard to STDOUT. So if you put backticks around pbpaste, the text from the OS X clipboard is executed. If you add the pipeline | pbcopy, the output from executing the command on the clipboard is placed back on the clipboard. Note: make sure the clipboard is text only. Show Sample Output


    7
    `pbpaste` | pbcopy
    drewk · 2009-09-21 23:10:11 36
  • This pipeline will find, sort and display all files based on mtime. This could be done with find | xargs, but the find | xargs pipeline will not produce correct results if the results of find are greater than xargs command line buffer. If the xargs buffer fills, xargs processes the find results in more than one batch which is not compatible with sorting. Note the "-print0" on find and "-0" switch for perl. This is the equivalent of using xargs. Don't you love perl? Note that this pipeline can be easily modified to any data produced by perl's stat operator. eg, you could sort on size, hard links, creation time, etc. Look at stat and just change the '9' to what you want. Changing the '9' to a '7' for example will sort by file size. A '3' sorts by number of links.... Use head and tail at the end of the pipeline to get oldest files or most recent. Use awk or perl -wnla for further processing. Since there is a tab between the two fields, it is very easy to process. Show Sample Output


    3
    find $HOME -type f -print0 | perl -0 -wn -e '@f=<>; foreach $file (@f){ (@el)=(stat($file)); push @el, $file; push @files,[ @el ];} @o=sort{$a->[9]<=>$b->[9]} @files; for $i (0..$#o){print scalar localtime($o[$i][9]), "\t$o[$i][-1]\n";}'|tail
    drewk · 2009-09-21 22:11:16 11
  • This finds all the PowerPC apps recognized by OS X. A better version is: system_profiler SPApplicationsDataType 2> /dev/null | perl - wnl -e '$i=$j=$k=$p=0; @al=; $c=@al; while($j s[$i].=$al[$j]; $i++ if ($al[$j]) =~ /^\s\s\s\s\S.*:$/; $j++} while($k apps[$k++]; if (/Kind: PowerPC/s) {print; $p++;}} print "$i applications, $p P owerPC applications\n\n"' but that is more than 255 characters...


    0
    system_profiler SPApplicationsDataType | perl -nl -e '@al=<>; $c=@al; while($j<$c){ $apps[$i].=$al[$j]; $i++ if ($al[$j] ) =~ /^\s\s\s\s\S.*:$/; $j++} while($k<$i){ $_=$apps[$k++]; if (/Kind: PowerPC/s) {print;}}'
    drewk · 2009-09-06 20:56:48 8
  • diff is designed to compare two files. You can also compare directories. In this form, bash uses 'process substitution' in place of a file as an input to diff. Each input to diff can be filtered as you choose. I use find and egrep to select the files to compare.


    3
    diff <(cd /path-1; find . -type f -print | egrep -i '\.m4a$|\.mp3$') <(cd /path-2; find . f -print | egrep -i '\.m4a$|\.mp3$')
    drewk · 2009-08-17 00:49:31 6

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Run a command only when load average is below a certain threshold
Good for one off jobs that you want to run at a quiet time. The default threshold is a load average of 0.8 but this can be set using atrun.

Convert spaces in file names to underscores

Which processes are listening on a specific port (e.g. port 80)
swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

list folders containing less than 2 MB of data
This command will search all subfolders of the current directory and list the names of the folders which contain less than 2 MB of data. I use it to clean up my mp3 archive and to delete the found folders pipe the output to a textfile & run: $ while read -r line; do rm -Rv "$line"; done < textfile

Burn CD/DVD from an iso, eject disc when finished.
cdrecord -scanbus will tell you the (x,y,z) value of your cdr (for example, mine is 3,0,0)

Download entire website

Create a single PDF from multiple images with ImageMagick
Given some images (jpg or other supported formats) in input, you obtain a single PDF file with an image for every page.

batch convert Nikon RAW (nef) images to JPG
converts RAW files from a Nikon DSLR to jpg for easy viewing etc. requires ufraw package

a function to create a box of '=' characters around a given string.
First argument: string to put a box around. Second argument: character to use for box (default is '=') Same as command #4948, but shorter, and without the utility function.

Let's make screen and ssh-agent friends
When you start screen as `ssh-agent screen`, agent will die after detatch. If you don't want to take care about files when stored agent's pid/socket/etc, you have to use this command.


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: