Commands by jnash (12)

  • It uses curl --url-encode to encode long URLs *properly* and parses XML with xmlstarlet. If ~/.bitlyrc were to contain login:apikey then a script could read the apiKey and login from ~/.bitlyrc like so: login=$(sed 's/:.*//' < $HOME/.bitlyrc) apikey=$(sed 's/[^:]*://' < $HOME/.bitlyrc) curl -s --data-urlencode 'longUrl='$1 --data-urlencode 'login='$login --data-urlencode 'apiKey='$apikey 'http://api.bit.ly/shorten?version=2.0.1&format=xml' | xmlstarlet sel -T -t -m "//shortUrl" -v "." | line Show Sample Output


    0
    curl -s --data-urlencode 'longUrl='$1 --data-urlencode 'login='$login --data-urlencode 'apiKey='$apikey 'http://api.bit.ly/shorten?version=2.0.1&format=xml' | xmlstarlet sel -T -t -m "//shortUrl" -v "." | line
    jnash · 2010-01-02 11:32:42 0
  • If you are behind a restrictive proxy/firewall that blocks port 22 connections but allows SSL on 443 (like most do) then you can still push changes to your github repository. Your .ssh/config file should contain: Host * ForwardX11 no TCPKeepAlive yes ProtocolKeepAlives 30 ProxyCommand /usr/local/bin/proxytunnel -v -p -d %h:443 Host User git Hostname ssh.github.com ChallengeResponseAuthentication yes IdentityFile ~/.ssh/id_rsa IdentitiesOnly yes Basically proxytunnel "tunnels" your ssh connection through port 443. You could also use corkscrew or some other tunneling program that is available in your distro's repository. PS: I generally use "github.com" as the SSH-HOST so that urls of the kind git@github.com:USER/REPO.git work transparently :) You


    3
    git remote add origin git@SSH-HOST:<USER>/<REPOSITORY>.git
    jnash · 2009-11-19 06:57:50 1
  • I know this has been beaten to death but finding video files using mime types and printing the "hours of video" for each directory is (IMHO) easier to parse than just a single total. Output is in minutes. Among the other niceties is that it omits printing of non-video files/folders PS: Barely managed to fit it within the 255 character limit :D Show Sample Output


    0
    for item in *;do echo -n "$item - ";find "$item" -type f -print0 | xargs -0 file -iNf - | grep video | cut -d: -f1 | xargs -d'\n' /usr/share/doc/mplayer/examples/midentify | grep ID_LENGTH | awk -F= '{sum+=$2} END {print(sum/60)}'; done | grep -v ' - 0$'
    jnash · 2009-11-19 06:28:15 0
  • Uses mime-type of files rather than relying on file extensions to find files of a certain type. This can obviously be extended to finding files of any other type as well.. like plain text files, audio, etc.. In reference to displaying the total hours of video (which was earlier posted in command line fu, but relied on the user having to supply all possible video file formats) we can now do better: find ./ -type f -print0 | xargs -0 file -iNf - | grep video | cut -d: -f1 | xargs -d'\n' /usr/share/doc/mplayer/examples/midentify | grep ID_LENGTH | awk -F "=" '{sum += $2} END {print sum/60/60; print "hours"}'


    1
    find ./ -type f -print0 | xargs -0 file -iNf - | grep video | cut -d: -f1
    jnash · 2009-11-19 06:05:36 3
  • enlubtsqyuse cat /tmp/out subsequently Show Sample Output


    1
    shuf -n1 /usr/share/dict/words | tee >(sed -e 's/./&\n/g' | shuf | tr -d '\n' | line) > /tmp/out
    jnash · 2009-04-05 05:29:06 3
  • Does that count as a win for bzip2? Show Sample Output


    -2
    < /dev/urandom tr -dc A-Za-z0-9_ | head -c $((1024 * 1024)) | tee >(gzip -c > out.gz) >(bzip2 -c > out.bz) > /dev/null
    jnash · 2009-04-04 13:23:01 8
  • Are there any creative pieces of music that can be created using beep and the shell? I'd love to hear it!


    5
    man beep | sed -e '1,/Note/d; /BUGS/,$d' | awk '{print $2}' | xargs -IX sudo beep -f X -l 500
    jnash · 2009-04-01 06:48:48 5
  • http://en.wikipedia.org/wiki/Year_2038_problem Some other notable dates that have passed: date -d@1234567890 date -d@1000000000 Show Sample Output


    1
    date -d @$(echo $((2 ** 31 - 1)))
    jnash · 2009-03-30 19:42:20 1
  • Extremely useful to maintain backups if you're using Dropbox. This mirrors the entire directory structure and places symlinks in each to the original file. Instead of copying over the data again to the ~/Dropbox folder creating a symbolic link tree is much more sensible in terms of space usage. This has to be supplemented by another script that removes dead symlinks in the Dropbox folder which point to files that have been moved/removed. find -L ./ -type l -delete And then removing empty directories find ./ -type d -exec rmdir 2>/dev/null {} \; **Actually after some finding I found lndir which creates symbolic trees but it wasn't in the Arch repos so.. ;)


    0
    find /home/user/doc/ -type d -printf "mkdir -vp '/home/user/Dropbox%p'\n" -o -type f -printf "ln -vs '%p' '/home/user/Dropbox%p'\n" | sh
    jnash · 2009-03-29 09:25:12 8
  • Might be more useful if you were able to print it in Days HH:MM:SS format as: perl -e '@p=gmtime(234234);printf("%d Days %02d:%02d:%02ds\n",@p[7,2,1,0]);' But I'm not exactly sure how to replace the 234234 with the output of the countdown time. (Having some problems with nested quoting/command substitution). Help would be appreciated :)


    0
    watch --no-title -d -n 1 'echo `date -d "next Thursday" +%s` "-" `date +%s` | bc -l'
    jnash · 2009-03-29 06:53:09 5
  • Python comments begin with a #. Modify to suit other languages. Other uses: Instead of m0 use m$ for end of file or d for deleting all comments.


    4
    :g:^\s*#.*:m0
    jnash · 2009-03-27 18:56:36 7
  • Extensible to other ugly extensions like *.JPG, *.Jpg etc.. Leave out the last pipe to sh to perform a dry run.


    -1
    find ./ -iname "*.mp3" -type f -printf "mv '%p' '%p'\n" | sed -e "s/mp3'$/mp3'/I" | sh
    jnash · 2009-03-27 13:42:40 7

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

split a file by a specific number of lines
Splits the file "my_file" every 500 lines. Will create files called xx01 xx02 and so on. You can change the prefix by using the -f option. Comes in handy for splitting logfiles for example. I am using it for feeding a logfile parser with smaller files instead of one big file (due to performance reasons)

Download file with multiple simultaneous connections
`aria2c` (from the aria2 project) allows. Change -s 4 to an arbitrary number of segments to control the number of concurrent connections. It is also possible to provide multiple URLs to the same content (potentially over multiple protocols) to download the file concurrently from multiple hosts.

Remove security limitations from PDF documents using ghostscript (for Windows)
#4345 also works under windows

Send an email from the terminal when job finishes
Might as well include the status code it exited with so you know right away if it failed or not.

Simple Gumblar check command

Exclude inserting a table from a sql import
Starting with a large MySQL dump file (*.sql) remove any lines that have inserts for the specified table. Sometimes one or two tables are very large and uneeded, eg. log tables. To exclude multiple tables you can get fancy with sed, or just run the command again on subsequently generated files.

Content search.
Grep will read the contents of each file in PWD and will use the REs $1 $2 ... $n to match the contents. In case of match, grep will print the appropriate file, line number and the matching line. It's just easier to write $ ff word1 word2 word3 Instead of $ grep -rinE 'word1|word2|word3' .

Wait for file to stop changing
This loop will finish if a file hasn't changed in the last 10 seconds. . It checks the file's modification timestamp against the clock. If 10 seconds have elapsed without any change to the file, then the loop ends. . This script will give a false positive if there's a 10 second delay between updates, e.g. due to network congestion . How does it work? 'date +%s' gives the current time in seconds 'stat -c %Y' gives the file's last modification time in seconds '$(( ))' is bash's way of doing maths '[ X -lt 10 ]' tests the result is Less Than 10 otherwise sleep for 1 second and repeat . Note: Clever as this script is, inotify is smarter.

Using NMAP to check if a port is open or close
Using NMAP to check to see if port 22(SSH) is open on servers and network devices.

Ask user to confirm
Returns true if user presses the key. Use it like $ Confirm "Continue" && do action


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: