Commands tagged url sorted by votes

Commands tagged url (23)

sorted by

Match a URL

For the record: I didn't build this. Just shared what I found that worked. Apologies to the original author! I decided I should fix the case where http://example.com is not matched for the next time I need this. So I read rfc1035 and formalized the host name regex. If anyone finds any more holes, please comment.
This is sample output - yours may be different.
6

egrep 'https?://([[:alpha:]]([-[:alnum:]]+[[:alnum:]])*\.)+[[:alpha:]]{2,3}(:\d+)?(/([-\w/_\.]*(\?\S+)?)?)?'

putnamhill · 2009-11-28 15:41:42 33
Google URL shortener

Use curl and sed to shorten an URL using goo.gl without any other api Show Sample Output
This is sample output - yours may be different.
```
curl -s -d'&url=www.google.com' http://goo.gl/api/url | sed -e 's/{"short_url":"//' -e 's/","added_to_history":false}/\n/'

http://goo.gl/fbsS
```
6

curl -s -d'&url=URL' http://goo.gl/api/url | sed -e 's/{"short_url":"//' -e 's/","added_to_history":false}/\n/'

Soubsoub · 2010-10-01 23:20:08 11
url redirect tracer with curl

tracing redirects for a given url shortener Show Sample Output
This is sample output - yours may be different.
```
Location: https://www.shorturl.at/dfIJQ
location: https://youtu.be/dQw4w9WgXcQ
location: https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be
```
6

curl --silent -I -L shorturl.at/dfIJQ | grep -i location

aysadk · 2022-09-04 19:31:46 840
Google URL shortener

Shorter and made into a function. Show Sample Output
This is sample output - yours may be different.
```
http://goo.gl/jYVX
```
4

googl () { curl -s -d "url=${1}" http://goo.gl/api/url | sed -n "s/.*:\"$[^\"]*$.*/\1\n/p" ;}

dabom · 2010-10-03 02:52:44 4
ThePirateBay.org torrent search

This one-liner greps first 30 direct URLs for .torrent files matching your search querry, ordered by number of seeds (descending; determined by the second number after your querry, in this case 7; for other options just check the site via your favorite web-browser). You don't have to care about grepping the torrent names as well, because they are already included in the .torrent URL (except for spaces and some other characters replaced by underscores, but still human-readable). Be sure to have some http://isup.me/ macro handy (someone often kicks the ethernet cables out of their servers ;) ). I've also coded a more user-friendly ash (should be BASH compatible) script, which also lists the total size of download and number of seeds/peers (available at http://saironiq.blogspot.com/2011/04/my-shell-scripts-4-thepiratebayorg.html - may need some tweaking, as it was written for a router running OpenWrt and transmission). Happy downloading!
This is sample output - yours may be different.
4

wget -U Mozilla -qO - "http://thepiratebay.org/search/your_querry_here/0/7/0" | grep -o 'http\:\/\/torrents\.thepiratebay\.org\/.*\.torrent'

sairon · 2011-04-15 15:01:16 32
ThePirateBay.org torrent search

usage: tpb searchterm example: tpb the matrix trilogy This searches for torrents from thepiratebay and displays the top results in reverse order, so the 1st result is at the bottom instead of the top -- which is better for command line users
This is sample output - yours may be different.
4

tpb() { wget -U Mozilla -qO - $(echo "http://thepiratebay.org/search/$@/0/7/0" | sed 's/ /\%20/g') | grep -o 'http\:\/\/torrents\.thepiratebay\.org\/.*\.torrent' | tac; }

Bonster · 2011-10-26 12:15:55 29
Google URL shortener

use curl and sed to shorten an url via goo.gl
This is sample output - yours may be different.
3

curl -s 'http://ggl-shortener.appspot.com/?url='"$1" | sed -e 's/{"short_url":"//' -e 's/"}/\n/g'

mvrilo · 2010-03-26 22:31:06 23
Parse YouTube url (get youtube video id)

url can be like any one of followings: url="MejbOFk7H6c" url="http://youtu.be/MejbOFk7H6c" url="https://youtube.com/watch?feature=player_embedded&v=MejbOFk7H6c#t" url="//www.youtube.com/v/MejbOFk7H6c?hl=ru_RU&version=3&rel=0" url="http://www.youtube.com/embed/MejbOFk7H6c?feature=player_embedded" If url mismatching, whole url will be returned. Show Sample Output
This is sample output - yours may be different.
```
MejbOFk7H6c
```
3

sh -c 'url="http://youtu.be/MejbOFk7H6c"; vid="`for i in ".*youtu\.be/$[^\/&?#]\+$" ".*youtu.\+v[=/]$[^\/&?#]\+$" ".*youtu.\+embed/$[^\/&?#]\+$"; do expr "${url}" : "${i}"; done`"; if [ -n "${vid}" ]; then echo ${vid}; else echo "${url}"; fi'

qwertyroot · 2013-09-04 19:33:09 9
Google URL shortener

(1) required: python-googl ( install by: pip install python-googl ) (2) get from google API console https://code.google.com/apis/console/ Show Sample Output
This is sample output - yours may be different.
```
$ python -c 'import googl; print googl.Googl("<your_google_api_key").shorten("'http://commandlinefu.com/'")[u"id"]'
http://goo.gl/zYAip
```
2

python -c 'import googl; print googl.Googl("<your_google_api_key>").shorten("'$someurl'")[u"id"]'

shr386 · 2012-05-31 17:14:17 4
Generate a shortened URL with is.gd

Check the API. You shouldn't need sed. The print-newline at the end is to prevent zsh from inserting a % after the end-of-output. Also works with http://v.gd Show Sample Output
This is sample output - yours may be different.
```
% isgd 'commandlinefu.com'
http://is.gd/IjoOpX
```
1

isgd () { curl 'http://is.gd/create.php?format=simple&url='"$1" ; printf "\n" }

dbbolton · 2011-08-14 23:31:39 4
Encode file path to URL

Really helpfull when play with files having spaces an other bad name. Easy to store and access names and path in just a field while saving it in a file. This format (URL) is directly supported by nautilus and firefox (and other browsers) Show Sample Output
This is sample output - yours may be different.
```
$ convert_path2uri '/tmp/a/b c+d'
file:///tmp/a/b%20c%2bd
```
1

convert_path2uri () { echo -n 'file://'; echo -n "$1" | perl -pe 's/([^a-zA-Z0-9_\/.])/sprintf("%%%.2x", ord($1))/eg' ;} #convert2uri '/tmp/a b' ### convert file path to URI

totti · 2013-07-01 08:54:45 8
Get IP address from domain

I'm not sure how reliable this command is, but it works for my needs. Here's also a variant using grep. nslookup www.example.com | grep "^Address: " | awk '{print $2}' Show Sample Output
This is sample output - yours may be different.
```
123.123.123.123
```
1

nslookup www.example.com | tail -2 | head -1 | awk '{print $2}'

wsams · 2013-09-05 20:26:45 12
Get IP address from domain

shorter (thus better ;-) Show Sample Output
This is sample output - yours may be different.
```
$ dig +short nu.nl
62.69.166.254
```
1

dig +short <domain>

Iz7ueNae · 2013-09-11 13:01:03 7
Retrieve "last modified" timestamp of web resource in UTC seconds

This command line assumes that "${url}" is the URL of the web resource. It can be useful to check the "freshness" of a download URL before a GET request. Show Sample Output
This is sample output - yours may be different.
```
1464896240
```
1

curl --silent --head "${url}" | grep 'Last-Modified:' | cut -c 16- | date -f - +'%s'

oldo · 2016-06-02 22:20:55 11
Match a URL

From http://daringfireball.net/2009/11/liberal_regex_for_matching_urls Thought it would be useful to commandlinefuers. Show Sample Output
This is sample output - yours may be different.
```
http://foo.com/blah_blah
```
0

echo "(Something like http://foo.com/blah_blah)" | grep -oP "\b(([\w-]+://?|www[.])[^\s()<>]+(?:$[\w\d]+$|([^[:punct:]\s]|/)))"

matthewbauer · 2009-11-28 00:41:57 4
Shorten url with is.gd using curl, perl

// This is description for the old command: Unfortunately we to encode the URL. It can't be done with bash (without building it ourselves) so I used Perl? Example with Perl: curl -s http://is.gd/api.php?longurl=`perl -MURI::Escape -e "print uri_escape('http://www.google.com/search?hl=en&source=hp&q=commandlinefu&aq=0&oq=commandline');"` Example without Perl: curl http://is.gd/api.php?longurl=http://www.google.com Most urls doesn't use & and ? anymore (SEO etc) so in most cases you can just use the simple version. :) Show Sample Output
This is sample output - yours may be different.
```
http://is.gd/5faqp
```
0

curl -s "http://is.gd/api.php?longurl=[long_url]"

Josso · 2009-12-07 18:52:04 9
Retrieve the final URL after redirect

Sometimes it could be very useful to obtain the final URL you'll use after several redirects. (I use this command line for my automated tests to check if every redirections are ok) Show Sample Output
This is sample output - yours may be different.
```
$ curl http://google.com -s -L -o /dev/null -w '%{url_effective}'
http://www.google.fr/
```
0

curl $URL -s -L -o /dev/null -w '%{url_effective}'

labynocle · 2012-01-25 16:11:24 5
Get IP address from domain

Show Sample Output
This is sample output - yours may be different.
```
123.123.123.123
```
0

host example.com | head -1 | awk '{print $4}'

wes · 2013-09-05 22:20:49 8
Extract domain from URl

Extracts domain and subdomain from given URl. See examples. Show Sample Output
This is sample output - yours may be different.
```
http://www.example.com/more/stuff > example.com
https://test.example.com > example.com
http://example.com > example.com
http://www.example.co.uk > example.co.uk
https://www.example.com.au > example.com.au
```
0

MYURL=http://test.example.com ; awk -F/ '{ print $3 }' <<< $MYURL | awk -F. '{ if ( $(NF-1) == "co" || $(NF-1) == "com" ) printf $(NF-2)"."; print $(NF-1)"."$(NF); }'

snafu · 2014-05-26 07:31:40 7
Opens Windows internet shortcut files (*.url files) in Firefox.

This command will use grep to read the shortcut (which in the above examle is file.url), and filter out all but the only important line, which contains the website URL, and some extra characters that will need to be removes (for example, URL=http://example.com). The cut command is then used to get rid of the URL= at the beginning. The output is then piped into Firefox, which should interpret the it as a web URL to be opened. Of course, you can replace Firefox with any other broswer. Tested in bash and sh.
This is sample output - yours may be different.
0

firefox "$(grep -i ^url=* file.url | cut -b 5-)"

nachos117 · 2014-10-08 05:56:27 8
Screenshot a part of the screen, upload to S3 and copy URL to clipboard

Poor man's Clipular.com
This is sample output - yours may be different.
0

scrot -s -e 's3cmd -P put $f s3://my-bucket-screenshots/ | tail -n 1 | cut -d " " -f 7 | xsel -i -v --primary'

runixo · 2015-10-29 13:43:58 11
Match a URL

don't have to be that complicated
This is sample output - yours may be different.
-1

cho "(Something like http://foo.com/blah_blah)" | awk '{for(i=1;i<=NF;i++){if($i~/^(http|ftp):\/\//)print $i}}'

unixmonkey7109 · 2009-11-28 03:31:41 7
Get all the HTTP HEAD responses from a list of urls in a file

urls.txt should have a fully qualified url on each line prefix with rm log.txt; to clear the log change curl command to curl --head $file | head -1 >> log.txt to just get the http status Show Sample Output
This is sample output - yours may be different.
```
==log.txt:==
/downloads HTTP/1.1 301 Moved Permanently
/downloads/ HTTP/1.1 404 Not Found
/downloads/downloads/press/logos.zip HTTP/1.1 200 OK
```
-1

for file in `cat urls.txt`; do echo -n "$file " >> log.txt; curl --head $file >> log.txt ; done

Glutnix · 2010-10-19 02:54:13 6

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Count number of files in subdirectories

For each directory from the current one, list the counts of files in each of these directories. Change the -maxdepth to drill down further through directories.

Find Files That Exceed a Specified Size Limit

Trace and view network traffic

Trace and view network traffic. I made this far too complicated.. now fixed, thanks zolden.

Disable system bell in an X session

Execute this command in a terminal to disable the system-bell during X-session lifetime.

Delete specific remote 'origin' branch 'gh-pages'

Find the package that installed a command

Search for files older than 30 days in a directory and list only their names not the full path

Create the authorization header required for a Twitter stream feed

This is the FOURTH in a set of five commands. Please see my other commands for the previous three steps. This command builds the authorization header required by Twitter. For this command to work, see my previous 3 commands (step1, step2 and step3) as they are required to build the environment variables used in this command. For more information on the authorization header, go to dev.twitter.com/apps, click on any of your apps (or create a new one) and then click on the "OAuth Tool" tab.

Get AWS temporary credentials ready to export based on a MFA virtual appliance

You might want to secure your AWS operations requiring to use a MFA token. But then to use API or tools, you need to pass credentials generated with a MFA token. This commands asks you for the MFA code and retrieves these credentials using AWS Cli. To print the exports, you can use: `awk '{ print "export AWS_ACCESS_KEY_ID=\"" $1 "\"\n" "export AWS_SECRET_ACCESS_KEY=\"" $2 "\"\n" "export AWS_SESSION_TOKEN=\"" $3 "\"" }'` You must adapt the command line to include: * $MFA_IDis ARN of the virtual MFA or serial number of the physical one * TTL for the credentials

Get count of kilobytes brew cleanup would free

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged url

Commands tagged url (23) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged url (23)

sorted by