Commands tagged url sorted by votes

Commands tagged url (23)

sorted by

Match a URL

For the record: I didn't build this. Just shared what I found that worked. Apologies to the original author! I decided I should fix the case where http://example.com is not matched for the next time I need this. So I read rfc1035 and formalized the host name regex. If anyone finds any more holes, please comment.
This is sample output - yours may be different.
6

egrep 'https?://([[:alpha:]]([-[:alnum:]]+[[:alnum:]])*\.)+[[:alpha:]]{2,3}(:\d+)?(/([-\w/_\.]*(\?\S+)?)?)?'

putnamhill · 2009-11-28 15:41:42 33
Google URL shortener

Use curl and sed to shorten an URL using goo.gl without any other api Show Sample Output
This is sample output - yours may be different.
```
curl -s -d'&url=www.google.com' http://goo.gl/api/url | sed -e 's/{"short_url":"//' -e 's/","added_to_history":false}/\n/'

http://goo.gl/fbsS
```
6

curl -s -d'&url=URL' http://goo.gl/api/url | sed -e 's/{"short_url":"//' -e 's/","added_to_history":false}/\n/'

Soubsoub · 2010-10-01 23:20:08 11
url redirect tracer with curl

tracing redirects for a given url shortener Show Sample Output
This is sample output - yours may be different.
```
Location: https://www.shorturl.at/dfIJQ
location: https://youtu.be/dQw4w9WgXcQ
location: https://www.youtube.com/watch?v=dQw4w9WgXcQ&feature=youtu.be
```
6

curl --silent -I -L shorturl.at/dfIJQ | grep -i location

aysadk · 2022-09-04 19:31:46 935
Google URL shortener

Shorter and made into a function. Show Sample Output
This is sample output - yours may be different.
```
http://goo.gl/jYVX
```
4

googl () { curl -s -d "url=${1}" http://goo.gl/api/url | sed -n "s/.*:\"$[^\"]*$.*/\1\n/p" ;}

dabom · 2010-10-03 02:52:44 4
ThePirateBay.org torrent search

This one-liner greps first 30 direct URLs for .torrent files matching your search querry, ordered by number of seeds (descending; determined by the second number after your querry, in this case 7; for other options just check the site via your favorite web-browser). You don't have to care about grepping the torrent names as well, because they are already included in the .torrent URL (except for spaces and some other characters replaced by underscores, but still human-readable). Be sure to have some http://isup.me/ macro handy (someone often kicks the ethernet cables out of their servers ;) ). I've also coded a more user-friendly ash (should be BASH compatible) script, which also lists the total size of download and number of seeds/peers (available at http://saironiq.blogspot.com/2011/04/my-shell-scripts-4-thepiratebayorg.html - may need some tweaking, as it was written for a router running OpenWrt and transmission). Happy downloading!
This is sample output - yours may be different.
4

wget -U Mozilla -qO - "http://thepiratebay.org/search/your_querry_here/0/7/0" | grep -o 'http\:\/\/torrents\.thepiratebay\.org\/.*\.torrent'

sairon · 2011-04-15 15:01:16 34
ThePirateBay.org torrent search

usage: tpb searchterm example: tpb the matrix trilogy This searches for torrents from thepiratebay and displays the top results in reverse order, so the 1st result is at the bottom instead of the top -- which is better for command line users
This is sample output - yours may be different.
4

tpb() { wget -U Mozilla -qO - $(echo "http://thepiratebay.org/search/$@/0/7/0" | sed 's/ /\%20/g') | grep -o 'http\:\/\/torrents\.thepiratebay\.org\/.*\.torrent' | tac; }

Bonster · 2011-10-26 12:15:55 29
Google URL shortener

use curl and sed to shorten an url via goo.gl
This is sample output - yours may be different.
3

curl -s 'http://ggl-shortener.appspot.com/?url='"$1" | sed -e 's/{"short_url":"//' -e 's/"}/\n/g'

mvrilo · 2010-03-26 22:31:06 23
Parse YouTube url (get youtube video id)

url can be like any one of followings: url="MejbOFk7H6c" url="http://youtu.be/MejbOFk7H6c" url="https://youtube.com/watch?feature=player_embedded&v=MejbOFk7H6c#t" url="//www.youtube.com/v/MejbOFk7H6c?hl=ru_RU&version=3&rel=0" url="http://www.youtube.com/embed/MejbOFk7H6c?feature=player_embedded" If url mismatching, whole url will be returned. Show Sample Output
This is sample output - yours may be different.
```
MejbOFk7H6c
```
3

sh -c 'url="http://youtu.be/MejbOFk7H6c"; vid="`for i in ".*youtu\.be/$[^\/&?#]\+$" ".*youtu.\+v[=/]$[^\/&?#]\+$" ".*youtu.\+embed/$[^\/&?#]\+$"; do expr "${url}" : "${i}"; done`"; if [ -n "${vid}" ]; then echo ${vid}; else echo "${url}"; fi'

qwertyroot · 2013-09-04 19:33:09 9
Google URL shortener

(1) required: python-googl ( install by: pip install python-googl ) (2) get from google API console https://code.google.com/apis/console/ Show Sample Output
This is sample output - yours may be different.
```
$ python -c 'import googl; print googl.Googl("<your_google_api_key").shorten("'http://commandlinefu.com/'")[u"id"]'
http://goo.gl/zYAip
```
2

python -c 'import googl; print googl.Googl("<your_google_api_key>").shorten("'$someurl'")[u"id"]'

shr386 · 2012-05-31 17:14:17 4
Generate a shortened URL with is.gd

Check the API. You shouldn't need sed. The print-newline at the end is to prevent zsh from inserting a % after the end-of-output. Also works with http://v.gd Show Sample Output
This is sample output - yours may be different.
```
% isgd 'commandlinefu.com'
http://is.gd/IjoOpX
```
1

isgd () { curl 'http://is.gd/create.php?format=simple&url='"$1" ; printf "\n" }

dbbolton · 2011-08-14 23:31:39 4
Encode file path to URL

Really helpfull when play with files having spaces an other bad name. Easy to store and access names and path in just a field while saving it in a file. This format (URL) is directly supported by nautilus and firefox (and other browsers) Show Sample Output
This is sample output - yours may be different.
```
$ convert_path2uri '/tmp/a/b c+d'
file:///tmp/a/b%20c%2bd
```
1

convert_path2uri () { echo -n 'file://'; echo -n "$1" | perl -pe 's/([^a-zA-Z0-9_\/.])/sprintf("%%%.2x", ord($1))/eg' ;} #convert2uri '/tmp/a b' ### convert file path to URI

totti · 2013-07-01 08:54:45 8
Get IP address from domain

I'm not sure how reliable this command is, but it works for my needs. Here's also a variant using grep. nslookup www.example.com | grep "^Address: " | awk '{print $2}' Show Sample Output
This is sample output - yours may be different.
```
123.123.123.123
```
1

nslookup www.example.com | tail -2 | head -1 | awk '{print $2}'

wsams · 2013-09-05 20:26:45 12
Get IP address from domain

shorter (thus better ;-) Show Sample Output
This is sample output - yours may be different.
```
$ dig +short nu.nl
62.69.166.254
```
1

dig +short <domain>

Iz7ueNae · 2013-09-11 13:01:03 7
Retrieve "last modified" timestamp of web resource in UTC seconds

This command line assumes that "${url}" is the URL of the web resource. It can be useful to check the "freshness" of a download URL before a GET request. Show Sample Output
This is sample output - yours may be different.
```
1464896240
```
1

curl --silent --head "${url}" | grep 'Last-Modified:' | cut -c 16- | date -f - +'%s'

oldo · 2016-06-02 22:20:55 11
Match a URL

From http://daringfireball.net/2009/11/liberal_regex_for_matching_urls Thought it would be useful to commandlinefuers. Show Sample Output
This is sample output - yours may be different.
```
http://foo.com/blah_blah
```
0

echo "(Something like http://foo.com/blah_blah)" | grep -oP "\b(([\w-]+://?|www[.])[^\s()<>]+(?:$[\w\d]+$|([^[:punct:]\s]|/)))"

matthewbauer · 2009-11-28 00:41:57 4
Shorten url with is.gd using curl, perl

// This is description for the old command: Unfortunately we to encode the URL. It can't be done with bash (without building it ourselves) so I used Perl? Example with Perl: curl -s http://is.gd/api.php?longurl=`perl -MURI::Escape -e "print uri_escape('http://www.google.com/search?hl=en&source=hp&q=commandlinefu&aq=0&oq=commandline');"` Example without Perl: curl http://is.gd/api.php?longurl=http://www.google.com Most urls doesn't use & and ? anymore (SEO etc) so in most cases you can just use the simple version. :) Show Sample Output
This is sample output - yours may be different.
```
http://is.gd/5faqp
```
0

curl -s "http://is.gd/api.php?longurl=[long_url]"

Josso · 2009-12-07 18:52:04 9
Retrieve the final URL after redirect

Sometimes it could be very useful to obtain the final URL you'll use after several redirects. (I use this command line for my automated tests to check if every redirections are ok) Show Sample Output
This is sample output - yours may be different.
```
$ curl http://google.com -s -L -o /dev/null -w '%{url_effective}'
http://www.google.fr/
```
0

curl $URL -s -L -o /dev/null -w '%{url_effective}'

labynocle · 2012-01-25 16:11:24 5
Get IP address from domain

Show Sample Output
This is sample output - yours may be different.
```
123.123.123.123
```
0

host example.com | head -1 | awk '{print $4}'

wes · 2013-09-05 22:20:49 8
Extract domain from URl

Extracts domain and subdomain from given URl. See examples. Show Sample Output
This is sample output - yours may be different.
```
http://www.example.com/more/stuff > example.com
https://test.example.com > example.com
http://example.com > example.com
http://www.example.co.uk > example.co.uk
https://www.example.com.au > example.com.au
```
0

MYURL=http://test.example.com ; awk -F/ '{ print $3 }' <<< $MYURL | awk -F. '{ if ( $(NF-1) == "co" || $(NF-1) == "com" ) printf $(NF-2)"."; print $(NF-1)"."$(NF); }'

snafu · 2014-05-26 07:31:40 8
Opens Windows internet shortcut files (*.url files) in Firefox.

This command will use grep to read the shortcut (which in the above examle is file.url), and filter out all but the only important line, which contains the website URL, and some extra characters that will need to be removes (for example, URL=http://example.com). The cut command is then used to get rid of the URL= at the beginning. The output is then piped into Firefox, which should interpret the it as a web URL to be opened. Of course, you can replace Firefox with any other broswer. Tested in bash and sh.
This is sample output - yours may be different.
0

firefox "$(grep -i ^url=* file.url | cut -b 5-)"

nachos117 · 2014-10-08 05:56:27 9
Screenshot a part of the screen, upload to S3 and copy URL to clipboard

Poor man's Clipular.com
This is sample output - yours may be different.
0

scrot -s -e 's3cmd -P put $f s3://my-bucket-screenshots/ | tail -n 1 | cut -d " " -f 7 | xsel -i -v --primary'

runixo · 2015-10-29 13:43:58 11
Match a URL

don't have to be that complicated
This is sample output - yours may be different.
-1

cho "(Something like http://foo.com/blah_blah)" | awk '{for(i=1;i<=NF;i++){if($i~/^(http|ftp):\/\//)print $i}}'

unixmonkey7109 · 2009-11-28 03:31:41 7
Get all the HTTP HEAD responses from a list of urls in a file

urls.txt should have a fully qualified url on each line prefix with rm log.txt; to clear the log change curl command to curl --head $file | head -1 >> log.txt to just get the http status Show Sample Output
This is sample output - yours may be different.
```
==log.txt:==
/downloads HTTP/1.1 301 Moved Permanently
/downloads/ HTTP/1.1 404 Not Found
/downloads/downloads/press/logos.zip HTTP/1.1 200 OK
```
-1

for file in `cat urls.txt`; do echo -n "$file " >> log.txt; curl --head $file >> log.txt ; done

Glutnix · 2010-10-19 02:54:13 6

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

colorize sequences of digits

Credits go to Flatcap https://www.commandlinefu.com/commands/by/flatcap

Rename files in batch

list block devices

Shows all block devices in a tree with descruptions of what they are.

Rename all .jpeg and .JPG files to .jpg

Remove comments and empty lines from a conf file

Some conf file are very long (squid.conf) This command help to read it.

last.fm rss parser

Quick and kludgy rss parser for the recent tracks rss feed from last.fm. Extracts artist and track link.

Get info about a GitHub project

In this example we search for 'vim' but vim doesn't have a project on github right now. That's ok, this command still searches for every project that has 'vim' in their description (forks, plugins, etc). To get XML or JSON output just replace 'yaml' in the url with 'xml' or 'json'.

Calculate days on which Friday the 13th occurs (inspired from the work of the user justsomeguy)

Friday is the 5th day of the week, monday is the 1st. Output may be affected by locale.

Clear all Windows Event Log entries (cygwin)

Efficiently clear all Windows Event log entries from within a Cygwin terminal. Uses "cygstart" to launch a hidden "PowerShell" session passing a Powershell command to loop through and clear all Windows Event Log entries. Very useful for troubleshooting and debugging. The command should in theory elevate you session if needed. One liner is based on the PowerShell command: $ wevtutil el | foreach { wevtutil cl $_ }

recursively change file name from uppercase to lowercase (or viceversa)

easier way to recursively change files to lowercase using rename instead

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged url

Commands tagged url (23) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged url (23)

sorted by