Get a stream feed from a Twitter user

step1 ; step2 ; step3 ; step4 ; curl -o- --get 'https://stream.twitter.com/1/statuses/filter.json' --header "$oauth_header" --data "follow=$id"
*** CAREFULLY READ THE NOTES **** *** THIS DOES NOT WORK "OUT OF THE BOX" *** You'll need a few minutes of CAREFUL reading before making your own Twitter feed: In 2010 simple command line Twitter feed requests all stopped working because Twitter upgraded to SSL security. Https requests for a filtered Twitter stream feed now require a special header called "oauth_header". The benefit is that your stream feed and login info is securely encrypted. The bad news is that an "oauth_header" takes some work to build. Fortunately, four functions, imaginatively named step1, step2, step3 and step4 can be used to build a customized oauth_header for you in a few minutes. Now, go look at "step1" to start creating your own oauth_header!
Sample Output
(this output is with the --verbose option added to the end of the curl command)
* About to connect() to stream.twitter.com port 443 (#0)
*   Trying 199.59.148.138... connected
* Connected to stream.twitter.com (199.59.148.138) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using SSL_RSA_WITH_RC4_128_MD5
* Server certificate:
*       subject: CN=stream.twitter.com,OU=Twitter Operations,O="Twitter, Inc.",L=San Francisco,ST="California    ",C=US
*       start date: Jul 26 00:00:00 2011 GMT
*       expire date: Jul 25 23:59:59 2013 GMT
*       common name: stream.twitter.com
*       issuer: CN=VeriSign Class 3 Secure Server CA - G3,OU=Terms of use at https://www.verisign.com/rpa (c)10,OU=VeriSign Trust Network,O="VeriSign, Inc.",C=US
> GET /1/statuses/filter.json?follow=19258798 HTTP/1.1
> User-Agent: curl/7.21.7 (x86_64-redhat-linux-gnu) libcurl/7.21.7 NSS/3.13.1.0 zlib/1.2.5 libidn/1.22 libssh2/1.2.7
> Host: stream.twitter.com
> Accept: */*
> Authorization: OAuth oauth_consumer_key="YuxgmmO3Ysd", oauth_nonce="28767", oauth_signature="4v0%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1331680261", oauth_token="jBDQXlldo33LQNnds", oauth_version="1.0"
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Transfer-Encoding: chunked

These Might Interest You

  • This is the SECOND command in a set for five that are needed for a Twitter stream feed. This command creates variable "b", the so-called "base string" required for oauth in Twitter stream feed requests. (The 256 char limit prevents giving it a better name) We use five environment variables created by a previous step: id, k1, once, ts and k3. The five environment variables are created in a separate command, please see my other commands. For more information on the signature base string, see dev.twitter.com/apps, click on any app (or create a new one) and then go to the "OAuth Tool" tab.


    10
    step2(){ b="GET&https%3A%2F%2Fstream.twitter.com%2F1%2Fstatuses%2Ffilter.json&follow%3D${id}%26oauth_consumer_key%3D${k1}%26oauth_nonce%3D${once}%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D${ts}%26oauth_token%3D${k3}%26oauth_version%3D1.0";}
    nixnax · 2012-03-11 22:16:11 0
  • Twitter stream feeds now require authentication. This command is the FIRST in a set of five commands you'll need to get Twitter authorization for your final Twitter command. *** IMPORTANT *** Before you start, you have to get some authorization info for your "app" from Twitter. Carefully follow the instructions below: Go to dev.twitter.com/apps and choose "Create a new application". Fill in the form. You can pick any name for your app. After submitting, click on "Create my access token". Keep the resulting page open, as you'll need information from it below. If you closed the page, or want to get back to it in the future, just go to dev.twitter.com/apps Now customize FIVE THINGS on the command line as follows: 1. Replace the string "Consumer key" by copying & pasting your custom consumer key from the Twitter apps page. 2. Replace the string "Consumer secret" by copying & pasting your consumer secret from the Twitter apps page. 3. Replace the string "Access token" by copying & pasting your access token from the Twitter apps page. 4. Replace string "Access token secret" by copying & pasting your own token secret from the Twitter apps page. 5. Replace the string 19258798 with the Twitter UserID NUMBER (this is **NOT** the normal Twitter NAME of the user you want the tweet feed from. If you don't know the UserID number, head over to www.idfromuser.com and type in the user's regular Twitter name. The site will return their Twitter UserID number to you. 19258798 is the Twitter UserID for commandlinefu, so if you don't change that, you'll receive commandlinefu tweets, uhm... on the commandline :) Congratulations! You're done creating all the keys! Environment variables k1, k2, k3, and k4 now hold the four Twitter keys you will need for your next step. The variables should really have been named better, e.g. "Consumer_key", but in later commands the 256-character limit forced me to use short, unclear names here. Just remember k stands for "key". Again, remember, you can always review your requested Twitter keys at dev.twitter.com/apps. Our command line also creates four additional environment variables that are needed in the oauth process: "once", "ts", "hmac" and "id". "once" is a random number used only once that is part of the oauth procedure. HMAC is the actual key that will be used later for signing the base string. "ts" is a timestamp in the Posix time format. The last variable (id) is the user id number of the Twitter user you want to get feeds from. Note that id is ***NOT*** the twitter name, if you didn't know that, see www.idfromuser.com If you want to learn more about oauth authentication, visit oauth.net and/or go to dev.twitter.com/apps, click on any of your apps and then click on "Oauth tool" Now go look at my next command, i.e. step2, to see what happens next to these eight variables.


    9
    step1() { k1="Consumer key" ; k2="Consumer secret" ; k3="Access token" ; k4="Access token secret" ; once=$RANDOM ; ts=$(date +%s) ; hmac="$k2&$k4" ; id="19258798" ; }
    nixnax · 2012-03-11 20:40:56 0
  • I was using some twitter bots and twitter bloks my IP. Now im using some proxys and i use these command to validate the tweet was published.


    1
    lynx --dump twitter.com/(username) | sed -n "132,134 p"
    xmuda · 2013-07-06 16:28:10 0
  • This is the THIRD in a set of five commands. See my other commands for the previous two. This step creates the oauth 1.0 token as explained in http://oauth.net/core/1.0/ The token is required for a Twitter filtered stream feed (and almost all Twitter API calls) This token is simply an encrypted version of your base string. The encryption key used is your hmac. The last part of the command scans the Base64 token string for '+', '/', and '=' characters and converts them to percentage-hex escape codes. (URI-escapeing). This is also a good example of where the $() syntax of Bash command substitution fails, while the backtick form ` works - the right parenthesis in the case statement causes a syntax error if you try to use the $() syntax here. See my previous two commands step1 and step2 to see how the base string variable $b and hmac variable $hmac are generated.


    10
    step3() { s=$(echo -n $b | openssl dgst -sha1 -hmac $hmac -binary | openssl base64); signature=`for((i=0;i<${#s};i++)); do case ${s:i:1} in +) e %2B;; /) e %2F;; =) e %3D;; *) e ${s:i:1};; esac ; done` ; } ; e() { echo -n $1; }
    nixnax · 2012-03-11 10:44:01 0

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands



Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: