Facebook Email Scraper

fbemailscraper YourFBEmail Password
(Apparently it is too long so I put it in sample output, I hope that is OK.) Run the long command (or put it in your .bashrc) in sample output then run: fbemailscraper YourFBEmail Password Voila! Your contacts' emails will appear. Facebook seems to have gotten rid of the picture encoding of emails and replaced it with a text based version making it easy to scrape! Needs curl to run and it was made pretty quickly so there might be bugs.
Sample Output
fbemailscraper () { email="$1" ; password="$2" ; id="$(curl -m 5 --retry 1 -c /tmp/cookies -A "Opera" -Ls -d "email=$email&pass=$password&persistent=1" "https://login.facebook.com/login.php?m&next=http://m.facebook.com/profile.php" | grep -o "\&am\p\;id=.*&am\p;v=feed\&am\p;refid=17" | sed -e "s/.*profile.*id=\(.*\)\&am\p\;v=feed.*/\1/g")" ; curl -m 5 --retry 1 -b /tmp/cookies -Ls -A "Opera" "http://www.facebook.com/ajax/typeahead_friends.php?u=$id&__a=1" | tr "\"" "\n" | grep facebook | tr -d "\\\\" | sed "s/www/m/g" | awk '{print $0 "?v=info&refid=17"}' | sed "/profile.php/s/?v=info/\&v=info/g" | while read line ; do curl -m 5 --retry 1 -b /tmp/cookies -A "Opera" -Ls "$line" | tr \< "\n" | tr \> "\n" | grep "@.*\." | grep -v " " ; done ; rm /tmp/cookies ; }

4
By: dabom
2010-01-31 00:44:35

1 Alternatives + Submit Alt

What Others Think

Didn't work for me. All I got in return was: rm: remove regular file `/tmp/cookies'?
bkuri · 661 weeks and 5 days ago
OK, fixed it. There was a problem with "amp" being erased messing up the "id" code. Copy it and try again.
dabom · 661 weeks and 5 days ago
Still a no-go, sorry...
bkuri · 661 weeks and 4 days ago
Do you have curl and any of the other programs I used installed? If your password has weird symbols/spaces in it that might cause problems. Make sure you are using bash I guess, not sure how it will work with other shells.
dabom · 661 weeks and 4 days ago
Here it did not work until I logged out on the website. Script only gives you the mail addresses without anything else. Is there a way to add name/surname to the list?
TabTwo · 661 weeks and 4 days ago
I've updated the command to add a timeout and retry. As for names, here is a quick fix, it will put the profile URL above the email (this is all I will be doing, but it shouldn't be too hard to make it extract names.) fbemailscraper () { email="$1" ; password="$2" ; id="$(curl -m 5 --retry 1 -c /tmp/cookies -A "Opera" -Ls -d "email=$email&pass=$password&persistent=1" "https://login.facebook.com/login.php?m&next=http://m.facebook.com/profile.php" | grep -o "\&am\p\;id=.*&am\p;v=feed\&am\p;refid=17" | sed -e "s/.*profile.*id=\(.*\)\&am\p\;v=feed.*/\1/g")" ; curl -m 5 --retry 1 -b /tmp/cookies -Ls -A "Opera" "http://www.facebook.com/ajax/typeahead_friends.php?u=$id&__a=1" | tr "\"" "\n" | grep facebook | tr -d "\\\\" | sed "s/www/m/g" | awk '{print $0 "?v=info&refid=17"}' | while read line ; do echo "$line" ; curl -m 5 --retry 1 -b /tmp/cookies -A "Opera" -Ls "$line" | tr \< "\n" | tr \> "\n" | grep "@.*\." | grep -v " " ; done ; rm /tmp/cookies ; }
dabom · 661 weeks and 4 days ago
Found a problem with getting email from people without nicknames. The updated profile URL script is here and the one in sample output it fixed. fbemailscraper () { email="$1" ; password="$2" ; id="$(curl -m 5 --retry 1 -c /tmp/cookies -A "Opera" -Ls -d "email=$email&pass=$password&persistent=1" "https://login.facebook.com/login.php?m&next=http://m.facebook.com/profile.php" | grep -o "\&am\p\;id=.*&am\p;v=feed\&am\p;refid=17" | sed -e "s/.*profile.*id=\(.*\)\&am\p\;v=feed.*/\1/g")" ; curl -m 5 --retry 1 -b /tmp/cookies -Ls -A "Opera" "http://www.facebook.com/ajax/typeahead_friends.php?u=$id&__a=1" | tr "\"" "\n" | grep facebook | tr -d "\\\\" | sed "s/www/m/g" | awk '{print $0 "?v=info&refid=17"}' | sed "/profile.php/s/?v=info/\&v=info/g" | while read line ; do echo "$line" ; curl -m 5 --retry 1 -b /tmp/cookies -A "Opera" -Ls "$line" | tr \< "\n" | tr \> "\n" | grep "@.*\." | grep -v " " ; done ; rm /tmp/cookies ; }
dabom · 661 weeks and 3 days ago
it doesn't work for me... i'm trying on snow leopard with curl i'm getting this if i manually execute the second curl with my id and a correct path with the cookies: for (;;);{"error":0,"errorSummary":"","errorDescription":"","errorIsWarning":false,"payload":{"friends":[]}} btw this should "export" all your friends emails, right?
asterlux · 661 weeks and 3 days ago
Yes, it will export all of your friends' emails, even if the have more than one, but not if they don't list any emails. As for the error it doesn't look like any bash error I've ever seen... Run: echo $SHELL to make sure you are running bash. Or once you open Terminal just run: bash and just run the command from there. Hope that helps.
dabom · 661 weeks and 3 days ago
yeah i got bash. no luck also couldn't it be a problem $line as it's not defined? or is it's ok? sorry for being so noob but how do i run it if save I it as .bashrc? should i do like: fbemailscraper.bashrc fbemailscraper YourFBEmail Password ?
asterlux · 661 weeks and 2 days ago
"$line" is defined by "while read line." You don't name it "fbemailscraper.bashrc" there is a file called ".bashrc" and you put the line where ever in there and you should just be able to run: fbemailscraper YourFBEmail Password Just make sure you run bash first.
dabom · 661 weeks and 2 days ago
i don't get anything, no error nothing. i should get all the emails in the commandline isn't it? i didn't have a .bashrc file so i created in my home directory /User/my_real_user_name/ but nothing and i'm running bash-3.2 maybe i need to change it some how to make work in snow leopard? any clues?
asterlux · 661 weeks and 2 days ago
Hmm, I'm not really sure but I will offer you an alternate way. Save the script below in /usr/local/bin or /usr/bin (call it fbemailscraper or something) and run: sudo chmod +x /usr/local/bin/fbemailscraper (or where ever you put it) Then run it like: fbemailscraper YourFBEmail Password Script below. #!/bin/bash email="$1" password="$2" id="$(curl -m 5 --retry 1 -c /tmp/cookies -A "Opera" -Ls -d "email=$email&pass=$password&persistent=1" "https://login.facebook.com/login.php?m&next=http://m.facebook.com/profile.php" | grep -o "\&am\p\;id=.*&am\p;v=feed\&am\p;refid=17" | sed -e "s/.*profile.*id=\(.*\)\&am\p\;v=feed.*/\1/g")" curl -m 5 --retry 1 -b /tmp/cookies -Ls -A "Opera" "http://www.facebook.com/ajax/typeahead_friends.php?u=$id&__a=1" | tr "\"" "\n" | grep facebook | tr -d "\\\\" | sed "s/www/m/g" | awk '{print $0 "?v=info&refid=17"}' | sed "/profile.php/s/?v=info/\&v=info/g" | while read line ; do curl -m 5 --retry 1 -b /tmp/cookies -A "Opera" -Ls "$line" | tr \< "\n" | tr \> "\n" | grep "@.*\." | grep -v " " ; done rm /tmp/cookies Note this one does not display profile URLs, if you want that add "echo $line ;" after "read line ; do "
dabom · 661 weeks and 2 days ago
dabom!!!! you're man ! i love you!!! thanks i finally did it :D i just had to put few new lines in the script and it worked :D
asterlux · 661 weeks ago
Try this: (replace $facebookemail, $facebookpass, $gmailemail, $gmailpass) (id="$(curl -m 5 --retry 1 -c /tmp/cookies -A "Opera" -Ls -d "email=$facebookemail&pass=$facebookpass&persistent=1" "https://login.facebook.com/login.php?m&next=http://m.facebook.com/profile.php" | grep -o "\&am\p\;id=.*&am\p;v=feed\&am\p;refid=17" | sed -e "s/.*profile.*id=\(.*\)\&am\p\;v=feed.*/\1/g")" ; curl -m 5 --retry 1 -b /tmp/cookies -Ls -A "Opera" "http://www.facebook.com/ajax/typeahead_friends.php?u=$id&__a=1" | tr "\"" "\n" | grep facebook | tr -d "\\\\" | sed "s/www/m/g" | awk '{print $0 "?v=info&refid=17"}' | sed "/profile.php/s/?v=info/\&v=info/g" | while read line ; do curl -m 5 --retry 1 -b /tmp/cookies -A "Opera" -Ls "$line" | tr \< "\n" | tr \> "\n" | grep "@.*\." | grep -v " " ; done) | xargs -I '{}' curl -s -H "Content-Type: application/atom+xml" -d "<atom:entry xmlns:atom='http://www.w3.org/2005/Atom' xmlns:gd='http://schemas.google.com/g/2005'><atom:category scheme='http://schemas.google.com/g/2005#kind' term='http://schemas.google.com/contact/2008#contact' /><gd:email rel='http://schemas.google.com/g/2005#home' address='{}' /></atom:entry>" -H "Authorization: GoogleLogin auth=$(curl -s https://www.google.com/accounts/ClientLogin -d Email=$googleemail -d Passwd=$googlepassword -d service=cp | grep Auth | sed 's/Auth=\(.*\)/\1/')" "http://www.google.com/m8/feeds/contacts/default/full"
matthewbauer · 661 weeks ago
You should probably state what the above function does but, I'm fairly sure it adds your Facebook contacts to your GMail address book. :D Pretty cool use but I suppose the next step would be to rip the names also.
dabom · 660 weeks and 6 days ago
Here's an addition to the curl part of fbemailscraper that adds names: curl -m 5 --retry 1 -b /tmp/cookies -Ls -A "Opera" "http://www.facebook.com/ajax/typeahead_friends.php?u=$id&__a=1" | tr "\"" "\n" | grep facebook | tr -d "\\\\" | sed "s/www/m/g" | awk '{print $0 "?v=info&refid=17"}' | sed "/profile.php/s/?v=info/\&v=info/g" | while read line ; do curl -m 5 --retry 1 -b /tmp/cookies -A "Opera" -Ls "$line" > /tmp/fbinfo; echo $(cat /tmp/fbinfo | grep div | sed 's|.*<div class="section"><div class="section_title">\([^<]*\)</div>.*|\1|')','$(cat /tmp/fbinfo | tr \< "\n" | tr \> "\n" | grep "@.*\." | grep -v " "); done
matthewbauer · 660 weeks and 6 days ago
curl -m 5 --retry 1 -b /tmp/cookies -Ls -A "Opera" "http://www.facebook.com/ajax/typeahead_friends.php?u=$id&__a=1" | tr "{" "\n" | grep facebook | sed "s/\"t\":\"\(.*\)\",\"i\":.*\"u\":\"\(.*\)\",\"n\":\".*/\1 # \2/g" That should get the name and URL without resorting to a temporary file if used correctly. (echo $line | awk -F # '{print $1}' (or $2) or you could use cut -d # -f1 (or 2))
dabom · 660 weeks and 5 days ago
matt's script hangs after a few contacts. maybe there is a regular expression which fuck's up something. as my cpu goes to 100% with sed and doesn't do any thing else. dacom any way to get "name,email"? cheers
asterlux · 660 weeks and 5 days ago
Try this, it will output Name,Email curl -m 5 --retry 1 -b /tmp/cookies -Ls -A "Opera" "http://www.facebook.com/ajax/typeahead_friends.php?u=$id&__a=1" | tr "{" "\n" | grep facebook | sed "s/\"t\":\"\(.*\)\",\"i\":.*\"u\":\"\(.*\)\",\"n\":\".*/\1,\2/g" | tr -d "\\\\" | while read line; do echo $(echo $line | cut -d , -f1)','$(curl -m 5 --retry 1 -b /tmp/cookies -A "Opera" -Ls $(echo $line | cut -d , -f2 | sed "s/www/m/g" | awk '{print $0 "?v=info&refid=17"}' | sed "/profile.php/s/?v=info/\&v=info/g") | tr \< "\n" | tr \> "\n" | grep "@.*\." | grep -v " "); done
matthewbauer · 660 weeks and 5 days ago
cheers mat! it works as charm ;) it outputs name, and all emails of your facebook friends :D cool
asterlux · 660 weeks and 4 days ago
Hey, Thanks for this, but I'm only getting email addresses for 1 in 20 of the names. I'm using Matthew's last post. However I hack it I can't seem to get it to work. Would love to get birthdays as well.
chawsa · 659 weeks and 4 days ago
Yeah, facebook broke the script... I was ripping the emails off of mobile facebook and facebook just implemented a way of stopping this. After so many requests in a certain amount of time (or at least that's how I think it works but I'm not really sure) the emails switch back to pictures for 24 hours making the script basically useless... I don't really care enough to remake this script but if anyone does fix it please post.
dabom · 659 weeks and 3 days ago
Unfortunately, it does not work anymore. The http://www.facebook.com/ajax/typeahead_friends.php?u=$id&__a=1 outputs every information but not the email. Working an alternative solution, i'd post it here when it is ready
samhagin · 572 weeks and 6 days ago
Email scrapping on Facebook is very popular among data driven websites and email marketing. Wit scrapping technique we can scrap data from social media websites like FB, Snapchat, Instagram etc. Apart from this we can also scrap data from ecommerce sites.
ashleyridley007 · 7 weeks and 1 day ago
Pug Puppies for Sale Near Me pugs puppies for sale teacup pugs for sale pug puppies for sale by owner pug puppies ohio PUG PUPPY FOR SALE NEAR ME PUG PUPPIES FOR SALE pug puppies for sale in kentucky Pug Puppies for Sale Under $500 Near Me pug puppies for sale in texas pug puppies for sale $200 pugs for sale near me under $500 pugs for sale under $400 near me pugs for sale near me puppies for sale near me under $500 pug puppies for sale under $1,000 near me pug for sale pug puppies for sale under $300 Brindle Pug Pitbull Pug Mix Pugs for sale cheap Cheap pug affordable pug puppies for sale near me black pugs for sale near me White Pugs for sale pug dog for sale free pug puppies pug puppies for sale in my area mn pug breeders pug puppies indiana pugs for sale michigan PUG PUPPY ADOPTION Pug puppies for sale Pug puppies for sale near me Pug puppies near me Pug Puppies for Sale Under $500 Near Me Cute Pug Puppies Black pug puppies Black pug puppies for sale pug puppies for adoption black pug puppies for sale near me chihuahua pug mix puppies how much is a pug puppy teacup pug puppies baby pug puppies pictures of pug puppies pug puppies for sale in Ohio pug puppies price pug mix puppies teacup pug puppies for sale best food for pug puppy newborn pug puppies pug puppies craigslist pug puppies for sale craigslist adorable pug puppies how much does a pug puppy cost Pitbull pug mix puppies pug pit mix puppy pug puppies for sale $200 pug puppies for sale in NJ Pug puppies for sale in Wisconsin pug puppy cost pug puppy food royal canin pug puppy royal canin pug puppy food fawn pug puppy pug puppies for sale florida pug puppies for sale in Indiana pug puppies for sale in KY pug puppies for sale in NC pug dog puppy AKC Registered Pug Puppies For sale cheap pug puppies for sale near me cheap pug puppies for sale in California cheap pug puppies for sale in nj Black Pug Puppies for sale pugs puppies for sale pug puppies indiana Amazing! This blog looks just like my old one! It's on a completely different subject but it has pretty much the same layout and design. Wonderful choice of colors!
rahimhh21 · 5 weeks ago

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands



Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: