Checks apache's access_log file, strips the search queries and shoves them up your e-mail

awk '/q=/{print $11}' /var/log/httpd/access_log.4 | awk -F 'q=' '{print $2}' | sed 's/+/ /g;s/%22/"/g;s/q=//' | cut -d "&" -f 1
as unixmonkey7109 pointed out, first awk parse replaces three steps.

1
By: isma
2009-11-22 17:13:48

1 Alternatives + Submit Alt

What Others Think

It is possible to use one instance awk to pull the search strings and a little bit of python to unescape them so that it catches more than just %22. awk '/q=/{FS="q=";split($2,a,"&");b=a[1];gsub(/".*$|\+/," ",b);if(b!="-")print b}' /var/log/httpd/access_log|python -c 'import sys,urllib2;print urllib2.unquote(sys.stdin.read().strip())'
eightmillion · 688 weeks and 2 days ago
in that case, why don't do everything in Python..?
unixmonkey7109 · 688 weeks and 1 day ago
Because it's longer and uglier and more error prone: python -c 'import urllib,re;print "\n".join(map(lambda x:urllib.unquote(re.findall(r"q=.*?[&\"]",x.split()[10])[0][2:-1].replace("+"," ")),filter(lambda x:"q=" in x,open("/var/log/httpd/access_log").readlines())))'
eightmillion · 688 weeks and 1 day ago
no its not. Do it with a Python script, not as one liner like that. Otherwise, use (g)awk but no need to create extra pipes to cut+grep+sed.
unixmonkey7109 · 688 weeks and 1 day ago
here's a sample matching line from access_log 190.132.248.102 - - [23/Nov/2009:10:00:22 -0200] "GET /2009/10/14/ceibal-reflexiones/ HTTP/1.1" 200 30418 " the output is ceibal reflexiones
isma · 688 weeks and 1 day ago
oh.. url filter .. lol pastebin.com/fa34b7a1
isma · 688 weeks and 1 day ago
@isma, ok, but i don't see any "q=" in that sample. please show an actual sample where there is "q=" in the line.
unixmonkey7109 · 688 weeks and 1 day ago
@unixmonkey7109 uh... sorry, I think my comments were unclear. The first one was filtered by commandlinefu, so it didn't show up completely. The sample line (which contains a q= as you will see) is in pastebin.com/fa34b7a1 . I'm sorry this is turning so confusing. It's actually a good idea for monitoring your SEO tasks, and thanks to you, the command was nicely reduced (and, believe me, I learned a lot in that process). So thanks again and maybe you can use it on your own access_log. PLEASE NOTICE (as I just realized): your httpd.conf should have the next line: CustomLog "/var/log/httpd/access_log" combined like that, suffixed by the word "combined" instead of "common" so it would bring the referer.
isma · 688 weeks ago
I think this is an informative post and it is very useful and knowledgeable. therefore, I would like to thank you for the efforts you have made in writing this article. rv locks
KayGasaway · 27 weeks and 2 days ago
Great write-up, I'm a huge believer in commenting on blogs to share with the blog writers know that they've added something worthwhile to the world wide web!.. http://www.khannadiamond.com">jewellery store in patiala
KayGasaway · 23 weeks and 2 days ago
One Tree Hill should be the best TV series for me, the characters are great., check it out
HAMZA · 23 weeks and 1 day ago
Pug Puppies for Sale Near Me pugs puppies for sale teacup pugs for sale pug puppies for sale by owner pug puppies ohio PUG PUPPY FOR SALE NEAR ME PUG PUPPIES FOR SALE pug puppies for sale in kentucky Pug Puppies for Sale Under $500 Near Me pug puppies for sale in texas pug puppies for sale $200 pugs for sale near me under $500 pugs for sale under $400 near me pugs for sale near me puppies for sale near me under $500 pug puppies for sale under $1,000 near me pug for sale pug puppies for sale under $300 Brindle Pug Pitbull Pug Mix Pugs for sale cheap Cheap pug affordable pug puppies for sale near me black pugs for sale near me White Pugs for sale pug dog for sale free pug puppies pug puppies for sale in my area mn pug breeders pug puppies indiana pugs for sale michigan PUG PUPPY ADOPTION Pug puppies for sale Pug puppies for sale near me Pug puppies near me Pug Puppies for Sale Under $500 Near Me Cute Pug Puppies Black pug puppies Black pug puppies for sale pug puppies for adoption black pug puppies for sale near me chihuahua pug mix puppies how much is a pug puppy teacup pug puppies baby pug puppies pictures of pug puppies pug puppies for sale in Ohio pug puppies price pug mix puppies teacup pug puppies for sale best food for pug puppy newborn pug puppies pug puppies craigslist pug puppies for sale craigslist adorable pug puppies how much does a pug puppy cost Pitbull pug mix puppies pug pit mix puppy pug puppies for sale $200 pug puppies for sale in NJ Pug puppies for sale in Wisconsin pug puppy cost pug puppy food royal canin pug puppy royal canin pug puppy food fawn pug puppy pug puppies for sale florida pug puppies for sale in Indiana pug puppies for sale in KY pug puppies for sale in NC pug dog puppy AKC Registered Pug Puppies For sale cheap pug puppies for sale near me cheap pug puppies for sale in California cheap pug puppies for sale in nj Black Pug Puppies for sale pugs puppies for sale pug puppies indiana Amazing! This blog looks just like my old one! It's on a completely different subject but it has pretty much the same layout and design. Wonderful choice of colors!
rahimhh21 · 21 weeks and 4 days ago
Since 2008, Perfect House Pug Puppies has been the leading Pug puppy breeder in the US, providing premium Pug puppies for sale. Each puppy exhibits the professionalism and knowledge that have set Perfect House Pug puppies apart from other pug puppy breeders. The professionals at Perfect House Pug Puppies are dedicated about finding their puppies loving, beloved homes where they can enjoy long, healthy lives. A companion that has been produced for quality, health, and temperament that satisfies the breed standard is guaranteed by Perfect House Pug puppies, the pug breeder that has set the benchmark for exceptional customer satisfaction. If you are looking to buy a baby pug, Black and white pugs, Mini pug puppies for sale, Black and tan pug for sale, Teacup pug puppies for sale, Female pug puppies, pug puppies for sale, pug puppies for sale near me, pug puppies for sale under $500 near me. Pug Puppies for Sale Near Me pugs puppies for sale teacup pugs for sale pug puppies for sale by owner pug puppies ohio PUG PUPPY FOR SALE NEAR ME PUG PUPPIES FOR SALE pug puppies for sale in kentucky Pug Puppies for Sale Under $500 Near Me pug puppies for sale in texas pug puppies for sale $200 pugs for sale near me under $500 pugs for sale under $400 near me pugs for sale near me puppies for sale near me under $500 pug puppies for sale under $1,000 near me pug for sale pug puppies for sale under $300 Brindle Pug Pitbull Pug Mix Pugs for sale cheap Cheap pug affordable pug puppies for sale near me black pugs for sale near me White Pugs for sale pug dog for sale free pug puppies pug puppies for sale in my area mn pug breeders pug puppies indiana pugs for sale michigan PUG PUPPY ADOPTION Pug puppies for sale Pug puppies for sale near me Pug puppies near me Pug Puppies for Sale Under $500 Near Me Cute Pug Puppies Black pug puppies Black pug puppies for sale pug puppies for adoption black pug puppies for sale near me chihuahua pug mix puppies how much is a pug puppy teacup pug puppies baby pug puppies baby pug puppies for sale pictures of pug puppies pug puppies for sale in Ohio pug puppies price pug mix puppies teacup pug puppies for sale best food for pug puppy newborn pug puppies pug puppies craigslist pug puppies for sale craigslist adorable pug puppies how much does a pug puppy cost Pitbull pug mix puppies pug pit mix puppy pug puppies for sale $200 pug puppies for sale in NJ Pug puppies for sale in Wisconsin pug puppy cost pug puppy food royal canin pug puppy royal canin pug puppy food fawn pug puppy pug puppies for sale florida pug puppies for sale in Indiana pug puppies for sale in KY pug puppies for sale in NC pug dog puppy AKC Registered Pug Puppies For sale cheap pug puppies for sale near me cheap pug puppies for sale in California cheap pug puppies for sale in nj Black Pug Puppies for sale pugs puppies for sale
Perfecthomepugs · 13 weeks ago
Some truly nice stuff on this site, I enjoy it. old cars
mahmishal · 12 weeks and 6 days ago
Thank you for each of your effort on this web site. My niece takes pleasure in managing internet research and it’s really easy to see why. My partner and i notice all relating to the dynamic form you make very useful thoughts on your website and as well as boost participation from some others on the point then our favorite simple princess is truly starting to learn a lot of things. Have fun with the rest of the year. You’re the one carrying out a splendid job. RTP slot
mahmishal · 12 weeks and 2 days ago
A lot of thanks for all your hard work on this website. Kim delights in setting aside time for investigations and it’s really easy to see why. We all learn all regarding the dynamic tactic you make sensible suggestions by means of the website and in addition foster participation from people on that topic then our own princess is actually discovering so much. Take pleasure in the remaining portion of the new year. You have been carrying out a wonderful job. mega game
HAMZA · 11 weeks and 6 days ago
This is really attractive, You’re a really experienced writer. I’ve enrolled with your feed plus expect enjoying your astounding write-ups. Moreover, I’ve got shared your blog inside our myspace. mp3 songs download
HAMZA · 10 weeks and 6 days ago
At eMoney247, we help international businesses with multi-currency digital banking services like a GBP Account, EUR IBAN, USD Account, Forex, FX Services. Digital Banking Services
goxah55535 · 10 weeks and 3 days ago
There is noticeably big money to comprehend this. I assume you’ve made specific nice points in features also. http://www.acmemoviestore.com/gambling/rival-live-slot-machines-and-the-story-behind-the-no-store-award.html">slot online
mahmishal · 10 weeks ago
Sci-Fi is the best, i love sci-fi movies, books and stuffs like that. I am a man of science that is why i love it; slot RTP
HAMZA · 9 weeks and 6 days ago
This internet site is often a walk-through for all of the internet you wanted about this and didn’t know who to question. Glimpse here, and you’ll definitely discover it. acim
mahmishal · 9 weeks and 3 days ago
Your blog has chock-a-block of useful information. I liked your blog's content as well as its look. In my opinion, this is a perfect blog in all aspects. raja gacor
mahmishal · 8 weeks and 3 days ago
At eMoney247, we help International Banking Services with multi currency digital banking services like a GBP Account, Euro IBAN Bank Account, USD Account UK, Forex, Best FX Services.
mahmishal · 7 weeks and 5 days ago
Music started playing when I opened this internet site, so irritating! spiritual awakening movies
HAMZA · 7 weeks and 5 days ago
Real superb information can be found on website . these details
mahmishal · 6 weeks and 3 days ago
i signed up on clicksor last month and i was impressed with the amount of money i made in one month` situs slot
HAMZA · 5 weeks and 5 days ago
I love someone's submitting. The great to see you truly describe inside terms with all the heart and soul additionally decision with this important make a difference is frequently successfully looked at. website
HAMZA · 5 weeks ago
Just to follow up on the up-date of this subject on your web page and wish to let you know just how much I prized the time you took to publish this beneficial post. Within the post, you actually spoke regarding how to truly handle this challenge with all comfort. It would be my own pleasure to collect some more thoughts from your web-site and come up to offer other folks what I have benefited from you. Thanks for your usual good effort. buy magic mushrooms
gabis31 · 4 weeks and 6 days ago
Perhaps this is a bit off topic but in any case, I have been surfing about your blog and it looks really neat. impassioned about your writing. I am creating a new site and hard-pressed to make it appear great, and supply excellent articles. I have discovered a lot on your site and I watch forward to additional updates and will be back. Alta en Directorios
goxah55535 · 4 weeks and 6 days ago
bar show flair… [...]z Awesome looking website. I recently built mine and I was looking for some d wk[...]… Link Bong88
lahotax6 · 4 weeks and 3 days ago
What i don’t understood is in fact how you are no longer really much more well-appreciated than you may be right now. You are very intelligent. You realize thus considerably in terms of this matter, produced me in my view consider it from so many numerous angles. Its like women and men aren’t involved until it is one thing to do with Woman gaga! Your personal stuffs outstanding. Always take care of it up! login jokers 4d
mahmishal · 4 weeks and 3 days ago
It is a fantastic post – immense clear and easy to understand. I am also holding out for the sharks too that made me laugh. judi bola
mahmishal · 4 weeks ago
the air compressors that we use at home are the high powered ones, we also use it for cleaning. aswinbet168
HAMZA · 3 weeks and 6 days ago
I have to express some appreciation to you just for rescuing me from this type of problem. Right after searching throughout the search engines and obtaining methods which were not powerful, I was thinking my entire life was gone. Living devoid of the solutions to the difficulties you’ve solved all through this website is a critical case, and the ones that could have adversely affected my entire career if I hadn’t noticed your site. Your personal know-how and kindness in maneuvering all areas was very helpful. I am not sure what I would’ve done if I hadn’t come across such a step like this. I’m able to now relish my future. Thanks for your time so much for your specialized and sensible guide. I will not think twice to propose your web sites to anybody who needs and wants guidelines about this problem. go to my blog
mahmishal · 3 weeks and 5 days ago

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands



Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: