FAST Search and Replace for Strings in all Files in Directory

sh -c 'S=askapache R=htaccess; find . -mount -type f|xargs -P5 -iFF grep -l -m1 "$S" FF|xargs -P5 -iFF sed -i -e "s%${S}%${R}%g" FF'

I needed a way to search all files in a web directory that contained a certain string, and replace that string with another string. In the example, I am searching for "askapache" and replacing that string with "htaccess". I wanted this to happen as a cron job, and it was important that this happened as fast as possible while at the same time not hogging the CPU since the machine is a server. So this script uses the nice command to run the sh shell with the command, which makes the whole thing run with priority 19, meaning it won't hog CPU processing. And the -P5 option to the xargs command means it will run 5 separate grep and sed processes simultaneously, so this is much much faster than running a single grep or sed. You may want to do -P0 which is unlimited if you aren't worried about too many processes or if you don't have to deal with process killers in the bg. Also, the -m1 command to grep means stop grepping this file for matches after the first match, which also saves time.

Sample Output

Cpu(s): 28.1% us,  9.5% sy,  3.0% ni, 48.6% id,  9.9% wa,  0.2% hi,  0.7% si

  PID NI  %CPU %MEM  COMMAND
20707 19  0.0  0.1   sh -c S=askapache;R=htaccess;find . -type f|xargs -P5 -iFF grep -l -m1 "$S" FF|xargs -P5 -iFF sed -i -e s%${S}%${R}% FF
20708 19  2.0  0.0   find . -type f
20709 19  2.0  0.0   xargs -P5 -iFF grep -l -m1 askapache FF
20710 19  0.5  0.0   xargs -P5 -iFF sed -i -e s%askapache%htaccess% FF
23291 19  0.0  0.0   grep -l -m1 askapache ./servers/util_ldap_cache_mgr.c.html
23293 19  0.0  0.0   grep -l -m1 askapache ./servers/util_md5.c.html
23308 19  0.0  0.0   grep -l -m1 askapache ./servers/waitio.c.html
23310 19  0.0  0.0   grep -l -m1 askapache ./servers/winconfig.h.html
23311 19  0.0  0.0   grep -l -m1 askapache ./servers/wintty.c.html
23265 19  0.0  0.1   sed -i -e s%askapache%htaccess% ./servers/apr_uri.h.html
23266 19  0.0  0.0   sed -i -e s%askapache%htaccess% ./servers/apr_uri.c.html
23267 19  0.0  0.0   sed -i -e s%askapache%htaccess% ./servers/apr_time.h.html
23268 19  0.0  0.0   sed -i -e s%askapache%htaccess% ./servers/apr_user.h.html
23269 19  0.0  0.0   sed -i -e s%askapache%htaccess% ./servers/apr_strmatch.h.html

By: AskApache

2009-10-02 05:03:10

find grep sed sh xargs cpu usage sed xargs grep multiple background nice search and replace sh processes

Submit An Alternative

What Others Think

seofox · 60 weeks and 2 days ago

seofox · 59 weeks and 3 days ago

rahimhh21 · 30 weeks ago

Perfecthomepugs · 21 weeks and 4 days ago

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Similar Commands

Replace strings in text

Replace strings in files

Search and replace text in all php files with ruby

replace strings in file names

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)