Commands tagged xargs (140)

  • If you have the fdupes command, you'll save a lot of typing. It can do recursive searches (-r,-R) and it allows you to interactively select which of the duplicate files found you wish to keep or delete.


    22
    fdupes -r .
    Vilemirth · 2011-02-19 17:02:30 0
  • I needed a way to search all files in a web directory that contained a certain string, and replace that string with another string. In the example, I am searching for "askapache" and replacing that string with "htaccess". I wanted this to happen as a cron job, and it was important that this happened as fast as possible while at the same time not hogging the CPU since the machine is a server. So this script uses the nice command to run the sh shell with the command, which makes the whole thing run with priority 19, meaning it won't hog CPU processing. And the -P5 option to the xargs command means it will run 5 separate grep and sed processes simultaneously, so this is much much faster than running a single grep or sed. You may want to do -P0 which is unlimited if you aren't worried about too many processes or if you don't have to deal with process killers in the bg. Also, the -m1 command to grep means stop grepping this file for matches after the first match, which also saves time. Show Sample Output


    10
    sh -c 'S=askapache R=htaccess; find . -mount -type f|xargs -P5 -iFF grep -l -m1 "$S" FF|xargs -P5 -iFF sed -i -e "s%${S}%${R}%g" FF'
    AskApache · 2009-10-02 05:03:10 0
  • Run this in the directory you store your music in. mp3gain and vorbisgain applies the ReplayGain normalization routine to mp3 and ogg files (respectively) in a reversible way. ReplayGain uses psychoacoustic analysis to make all files sound about the same loudness, so you don't get knocked out of your chair by loud songs after cranking up the volume on quieter ones.


    9
    find . -iname \*.mp3 -print0 | xargs -0 mp3gain -krd 6 && vorbisgain -rfs .
    Viaken · 2009-03-09 18:11:35 2
  • Search for files and list the 20 largest. find . -type f gives us a list of file, recursively, starting from here (.) -print0 | xargs -0 du -h separate the names of files with NULL characters, so we're not confused by spaces then xargs run the du command to find their size (in human-readable form -- 64M not 64123456) | sort -hr use sort to arrange the list in size order. sort -h knows that 1M is bigger than 9K | head -20 finally only select the top twenty out of the list Show Sample Output


    9
    find . -type f -print0 | xargs -0 du -h | sort -hr | head -20
    flatcap · 2012-03-30 10:21:12 3
  • By putting the "-not \( -name .svn -prune \)" in the very front of the "find" command, you eliminate the .svn directories in your find command itself. No need to grep them out. You can even create an alias for this command: alias svn_find="find . -not \( -name .svn -prune \)" Now you can do things like svn_find -mtime -3


    8
    find . -not \( -name .svn -prune \) -type f -print0 | xargs --null grep <searchTerm>
    qazwart · 2009-07-08 20:08:05 4
  • Convert all jpegs in the current directory into ~1024*768 pixels and ~ 150 KBytes jpegs


    8
    for file in *.jpg; do convert "$file" -resize 800000@ -quality 80 "small.$file"; done
    grinob · 2010-09-13 19:06:14 0
  • will purge: only installed apps: /^ii/!d avoiding current kernel stuff: /'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d using app names: s/^[^ ]* [^ ]* \([^ ]*\).*/\1/ avoiding stuff without a version number: /[0-9]/!d


    7
    dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge
    plasticdoc · 2009-06-19 10:11:00 0
  • I modify 4077 and marssi commandline to simplify it and skip an error when parsing the first line of lsmod (4077). Also, it's more concise and small now. I skip using xargs ( not required here ). This is only for GNU sed. For thoses without GNU sed, use that : modinfo $(lsmod | awk 'NR>1 {print $1}') | sed -e '/^dep/s/$/\n/g' -e '/^file/b' -e '/^desc/b' -e '/^dep/b' -e d


    6
    modinfo $(cut -d' ' -f1 /proc/modules) | sed '/^dep/s/$/\n/; /^file\|^desc\|^dep/!d'
    sputnick · 2009-11-18 23:40:46 1
  • This revision to my command (command #8851) was called for when it failed to find the parent package of 'rlogin', which is really a deep symbolic link to /usr/bin/ssh. This revision fixes this newfound issue, while ensuring fixes of other older issues work too. Show Sample Output


    6
    function whichpkg() { readlink -f "$(which $1)" | xargs --no-run-if-empty dpkg -S; }
    b_t · 2011-10-28 02:53:19 0
  • Parallel does not suffer from the risk of mixing of output that xargs suffers from. -j+0 will run as many jobs in parallel as you have cores. With parallel you only need -0 (and -print0) if your filenames contain a '\n'. Parallel is from https://savannah.nongnu.org/projects/parallel/


    5
    find . -type f | parallel -j+0 grep -i foobar
    unixmonkey8046 · 2010-01-30 02:08:46 2

  • 5
    svn st | grep -e '^M' | awk '{print $2}' | xargs svn revert
    ethanmiller · 2010-08-11 14:24:05 0
  • List out all the names from the zip file and pass it to xargs utility to delete each one of them


    5
    unzip -Z -1 <filename.zip> | xargs -I{} rm -v {}
    praton · 2018-08-19 20:49:25 0
  • xargs -P N spawns up to N worker processes. -n 40 means each grep command gets up to 40 file names each on the command line.


    4
    find . -type f -print0 | xargs -0 -P 4 -n 40 grep -i foobar
    ketil · 2009-08-05 23:18:44 4
  • With this form you dont need to cut out target directory using grep/sed/etc.


    4
    (ls; mkdir subdir; echo subdir) | xargs mv
    mechmind · 2009-11-08 11:40:55 5

  • 4
    awk '{print $1}' "/proc/modules" | xargs modinfo | awk '/^(filename|desc|depends)/'
    unixmonkey7109 · 2009-11-20 13:06:25 0
  • Use if you have pictures all over the place and you want to copy them to a central location Synopsis: Find jpg files translate all file names to lowercase backup existing, don't overwrite, preserve mode ownership and timestamps copy to a central location


    4
    find . -iname "*.jpg" -print0 | tr '[A-Z]' '[a-z]' | xargs -0 cp --backup=numbered -dp -u --target-directory {location} &
    oracular · 2009-12-10 08:47:04 0
  • xargs deals badly with special characters (such as space, ' and "). To see the problem try this: touch important_file touch 'not important_file' ls not* | xargs rm Parallel https://savannah.nongnu.org/projects/parallel/ does not have this problem.


    4
    grep -rl oldstring . | parallel sed -i -e 's/oldstring/newstring/'
    unixmonkey8046 · 2010-01-28 08:44:16 0
  • Using `-exec cmd {} +` causes find to build the command using all matching filenames before execution, rather than once per file.


    4
    find . -type f -exec chmod a-x {} +
    sanmiguel · 2012-06-11 12:50:56 1
  • Convert some SVG files into PNG using ImageMagick's convert command. Run the conversions in parallel to save time. This is safer than robinro's forkbomb approach :-) xargs runs four processes at a time -P4


    4
    find . -name \*.svg -print0 | xargs -0 -n1 -P4 -I{} bash -c 'X={}; convert "$X" "${X%.svg}.png"'
    flatcap · 2014-04-11 14:30:30 0
  • If you're like me and want to keep all your music rated, and you use xmms2, you might like this command. I takes 10 random songs from your xmms2 library that don't have any rating, and adds them to your current playlist. You can then rate them in another xmms2 client that supports rating (I like kuechenstation). I'm pretty sure there's a better way to do the grep ... | sed ... part, probably with awk, but I don't know awk, so I'd welcome any suggestions. Show Sample Output


    3
    xmms2 mlib search NOT +rating | grep -r '^[0-9]' | sed -r 's/^([0-9]+).*/\1/' | sort -R | head | xargs -L 1 xmms2 addid
    goodevilgenius · 2009-04-16 20:27:30 4

  • 3
    find . -name "*.[ch]" | xargs grep "TODO"
    freestyler · 2009-08-12 16:53:01 2
  • This pipeline will find, sort and display all files based on mtime. This could be done with find | xargs, but the find | xargs pipeline will not produce correct results if the results of find are greater than xargs command line buffer. If the xargs buffer fills, xargs processes the find results in more than one batch which is not compatible with sorting. Note the "-print0" on find and "-0" switch for perl. This is the equivalent of using xargs. Don't you love perl? Note that this pipeline can be easily modified to any data produced by perl's stat operator. eg, you could sort on size, hard links, creation time, etc. Look at stat and just change the '9' to what you want. Changing the '9' to a '7' for example will sort by file size. A '3' sorts by number of links.... Use head and tail at the end of the pipeline to get oldest files or most recent. Use awk or perl -wnla for further processing. Since there is a tab between the two fields, it is very easy to process. Show Sample Output


    3
    find $HOME -type f -print0 | perl -0 -wn -e '@f=<>; foreach $file (@f){ (@el)=(stat($file)); push @el, $file; push @files,[ @el ];} @o=sort{$a->[9]<=>$b->[9]} @files; for $i (0..$#o){print scalar localtime($o[$i][9]), "\t$o[$i][-1]\n";}'|tail
    drewk · 2009-09-21 22:11:16 4

  • 3
    find . -name "*.txt" -exec sed -i "s/old/new/" {} \;
    tyboon · 2009-09-24 20:30:00 2
  • Do the same as pssh, just in shell syntax. Put your hosts in hostlist, one per line. Command outputs are gathered in output and error directories.


    3
    xargs -n1 -P100 -I{} sh -c 'ssh {} uptime >output/{} 2>error/{}' <hostlist
    dooblem · 2010-08-20 11:03:11 0
  • Preserve file structure when coping and exclude some file o dir patterns Show Sample Output


    3
    find ./ ! -name 'excludepattern' | xargs -i cp --parents {} destdir
    starchox · 2010-09-27 21:36:50 2
  •  1 2 3 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Which processes are listening on a specific port (e.g. port 80)
swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

Convert CSV to JSON
Replace 'csv_file.csv' with your filename.

Count lines of code across multiple file types, sorted by least amount of code to greatest
Gives you a nice quick summary of how many lines each of your files is comprised of. (In this example, we just check .c, .h, .php and .pl). Since we just use wc -l to count, you'll just get a very rough estimate of how many lines of actual code there are. Use a more sophisticated algorithm instead if you need to.

LDAP search to query an ActiveDirectory server
These are the parameters to ldapsearch (from ldap-utils in Ubuntu), for searching for the record for Joe Blogg's user. sAMAccountName is the LDAP field that ActiveDirectory uses to store the user name. 'DOMAIN\Joe.Bloggs' where "DOMAIN" is the the active directory domain. Othewise you could use "CN=Joe.Bloggs,DC=example,DC=com" instead of "DOMAIN\Joe.Bloggs"

Copy all files, including hidden files, recursively without traversing backward
You could do the following, however, brace expansion with {} is not defined in POSIX, and therefore not guaranteed to work in all shells. But, if it does, it's more convenient (although it's certainly not less typing): $ cp -r {*,.??*} /dest Sometimes there are times when I need to cp(1), mv(1) or rm(1) files recursively, but don't want to traverse the previous directory by following ../../../../ etc out of the current directory. This command prevents that. The secret sauce is ".??*". The file globbing ensures that it must start with a dot, and be followed by at least two characters. So, three characters must exist in the filename, which eliminates "." and "..".

Convert CSV to JSON
Replace 'csv_file.csv' with your filename.

Convert seconds to [DD:][HH:]MM:SS
Converts any number of seconds into days, hours, minutes and seconds. sec2dhms() { declare -i SS="$1" D=$(( SS / 86400 )) H=$(( SS % 86400 / 3600 )) M=$(( SS % 3600 / 60 )) S=$(( SS % 60 )) [ "$D" -gt 0 ] && echo -n "${D}:" [ "$H" -gt 0 ] && printf "%02g:" "$H" printf "%02g:%02g\n" "$M" "$S" }

Create md5sum of a directory

list only directories in reverse order
list only directories in reverse order

extract element of xml
poor man's xml parser :)


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: