Commands tagged xargs (148)

  • If you have the fdupes command, you'll save a lot of typing. It can do recursive searches (-r,-R) and it allows you to interactively select which of the duplicate files found you wish to keep or delete.


    23
    fdupes -r .
    Vilemirth · 2011-02-19 17:02:30 9
  • Search for files and list the 20 largest. find . -type f gives us a list of file, recursively, starting from here (.) -print0 | xargs -0 du -h separate the names of files with NULL characters, so we're not confused by spaces then xargs run the du command to find their size (in human-readable form -- 64M not 64123456) | sort -hr use sort to arrange the list in size order. sort -h knows that 1M is bigger than 9K | head -20 finally only select the top twenty out of the list Show Sample Output


    11
    find . -type f -print0 | xargs -0 du -h | sort -hr | head -20
    flatcap · 2012-03-30 10:21:12 9
  • I needed a way to search all files in a web directory that contained a certain string, and replace that string with another string. In the example, I am searching for "askapache" and replacing that string with "htaccess". I wanted this to happen as a cron job, and it was important that this happened as fast as possible while at the same time not hogging the CPU since the machine is a server. So this script uses the nice command to run the sh shell with the command, which makes the whole thing run with priority 19, meaning it won't hog CPU processing. And the -P5 option to the xargs command means it will run 5 separate grep and sed processes simultaneously, so this is much much faster than running a single grep or sed. You may want to do -P0 which is unlimited if you aren't worried about too many processes or if you don't have to deal with process killers in the bg. Also, the -m1 command to grep means stop grepping this file for matches after the first match, which also saves time. Show Sample Output


    10
    sh -c 'S=askapache R=htaccess; find . -mount -type f|xargs -P5 -iFF grep -l -m1 "$S" FF|xargs -P5 -iFF sed -i -e "s%${S}%${R}%g" FF'
    AskApache · 2009-10-02 05:03:10 7
  • Run this in the directory you store your music in. mp3gain and vorbisgain applies the ReplayGain normalization routine to mp3 and ogg files (respectively) in a reversible way. ReplayGain uses psychoacoustic analysis to make all files sound about the same loudness, so you don't get knocked out of your chair by loud songs after cranking up the volume on quieter ones.


    9
    find . -iname \*.mp3 -print0 | xargs -0 mp3gain -krd 6 && vorbisgain -rfs .
    Viaken · 2009-03-09 18:11:35 12
  • By putting the "-not \( -name .svn -prune \)" in the very front of the "find" command, you eliminate the .svn directories in your find command itself. No need to grep them out. You can even create an alias for this command: alias svn_find="find . -not \( -name .svn -prune \)" Now you can do things like svn_find -mtime -3


    8
    find . -not \( -name .svn -prune \) -type f -print0 | xargs --null grep <searchTerm>
    qazwart · 2009-07-08 20:08:05 10
  • Convert all jpegs in the current directory into ~1024*768 pixels and ~ 150 KBytes jpegs


    8
    for file in *.jpg; do convert "$file" -resize 800000@ -quality 80 "small.$file"; done
    grinob · 2010-09-13 19:06:14 4
  • will purge: only installed apps: /^ii/!d avoiding current kernel stuff: /'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d using app names: s/^[^ ]* [^ ]* \([^ ]*\).*/\1/ avoiding stuff without a version number: /[0-9]/!d


    7
    dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge
    plasticdoc · 2009-06-19 10:11:00 9
  • This revision to my command (command #8851) was called for when it failed to find the parent package of 'rlogin', which is really a deep symbolic link to /usr/bin/ssh. This revision fixes this newfound issue, while ensuring fixes of other older issues work too. Show Sample Output


    7
    function whichpkg() { readlink -f "$(which $1)" | xargs --no-run-if-empty dpkg -S; }
    b_t · 2011-10-28 02:53:19 4
  • I modify 4077 and marssi commandline to simplify it and skip an error when parsing the first line of lsmod (4077). Also, it's more concise and small now. I skip using xargs ( not required here ). This is only for GNU sed. For thoses without GNU sed, use that : modinfo $(lsmod | awk 'NR>1 {print $1}') | sed -e '/^dep/s/$/\n/g' -e '/^file/b' -e '/^desc/b' -e '/^dep/b' -e d


    6
    modinfo $(cut -d' ' -f1 /proc/modules) | sed '/^dep/s/$/\n/; /^file\|^desc\|^dep/!d'
    sputnick · 2009-11-18 23:40:46 4
  • Parallel does not suffer from the risk of mixing of output that xargs suffers from. -j+0 will run as many jobs in parallel as you have cores. With parallel you only need -0 (and -print0) if your filenames contain a '\n'. Parallel is from https://savannah.nongnu.org/projects/parallel/


    5
    find . -type f | parallel -j+0 grep -i foobar
    unixmonkey8046 · 2010-01-30 02:08:46 17

  • 5
    svn st | grep -e '^M' | awk '{print $2}' | xargs svn revert
    ethanmiller · 2010-08-11 14:24:05 3
  • List out all the names from the zip file and pass it to xargs utility to delete each one of them


    5
    unzip -Z -1 <filename.zip> | xargs -I{} rm -v {}
    praton · 2018-08-19 20:49:25 321
  • xargs -P N spawns up to N worker processes. -n 40 means each grep command gets up to 40 file names each on the command line.


    4
    find . -type f -print0 | xargs -0 -P 4 -n 40 grep -i foobar
    ketil · 2009-08-05 23:18:44 8

  • 4
    find . -name "*.[ch]" | xargs grep "TODO"
    freestyler · 2009-08-12 16:53:01 5
  • With this form you dont need to cut out target directory using grep/sed/etc.


    4
    (ls; mkdir subdir; echo subdir) | xargs mv
    mechmind · 2009-11-08 11:40:55 13

  • 4
    awk '{print $1}' "/proc/modules" | xargs modinfo | awk '/^(filename|desc|depends)/'
    unixmonkey7109 · 2009-11-20 13:06:25 3
  • Use if you have pictures all over the place and you want to copy them to a central location Synopsis: Find jpg files translate all file names to lowercase backup existing, don't overwrite, preserve mode ownership and timestamps copy to a central location


    4
    find . -iname "*.jpg" -print0 | tr '[A-Z]' '[a-z]' | xargs -0 cp --backup=numbered -dp -u --target-directory {location} &
    oracular · 2009-12-10 08:47:04 5
  • xargs deals badly with special characters (such as space, ' and "). To see the problem try this: touch important_file touch 'not important_file' ls not* | xargs rm Parallel https://savannah.nongnu.org/projects/parallel/ does not have this problem.


    4
    grep -rl oldstring . | parallel sed -i -e 's/oldstring/newstring/'
    unixmonkey8046 · 2010-01-28 08:44:16 6
  • Using `-exec cmd {} +` causes find to build the command using all matching filenames before execution, rather than once per file.


    4
    find . -type f -exec chmod a-x {} +
    sanmiguel · 2012-06-11 12:50:56 8
  • Convert some SVG files into PNG using ImageMagick's convert command. Run the conversions in parallel to save time. This is safer than robinro's forkbomb approach :-) xargs runs four processes at a time -P4


    4
    find . -name \*.svg -print0 | xargs -0 -n1 -P4 -I{} bash -c 'X={}; convert "$X" "${X%.svg}.png"'
    flatcap · 2014-04-11 14:30:30 12
  • If you're like me and want to keep all your music rated, and you use xmms2, you might like this command. I takes 10 random songs from your xmms2 library that don't have any rating, and adds them to your current playlist. You can then rate them in another xmms2 client that supports rating (I like kuechenstation). I'm pretty sure there's a better way to do the grep ... | sed ... part, probably with awk, but I don't know awk, so I'd welcome any suggestions. Show Sample Output


    3
    xmms2 mlib search NOT +rating | grep -r '^[0-9]' | sed -r 's/^([0-9]+).*/\1/' | sort -R | head | xargs -L 1 xmms2 addid
    goodevilgenius · 2009-04-16 20:27:30 6
  • This pipeline will find, sort and display all files based on mtime. This could be done with find | xargs, but the find | xargs pipeline will not produce correct results if the results of find are greater than xargs command line buffer. If the xargs buffer fills, xargs processes the find results in more than one batch which is not compatible with sorting. Note the "-print0" on find and "-0" switch for perl. This is the equivalent of using xargs. Don't you love perl? Note that this pipeline can be easily modified to any data produced by perl's stat operator. eg, you could sort on size, hard links, creation time, etc. Look at stat and just change the '9' to what you want. Changing the '9' to a '7' for example will sort by file size. A '3' sorts by number of links.... Use head and tail at the end of the pipeline to get oldest files or most recent. Use awk or perl -wnla for further processing. Since there is a tab between the two fields, it is very easy to process. Show Sample Output


    3
    find $HOME -type f -print0 | perl -0 -wn -e '@f=<>; foreach $file (@f){ (@el)=(stat($file)); push @el, $file; push @files,[ @el ];} @o=sort{$a->[9]<=>$b->[9]} @files; for $i (0..$#o){print scalar localtime($o[$i][9]), "\t$o[$i][-1]\n";}'|tail
    drewk · 2009-09-21 22:11:16 11

  • 3
    find . -name "*.txt" -exec sed -i "s/old/new/" {} \;
    tyboon · 2009-09-24 20:30:00 15
  • Do the same as pssh, just in shell syntax. Put your hosts in hostlist, one per line. Command outputs are gathered in output and error directories.


    3
    xargs -n1 -P100 -I{} sh -c 'ssh {} uptime >output/{} 2>error/{}' <hostlist
    dooblem · 2010-08-20 11:03:11 3
  • Preserve file structure when coping and exclude some file o dir patterns Show Sample Output


    3
    find ./ ! -name 'excludepattern' | xargs -i cp --parents {} destdir
    starchox · 2010-09-27 21:36:50 5
  •  1 2 3 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Save your terminal commands in bash history in real time
Use this command if you want your terminal commands be saved in your history file in real time instead of waiting until the terminal is closed

Reset terminal that has been buggered by binary input or similar

Check for Firewall Blockage.
This is just one method of checking to see if an IP is blocked via IP tables or CSF. Simple and to the point. Replace xx.xx.xx.xx with the IP you wish to check.

List all authors of a particular git project
This should work even if the output format changes.

list all file extensions in a directory
Just a little simplification.

Find status of all symlinks
The symlinks command can show status of all symbolic links, including which links are dangling, which symlinks point to files on other file systems, which symlinks use ../ more than necessary, which symlinks are messy (e.g. having too many slashes or dots), etc. Other useful things it can do include removing all dangling links (-d) and converting absolute links to relative links (-c). The path given must be an absolute path (which is why I used $(pwd) in the example command).

Get the Volume labels all bitlocker volumes had before being encrypted
Get information of volume labels of bitlocker volumes, even if they are encrypted and locked (no access to filesystem, no password provided). Note that the volume labels can have spaces, but only if you name then before encryption. Renaming a bitlocker partition after being encrypted does not have the same effect as doing it before.

Randomize lines in a file
Works in sort (GNU coreutils) 7.4, don't know when it was implemented but sometime the last 6 years.

show all key and mouse events
for mousevents, move the mouse over the window and click/move etc. usefull for getting mouseKeys, or keyKeys. also usefull for checking if X gets those mouse-events.

Check a directory of PNG files for errors
Useful for checking if a large number of PNG files was downloaded successfully by verifying the built-in CRC checksum. For incomplete files, the command will print: "00002309.png EOF while reading IDAT data ERROR: 00002309.png" The process is very fast; checking 21,000 files of 5MB in size took only five minutes on a 2011 Intel mobile dual-core.


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: