Commands tagged xargs (148)

  • If you have the fdupes command, you'll save a lot of typing. It can do recursive searches (-r,-R) and it allows you to interactively select which of the duplicate files found you wish to keep or delete.


    24
    fdupes -r .
    Vilemirth · 2011-02-19 17:02:30 0
  • Search for files and list the 20 largest. find . -type f gives us a list of file, recursively, starting from here (.) -print0 | xargs -0 du -h separate the names of files with NULL characters, so we're not confused by spaces then xargs run the du command to find their size (in human-readable form -- 64M not 64123456) | sort -hr use sort to arrange the list in size order. sort -h knows that 1M is bigger than 9K | head -20 finally only select the top twenty out of the list Show Sample Output


    11
    find . -type f -print0 | xargs -0 du -h | sort -hr | head -20
    flatcap · 2012-03-30 10:21:12 3
  • I needed a way to search all files in a web directory that contained a certain string, and replace that string with another string. In the example, I am searching for "askapache" and replacing that string with "htaccess". I wanted this to happen as a cron job, and it was important that this happened as fast as possible while at the same time not hogging the CPU since the machine is a server. So this script uses the nice command to run the sh shell with the command, which makes the whole thing run with priority 19, meaning it won't hog CPU processing. And the -P5 option to the xargs command means it will run 5 separate grep and sed processes simultaneously, so this is much much faster than running a single grep or sed. You may want to do -P0 which is unlimited if you aren't worried about too many processes or if you don't have to deal with process killers in the bg. Also, the -m1 command to grep means stop grepping this file for matches after the first match, which also saves time. Show Sample Output


    10
    sh -c 'S=askapache R=htaccess; find . -mount -type f|xargs -P5 -iFF grep -l -m1 "$S" FF|xargs -P5 -iFF sed -i -e "s%${S}%${R}%g" FF'
    AskApache · 2009-10-02 05:03:10 0
  • Run this in the directory you store your music in. mp3gain and vorbisgain applies the ReplayGain normalization routine to mp3 and ogg files (respectively) in a reversible way. ReplayGain uses psychoacoustic analysis to make all files sound about the same loudness, so you don't get knocked out of your chair by loud songs after cranking up the volume on quieter ones.


    9
    find . -iname \*.mp3 -print0 | xargs -0 mp3gain -krd 6 && vorbisgain -rfs .
    Viaken · 2009-03-09 18:11:35 2
  • By putting the "-not \( -name .svn -prune \)" in the very front of the "find" command, you eliminate the .svn directories in your find command itself. No need to grep them out. You can even create an alias for this command: alias svn_find="find . -not \( -name .svn -prune \)" Now you can do things like svn_find -mtime -3


    8
    find . -not \( -name .svn -prune \) -type f -print0 | xargs --null grep <searchTerm>
    qazwart · 2009-07-08 20:08:05 4
  • Convert all jpegs in the current directory into ~1024*768 pixels and ~ 150 KBytes jpegs


    8
    for file in *.jpg; do convert "$file" -resize 800000@ -quality 80 "small.$file"; done
    grinob · 2010-09-13 19:06:14 0
  • will purge: only installed apps: /^ii/!d avoiding current kernel stuff: /'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d using app names: s/^[^ ]* [^ ]* \([^ ]*\).*/\1/ avoiding stuff without a version number: /[0-9]/!d


    7
    dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge
    plasticdoc · 2009-06-19 10:11:00 0
  • This revision to my command (command #8851) was called for when it failed to find the parent package of 'rlogin', which is really a deep symbolic link to /usr/bin/ssh. This revision fixes this newfound issue, while ensuring fixes of other older issues work too. Show Sample Output


    7
    function whichpkg() { readlink -f "$(which $1)" | xargs --no-run-if-empty dpkg -S; }
    b_t · 2011-10-28 02:53:19 0
  • I modify 4077 and marssi commandline to simplify it and skip an error when parsing the first line of lsmod (4077). Also, it's more concise and small now. I skip using xargs ( not required here ). This is only for GNU sed. For thoses without GNU sed, use that : modinfo $(lsmod | awk 'NR>1 {print $1}') | sed -e '/^dep/s/$/\n/g' -e '/^file/b' -e '/^desc/b' -e '/^dep/b' -e d


    6
    modinfo $(cut -d' ' -f1 /proc/modules) | sed '/^dep/s/$/\n/; /^file\|^desc\|^dep/!d'
    sputnick · 2009-11-18 23:40:46 1
  • Parallel does not suffer from the risk of mixing of output that xargs suffers from. -j+0 will run as many jobs in parallel as you have cores. With parallel you only need -0 (and -print0) if your filenames contain a '\n'. Parallel is from https://savannah.nongnu.org/projects/parallel/


    5
    find . -type f | parallel -j+0 grep -i foobar
    unixmonkey8046 · 2010-01-30 02:08:46 2

  • 5
    svn st | grep -e '^M' | awk '{print $2}' | xargs svn revert
    ethanmiller · 2010-08-11 14:24:05 0
  • List out all the names from the zip file and pass it to xargs utility to delete each one of them


    5
    unzip -Z -1 <filename.zip> | xargs -I{} rm -v {}
    praton · 2018-08-19 20:49:25 0
  • xargs -P N spawns up to N worker processes. -n 40 means each grep command gets up to 40 file names each on the command line.


    4
    find . -type f -print0 | xargs -0 -P 4 -n 40 grep -i foobar
    ketil · 2009-08-05 23:18:44 4

  • 4
    find . -name "*.[ch]" | xargs grep "TODO"
    freestyler · 2009-08-12 16:53:01 1
  • With this form you dont need to cut out target directory using grep/sed/etc.


    4
    (ls; mkdir subdir; echo subdir) | xargs mv
    mechmind · 2009-11-08 11:40:55 5

  • 4
    awk '{print $1}' "/proc/modules" | xargs modinfo | awk '/^(filename|desc|depends)/'
    unixmonkey7109 · 2009-11-20 13:06:25 0
  • Use if you have pictures all over the place and you want to copy them to a central location Synopsis: Find jpg files translate all file names to lowercase backup existing, don't overwrite, preserve mode ownership and timestamps copy to a central location


    4
    find . -iname "*.jpg" -print0 | tr '[A-Z]' '[a-z]' | xargs -0 cp --backup=numbered -dp -u --target-directory {location} &
    oracular · 2009-12-10 08:47:04 0
  • xargs deals badly with special characters (such as space, ' and "). To see the problem try this: touch important_file touch 'not important_file' ls not* | xargs rm Parallel https://savannah.nongnu.org/projects/parallel/ does not have this problem.


    4
    grep -rl oldstring . | parallel sed -i -e 's/oldstring/newstring/'
    unixmonkey8046 · 2010-01-28 08:44:16 0
  • Using `-exec cmd {} +` causes find to build the command using all matching filenames before execution, rather than once per file.


    4
    find . -type f -exec chmod a-x {} +
    sanmiguel · 2012-06-11 12:50:56 1
  • Convert some SVG files into PNG using ImageMagick's convert command. Run the conversions in parallel to save time. This is safer than robinro's forkbomb approach :-) xargs runs four processes at a time -P4


    4
    find . -name \*.svg -print0 | xargs -0 -n1 -P4 -I{} bash -c 'X={}; convert "$X" "${X%.svg}.png"'
    flatcap · 2014-04-11 14:30:30 0
  • If you're like me and want to keep all your music rated, and you use xmms2, you might like this command. I takes 10 random songs from your xmms2 library that don't have any rating, and adds them to your current playlist. You can then rate them in another xmms2 client that supports rating (I like kuechenstation). I'm pretty sure there's a better way to do the grep ... | sed ... part, probably with awk, but I don't know awk, so I'd welcome any suggestions. Show Sample Output


    3
    xmms2 mlib search NOT +rating | grep -r '^[0-9]' | sed -r 's/^([0-9]+).*/\1/' | sort -R | head | xargs -L 1 xmms2 addid
    goodevilgenius · 2009-04-16 20:27:30 4
  • This pipeline will find, sort and display all files based on mtime. This could be done with find | xargs, but the find | xargs pipeline will not produce correct results if the results of find are greater than xargs command line buffer. If the xargs buffer fills, xargs processes the find results in more than one batch which is not compatible with sorting. Note the "-print0" on find and "-0" switch for perl. This is the equivalent of using xargs. Don't you love perl? Note that this pipeline can be easily modified to any data produced by perl's stat operator. eg, you could sort on size, hard links, creation time, etc. Look at stat and just change the '9' to what you want. Changing the '9' to a '7' for example will sort by file size. A '3' sorts by number of links.... Use head and tail at the end of the pipeline to get oldest files or most recent. Use awk or perl -wnla for further processing. Since there is a tab between the two fields, it is very easy to process. Show Sample Output


    3
    find $HOME -type f -print0 | perl -0 -wn -e '@f=<>; foreach $file (@f){ (@el)=(stat($file)); push @el, $file; push @files,[ @el ];} @o=sort{$a->[9]<=>$b->[9]} @files; for $i (0..$#o){print scalar localtime($o[$i][9]), "\t$o[$i][-1]\n";}'|tail
    drewk · 2009-09-21 22:11:16 3

  • 3
    find . -name "*.txt" -exec sed -i "s/old/new/" {} \;
    tyboon · 2009-09-24 20:30:00 2
  • Do the same as pssh, just in shell syntax. Put your hosts in hostlist, one per line. Command outputs are gathered in output and error directories.


    3
    xargs -n1 -P100 -I{} sh -c 'ssh {} uptime >output/{} 2>error/{}' <hostlist
    dooblem · 2010-08-20 11:03:11 0
  • Preserve file structure when coping and exclude some file o dir patterns Show Sample Output


    3
    find ./ ! -name 'excludepattern' | xargs -i cp --parents {} destdir
    starchox · 2010-09-27 21:36:50 2
  •  1 2 3 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

updatedb for MAC OSX
MAC OSX doesn't come with an updatedb command by default, this will emulate the updatedb thats on a typical Linux OS. Simply add it to your ~/.bash_profile

Switch all connected PulseAudio bluetooth devices to A2DP profile
Tries to switch all audio devices to the A2DP profile for optimal sound quality. Useful for bluetooth speakers and headphones that always power up in HSP/HFP mode. Note however that this command is only a shorthand for the GUI, so it cannot fix stubborn BT controllers that leave your device stuck in HSP mode until a manual re-coupling.

most used unix commands

Find broken symlinks in the current directory and its subdirectories.
This is best run as root to avoid permission denials that can produce false positives. Obviously you can specify a directory in the usual way: $ find -L dirname -type l I can't remember where I read about this or who deserves the credit for it. The find(1) manual page hints strongly toward it, however.

How to pull out lines between two patterns
This command will display all lines between 2 patterns: word-a and word-b useful for grepping command outputs from file

delete multiple files from git index that have already been deleted from disk
delete multiple files from git index that have already been deleted from disk. this is pretty terrible, I'm looking for a better way. (much better!! http://www.commandlinefu.com/commands/view/1246/git-remove-files-which-have-been-deleted)

Find Duplicate Files (based on size first, then MD5 hash)
Finds duplicates based on MD5 sum. Compares only files with the same size. Performance improvements on: $find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate The new version takes around 3 seconds where the old version took around 17 minutes. The bottle neck in the old command was the second find. It searches for the files with the specified file size. The new version keeps the file path and size from the beginning.

Crash bash, in case you ever want to for whatever reason
This is a very hackish way to do it that I'm mainly just posting for fun, and I guess technically can more accurately be said to result in undefined behavior. What the command does is tell the shell to treat libpng like it's a shell plugin (which it's most certainly not) and attempt to install a "png_create_read" command from the library. It looks for the struct with the information about the command; since it's always the command name followed by "_struct", it'll look for a symbol called "png_create_read_struct". And it finds it, since this is the name of one of libpng's functions. But bash has no way to tell it's a function instead of a struct, so it goes ahead and parses the function's code as if it was command metadata. Inevitably, bash will attempt to dereference an invalid pointer or whatever, resulting in a segfault.

list files recursively by size

Create a DOS floppy image
mount with: mount -t msdos -o loop ./floppy.img /tmp/mnt


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: