Commands tagged xargs (148)

  • Liked command 4077 so I improved it, by doing all text manipulation with sed. "Run this as root, it will be helpful to quickly get information about the loaded kernel modules." THX mohan43u Show Sample Output


    1
    lsmod | sed -e '1d' -e 's/\(\([^ ]*\) \)\{1\}.*/\2/' | xargs modinfo | sed -e '/^dep/s/$/\n/g' -e '/^file/b' -e '/^desc/b' -e '/^dep/b' -e d
    marssi · 2009-11-17 22:51:08 4
  • There is a limit to how many processes you can run at the same time for each user, especially with web hosts. If the maximum # of processes for your user is 200, then the following sets OPTIMUM_P to 100. OPTIMUM_P=$(( (`ulimit -u` - `find /proc -maxdepth 1 \( -user $USER -o -group $GROUPNAME \) -type d|wc -l`) / 2 )) This is very useful in scripts because this is such a fast low-resource-intensive (compared to ps, who, lsof, etc) way to determine how many processes are currently running for whichever user. The number of currently running processes is subtracted from the high limit setup for the account (see limits.conf, pam, initscript). An easy to understand example- this searches the current directory for shell scripts, and runs up to 100 'file' commands at the same time, greatly speeding up the command. find . -type f | xargs -P $OPTIMUM_P -iFNAME file FNAME | sed -n '/shell script text/p' I am using it in my http://www.askapache.com/linux-unix/bash_profile-functions-advanced-shell.html especially for the xargs command. Xargs has a -P option that lets you specify how many processes to run at the same time. For instance if you have 1000 urls in a text file and wanted to download all of them fast with curl, you could download 100 at a time (check ps output on a separate [pt]ty for proof) like this: cat url-list.txt | xargs -I '{}' -P $OPTIMUM_P curl -O '{}' I like to do things as fast as possible on my servers. I have several types of servers and hosting environments, some with very restrictive jail shells with 20processes limit, some with 200, some with 8000, so for the jailed shells my xargs -P10 would kill my shell or dump core. Using the above I can set the -P value dynamically, so xargs always works, like this. cat url-list.txt | xargs -I '{}' -P $OPTIMUM_P curl -O '{}' If you were building a process-killer (very common for cheap hosting) this would also be handy. Note that if you are only allowed 20 or so processes, you should just use -P1 with xargs. Show Sample Output


    1
    echo $(( `ulimit -u` - `find /proc -maxdepth 1 \( -user $USER -o -group $GROUPNAME \) -type d|wc -l` ))
    AskApache · 2010-03-12 08:42:49 6

  • 1
    find . -name '*.?pp' -exec grep -H "string" {} \;
    rthemocap · 2010-07-14 15:10:23 5
  • This can be useful for transforming command-line args into input for xargs (one per line). This can also be done with ls if the args are filenames, but that's getting awfully close to Useless Use of Cat territory (http://partmaps.org/era/unix/award.html).


    1
    each() { (IFS=$'\n'; echo "$*") }
    BobbyTables · 2010-10-02 06:51:44 6
  • Grabs the cmdline used to execute the process, and the environment that the process is being run under. This is much different than the 'env' command, which only lists the environment for the shell. This is very useful (to me at least) to debug various processes on my server. For example, this lets me see the environment that my apache, mysqld, bind, and other server processes have. Here's a function I use: aa_ps_all () { ( cd /proc && command ps -A -opid= | xargs -I'{}' sh -c 'test $PPID -ne {}&&test -r {}/cmdline&&echo -e "\n[{}]"&&tr -s "\000" " "<{}/cmdline&&echo&&tr -s "\000\033" "\nE"<{}/environ|sort&&cat {}/limits' ); } From my .bash_profile at http://www.askapache.com/linux-unix/bash_profile-functions-advanced-shell.html Show Sample Output


    1
    cd /proc&&ps a -opid=|xargs -I+ sh -c '[[ $PPID -ne + ]]&&echo -e "\n[+]"&&tr -s "\000" " "<+/cmdline&&echo&&tr -s "\000\033" "\nE"<+/environ|sort'
    AskApache · 2010-10-22 02:34:33 14
  • Increase the modification date for the files selected with the find command.


    1
    find . -type f | while read line; do NEW_TS=`date -d@$((\`stat -c '%Y' $line\` + <seconds> )) '+%Y%m%d%H%M.%S'`; touch -t $NEW_TS ${line}; done
    angleto · 2010-11-18 14:03:32 3
  • recursively search dir for a a particular file type, search each file for a particular text. Show Sample Output


    1
    find /name/of/dir/ -name '*.txt' | xargs grep 'text I am searching for'
    erickb · 2011-01-05 15:20:40 3
  • for when find . -print | grep -v .svn | xargs doesnt cut it.


    1
    find . -type f ! -iwholename \*.svn\* -print0 [ | xargs -0 ]
    alustenberg · 2011-03-21 16:45:35 6

  • 1
    find . -type d -name .svn -prune -o -type f -print0 | xargs -r0 ...
    depesz · 2011-03-21 21:36:57 4
  • This example command fetches 'example.com' webpage and then fetches+saves all PDF files listed (linked to) on that webpage. [*Note: of course there are no PDFs on example.com. This is just an example]


    1
    curl -s http://example.com | grep -o -P "<a.*href.*>" | grep -o "http.*.pdf" | xargs -d"\n" -n1 wget -c
    b_t · 2011-06-09 14:42:46 5
  • Grabs the Apache config file (yielded from httpd) and returns the path specified as DocumentRoot. Show Sample Output


    1
    httpd -V | grep -i SERVER_CONFIG_FILE | cut -f2 -d'"' | xargs grep -i '^DocumentRoot' | cut -f2 -d'"'
    dcpesses · 2011-08-13 20:45:05 5
  • Written for Mac OSX. When you are working in a project and want to open it on Github.com, just type "gh" and your default browser will open with the repo you are in. Works for submodules, and repo's that you don't own. You'll need to copy / paste this command into a gh.sh file, then create an alias in your bash or zsh profile to the gh.sh script. Detailed instructions here if you still need help: http://gist.github.com/1917716


    1
    git remote -v | grep fetch | sed 's/\(.*github.com\)[:|/]\(.*\).git (fetch)/\2/' | awk {'print "https://github.com/" $1'} | xargs open
    brockangelo · 2012-04-15 20:48:46 20
  • first grep all href images then sed the url part then wget


    1
    curl -s $1 | grep -o -i '<a href="//images.4chan.org/[^>]*>' | sed -r 's%.*"//([^"]*)".*%\1%' | xargs wget
    bugmenot · 2013-07-22 10:33:55 6
  • Sorts by latest modified files by looking to current directory and all subdirectories Show Sample Output


    1
    find . -name '*pdf*' -print0 | xargs -0 ls -lt | head -20
    fuats · 2013-10-03 21:58:51 9
  • Use GNU Parallel: short, easy to read, and will run one job per core.


    1
    parallel convert {} {.}.png ::: *.svg
    unixmonkey74668 · 2014-04-12 06:39:02 7
  • bash brace expansion, sequence expression Show Sample Output


    1
    echo {-1..-5}days | xargs -n1 date +"%Y-%m-%d" -d
    grault · 2014-07-22 17:56:07 8

  • 1
    find -L /path/to/check -type l | xargs rm
    sn0w · 2015-11-10 12:42:00 11
  • Neither of the others worked for me. This does.


    1
    curl http://url/rss | grep -o '<enclosure url="[^"]*' | grep -o '[^"]*$' | xargs wget -c
    dakira · 2016-05-29 12:07:21 21
  • this is good for variables if you have many script created files and if you want to know which one is the last created/changed one..


    1
    find . -type f -print0 | xargs -0 stat -c '%y %n' | sort -n -k 1,1 | awk 'END{print $NF}'
    emphazer · 2018-05-14 08:47:41 148
  • First the find command finds all files in your current directory (.). This is piped to xargs to be able to run the next shell pipeline in parallel. The xargs -P argument specifies how many processes you want to run in parallel, you can set this higher than your core count as the duration reading is mainly IO bound. The -print0 and -0 arguments of find and xargs respectively are used to easily handle files with spaces or other special characters. A subshell is executed by xargs to have a shell pipeline for each file that is found by find. This pipeline extracts the duration and converts it to a format easily parsed by awk. ffmpeg reads the file and prints a lot of information about it, grep extracts the duration line. cut and sed cut out the time information, and tr converts the last . to a : to make it easier to split by awk. awk is a specialized programming language for use in shell scripts. Here we use it to split the time elements in 4 variables and add them up. Show Sample Output


    1
    find . -print0 | xargs -0 -P 40 -n 1 sh -c 'ffmpeg -i "$1" 2>&1 | grep "Duration:" | cut -d " " -f 4 | sed "s/.$//" | tr "." ":"' - | awk -F ':' '{ sum1+=$1; sum2+=$2; sum3+=$3; sum4+=$4 } END { printf "%.0f:%.0f:%.0f.%.0f\n", sum1, sum2, sum3, sum4 }'
    pingiun · 2019-03-01 20:21:48 40
  • Directly download all mp3 files of the desired podcast


    1
    curl http://radiofrance-podcast.net/podcast09/rss_14726.xml | grep -Eo "(http|https)://[a-zA-Z0-9./?=_%:-]*mp3" | sort -u | xargs wget
    pascalvaucheret · 2021-08-09 13:40:26 176

  • 1
    xargs -I% -P10 curl -sL "https://iconfig.co" < <(printf '%s\n' {1..10})
    wuseman1 · 2022-06-02 02:13:14 352

  • 0
    find . -name "*.txt" | xargs sed -i "s/old/new/"
    noblejasper · 2009-09-24 15:02:31 3
  • Nice command to create a list, you can create too with for command, but this is so faster. Show Sample Output


    0
    seq 10 |xargs -n1 echo Printing line
    Waldirio · 2009-10-15 11:05:35 7
  • A bit shorter and parallelized. Depending on the speed of your cpu and your disk this may run faster. Parallel is from https://savannah.nongnu.org/projects/parallel/


    0
    find -not -empty -type f -printf "%s\n" | sort | uniq -d | parallel find -type f -size {}c | parallel md5sum | sort | uniq -w32 --all-repeated=separate
    unixmonkey8046 · 2010-01-28 08:40:18 4
  •  < 1 2 3 4 5 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Which processes are listening on a specific port (e.g. port 80)
swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

HTTP GET request on wireshark remotly

cycle through everything sox knows how to read, playing only the first three seconds
I wasted two hours reading the sox documentation and searching on the web for the format of some obscure fscking sound sample, and then finally came up with this. This plays only the first three seconds of your unknown formatted sound file using every one of sox's built-in filetypes. If you don't get an exact match, you may get close. . I could not fit every single type in and keep it under 127 characters, so you will have to replace "..." with the full list obtainable by `$ sox --help` (or try `Show sample output`) . note: /usr/bin/play should be linked to sox on most systems.

kill all foo process
Kill all processes with foo in them. Similar to pkill but more complete and also works when there is no pkill command. Works on almost every Linux/Unix platform I have tried.

List out classes in of all htmls in directory
Lists out all classes used in all *.html files in the currect directory. usefull for checking if you have left out any style definitions, or accidentally given a different name than you intended. ( I have an ugly habit of accidentally substituting camelCase instead of using under_scores: i would name soemthing counterBox instead of counter_box) WARNING: assumes you give classnames in between double quotes, and that you apply only one class per element.

Remove the first character of each line in a file

Hide or show Desktop Icons on MacOS
Hides all Files and Folders on the MacOS Desktop. To show files and folders, type "true" instead of "false". "Finder" at the end is case sensitive, "finder" doesn’t work

generate random tone

Show crontabs for all users
This is flatcaps tweaked command to make it work on SLES 11.2

print crontab entries for all the users that actually have a crontab
This is how I list the crontab for all the users on a given system that actually have a crontab. You could wrap it with a function block and place it in your .profile or .bashrc for quick access. There's prolly a simpler way to do this. Discuss.


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: