'hpc' in the box - starts a maximum of n compute commands modulo n controlled in parallel

c=0; n=8; while true; do r=`echo $RANDOM%5 |bc`; echo "sleep $r"; sleep $r& 2>&1 >/dev/null && ((c++)); [ `echo "$c%$n" | bc` -eq 0 ] && echo "$c waiting" && wait; done

the block of the loop is useful whenever you have huge junks of similar jobs, e.g., convert high res images to thumbnails, and make usage out of all the SMP power on your compute box without flooding the system. note: c is used as counter and the random sleep r=`echo $RANDOM%5 |bc`; echo "sleep $r"; sleep $r is just used as a dummy command.

Sample Output

$ c=0; n=8; while true; do r=`echo $RANDOM%5 |bc`; echo "sleep $r"; sleep $r& 2>&1 >/dev/null && ((c++)); [ `echo "$c%$n" | bc` -eq 0 ] && echo "$c waiting" && wait; done 
sleep 1
[1] 25900
sleep 3
[2] 14967
sleep 0
[3] 3011
sleep 0
[4] 24050
sleep 3
[5] 25533
sleep 4
[6] 9794
sleep 3
[7] 17325
sleep 2
[8] 19922
8 waiting
sleep 4
[9] 29693
sleep 4
[10] 1863
sleep 0
[11] 21993
sleep 1
[12] 1450
sleep 2
[13] 23873
sleep 1
[14] 24563
sleep 3
[15] 625
sleep 0
[16] 9172
16 waiting
...

By: cp

2010-07-08 13:56:28

echo sleep

1 Alternatives + Submit Alt

'hpc' in the shell - starts a maximum of n compute commands modulo n controlled in parallel, using make

this oneliner uses make and it's jobserver for parallel execution of your script. The '-j' flag for make defines number of subprocesses to launch, '-f' tells make use stdin instead of Makefile. Also make have neat flag '-l', which "Specifies that no new jobs (commands) should be started if there are others jobs running and the load is at least load (a floating-point number)." Also you can use plain Makefile, for better readability: targets = $(subst .png,.jpg,$(wildcard *.png)) (targets): echo convert $(subst .jpg,.png,$@) $@ all : $(targets)
This is sample output - yours may be different.
5

echo -n 'targets = $(subst .png,.jpg,$(wildcard *.png))\n$(targets):\n convert $(subst .jpg,.png,$@) $@ \nall : $(targets)' | make -j 4 -f - all

mechmind · 2010-07-15 07:19:17 9

What Others Think

wow, good idea!

simonbarkow · 751 weeks and 4 days ago

rahimhh21 · 118 weeks and 3 days ago

Saxenda Kopen Oxycodon 40 mg kopen Champix kopen Oxycodon teva kopen Tramadol 100 mg Stilnox 10mg kopen Strattera kopen Oxycodon kopen Diazepam 10 mg kopen Morfine injecteren Tylenol kopen Tramadol dosering Xanax 2mg kopen Lyrica 300mg Kopen Azithromycin 250 Kopen Oxycodon hcl Kopen Sustanon Kopen Ritalin Kopen Tramadol 50 mg Kopen Nolvadex Kopen Morfine 60 mg Kopen Pijnstillers oxycodon Oxycodone 20 mg Kopen Amoxicilline Kopen Benzedrine Kopen Insuline Kopen Suboxone Kopen Percocet Kopen Amfetamine Kopen Quaalude Kopen Fentanyl pleister Kopen Oxycontin kopen Oxycontin 40 mg Doxycycline 50 mg Alprazolam kopen Diclofenac kopen ketamine kopen Methamfetamine kopen Cocaine kopen Dexedrine kopen Oxazepam 50 mg Codeine kopen Flunitrazepam 2mg Mirtazapine kopen Midazolam kopen Nembutal kopen Temazepam 10 mg Oxycodon retardtabletten Fentanyl pleister Bromazepam 3 mg kopen Bromazepam kopen Doxycycline kopen Dormicum 15 mg Concerta kopen Kamagra 100mg kopen Baclofen kopen Temazepam 20 mg kopen Pijnstiller oxycodon Lasix-40mg kopen Lorazepam kopen Lorazepam 2.5 MG kopen Lyrica medicatie kopen Ketamine kopen Enanthate kopen Finasteride kopen Flurazepam kopen Concerta 36 mg kopen Concerta 54 mg kopen Adderall Nederland Imovane 7.5 mg kopen Clonazepam kopen Naproxen kopen Mdma capsules Zolpidem kopen Saxenda kopen Pijnstiller kopen Morfine Kopen Oxycontin 40 mg kopen Suboxon kopen MDMA Kopen Methylphenidate Kopen Naproxen Kopen Quaalude Kopen Morfine Injecteren Fentanyl Pleister Clonazepam Kopen Imovane Kopen Adderall Nederland Concerta 54 mg Kopen Concerta 36 mg Kopen Flurazepam Kopen Finasteride Kopen Amfetamine Kopen Amoxicilline Kopen Cocaine Kopen Diazepam Kopen Enanthate Kopen Ketamine Kopen Lyrica Medicatie Ketamine 10 gram Lorazepam 25 mg Lorazepam Kopen Lasix 40mg Kopen Percocet kopen Pijnstiller Oxycodon Pijnstillers Oxycodon Temazepam 20 mg Saxenda Kopen Champix Kopen Baclofen Kopen Doxycycline 50 mg Kamagra Kopen Oxycodon hcl Azithromycin Kopen Adderall Pillen Kopen Dexedrine Kopen Methamfetamine Kopen Oxycontin Kopen Concerta Kopen Dormicum 15 mg Diclofenac Kopen Doxycycline Kopen Bromazepam Kopen Bromazepam 3 mg kopen Fentanyl Pleister Kopen Oxycodon Retardtabletten Temazepam 10 mg Kopen Strattera 40 mg Kopen Lyrica 300mg Kopen Nembutal Kopen Midazolam Kopen Mirtazapine Kopen Insuline 10ml Kopen Flunitrazepam 2mg Benzedrine kopen Tylenol Kopen Ritalin Kopen Codeine 15mg Oxazepam Kopen Oxycodon Kopen oxycodone 20mg Kopen Xanax gele kopen Stilnox Kopen Alprazolam Kopen Zolpidem Kopen Nolvadex 10mg Kopen Oxycodon Slapen Sustanon Kopen Tramadol 200mg Tramadol 100mg Tramadol Paracetamol Kaufen Sie Oxycodon ohne Rezept Schmerzmittel kaufen Etizolam kaufen Lormetazepam kaufen Mirtazapin kaufen Oxazepam kaufen Oxazepam 50mg kaufen Pantoprazol kaufen Xanax 2mg kaufen Xanax 1 mg kaufen Temazepam kaufen Tramadol 50 mg kaufen Genotropin pen kaufen Flurazepam kaufen Finasterid kaufen Dormicum kaufen Diclofenac kaufen Concerta Tabletten kaufen Champix kaufen Bromazepam 6 mg kaufen Bromazepam 3 mg kaufen Baclofen kaufen Oxycodon Naloxon kaufen Nitrazepam 10 mg kaufen Clonazepam 2mg kaufen kaufen Adderall Ritalin 10mg kaufen Concerta 36 mg kaufen Concerta 54 mg kaufen Morphin kaufen Nitrazepam kaufen Oxycodon 5 mg kaufen Lorazepam kaufen Fentanyl Pflaster kaufen Temazepam kaufen Diazepam Rektal kaufen Ephedrin kaufen Lorazepam kaufen Oxycodon droge 15mg kaufen Tramadol 100 mg kaufen Doxycyclin kaufen Oxycodon Schmerzmittel kaufen Saxenda kaufen Oxycodon 40mg kaufen Diazepam kaufen Zolpidem kaufen Oxycodon 20 mg kaufen oxycodon 80 mg kaufen Oxycodon 10 mg kaufen Zopiclon kaufen Buy shipping containers in the USA and Canada online only with Queen Containers your most trusted and reliable shipping container supplier. We operate out of Texas and California. Buy Container Chassis, Shipping Container Accessories, Cold Containers, Blast freezers etc. Contact Queencontainers.com at support@queencontainers.com for more info https://queencontainers.com/ container chassis for sale 40 tilt bed trailer 40 ft reefer container blast reezer Buy freight farms 40 shipping container for sale Buy 40ft shipping container 40 shipping container shipping container sizes storage container with roll up door 45 HC shipping containers Buy drum pallets Buy wing pallet Buy new wood pallets Buy block pallet 20ft reefer container modified shipping containers 20 ft container 20ft HC shipping containers 40 ft shipping container for sale 10ft reefer container oxycodone Buy Acxion 30 mg Buy 20 mg Adderall Buy Alprazolam 1mg Buy Benzedrine pills Buy Codeine Phosphate Buy Concerta 18 mg Buy Concerta 27 mg Buy Concerta 54 mg Buy Dexedrine 10 mg Buy Diazepam 10 mg Buy Finasteride 5mg Buy Flunitrazepam 2mg Buy flurazepam 30 mg Buy generic adderall 30 mg Buy kamagra 100mg Buy ketamine 500mh Buy Lorazepam 1mg Buy Lorazepam 2 mg Buy lyrica 300 mg Buy generic lyrica Buy methylphenidate 20 mg Buy mirtazapine 30mg Buy mirtazapine 15 mg Buy mirtazapine 7.5 mg Buy morphine sulfate Buy naproxen 500 mg Buy oxazepam 15 mg Buy oxycodone 15mg Buy oxycodone 10mg Buy oxycodone 20mg Buy oxycodone 30mg Buy oxycodone 5mg Buy oxycodone hcl 60mg Buy oxycodone hydrochloride Buy oxycodone pill 80 mg Buy quaalude 300 mg Buy saxenda Buy strattera 40 mg Buy temazepam 10mg Buy temazepam 15 mg Buy temazepam 20 mg Buy temazepam 30 mg Buy testosterone enanthate Buy tramadol 50 mg Buy tramadol 100mg Buy voltaren 75 mg Buy voltaren pill Buy wegovy 2.4 mg Buy xanax 2mg Buy zolpidem 10 mg Buy fentanyl patch 100mcgh Buy transdermal fentanyl patch Buy lasix 40 mg Buy zolpidem 5mg Buy naproxen 250 mg Buy morphine sulfate 20mg/ml Bent u op zoek naar een containerdepot, dockingdiensten, containertransport, warehousing, zeecontainerreparaties etc. dan is A.S.H.C.CO hier om ervoor te zorgen dat u de beste containerdiensten krijgt in heel Europa. Neem contact op met Support@ashcco.com voor meer details https://ashcco.com/ 10 ft Container Blast Freezer 40 ft Reefer Container Reefer Container 40ft Shipping Container 40 ft Container Kopen Zeecontainer 40ft Zeecontainer 20ft Kopen Zeecontainer 20ft Pallet Kopen Euro Pallet Blok Pallet Pallets

alexander99 · 111 weeks and 6 days ago

Perfecthomepugs · 109 weeks and 4 days ago

Outstanding Service! Great Execution of their plans in work! They are just perfect! I would like to recommend them to everyone out there! You will not regret it! website design company in San Francisco

martinduke · 107 weeks ago

pugpuppies95 · 71 weeks and 5 days ago

yeneba6574 · 29 weeks and 4 days ago

yeneba6574 · 26 weeks and 4 days ago

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Similar Commands

Compute the average number of KB per file for each dir

grep (or anything else) many files with multiprocessor power

run command on a group of nodes in parallel

Convert entire audio library in parallel

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)