Recompress all files in current directory from gzip to bzip2

find . -type f -name '*.gz'|awk '{print "zcat", $1, "| bzip2 -c >", $0.".tmp", "&& rename", "s/.gz.tmp/.bz2/", "*.gz.tmp", "&& rm", $0}'|bash
This solution is similar to [1] except that it does not have any dependency on GNU Parallel. Also, it tries to minimize the impact on the running system (using ionice and nice). [1] http://www.commandlinefu.com/commands/view/7009/recompress-all-.gz-files-in-current-directory-using-bzip2-running-1-job-per-cpu-core-in-parallel

-2
By: Ztyx
2013-04-11 10:17:57

2 Alternatives + Submit Alt

  • Find all .gz files and recompress them to bz2 on the fly. No temp files. edit: forgot the double quotes! jeez!


    1
    find . -type f -name "*.gz" | while read line ; do gunzip --to-stdout "$line" | bzip2 > "$(echo $line | sed 's/gz$/bz2/g')" ; done
    Kaurin · 2013-04-12 19:18:21 1
  • - recompresses all gz files to bz2 files from this point and below in the directory tree - output shows the size of the original file, and the size of the new file. Useful. - conceptually easier to understand than playing tricks with awk and sed. - don't like output? Use the following line: for gz in `find . -type f -name '*.gz' -print`; do f=`basename $gz .gz` && d=`dirname $gz` && gunzip -c $gz | bzip2 - -c > $d/$f.bz2 && rm -f $gz ; done Show Sample Output


    0
    for gz in `find . -type f -name '*.gz' -print`; do f=`basename $gz .gz` && d=`dirname $gz` && echo -n `ls -s $gz` "... " && gunzip -c $gz | bzip2 - -c > $d/$f.bz2 && rm -f $gz && echo `ls -s $d/$f.bz2`; done
    pdwalker · 2014-03-13 08:36:24 0

What Others Think

Recompressing is typically CPU hard and I/O light. The GNU Parallel version uses all CPU cores in parallel. The GNU Parallel version can be ioniced by simply prepending parallel with 'ionice -c 3'.
unixmonkey8046 · 270 weeks and 6 days ago
unixmonkey8046: You have a point with I/O lightness. In this case, I wanted to be better safe than sorry since the command was executed on an already loaded system. To try to keep things simple I've removed the (io)nice calls.
Ztyx · 270 weeks and 6 days ago
Oh, and the reason I'm not using parallel is because it wasn't installed on the system I was working with and I usually prefer using common unix tools.
Ztyx · 270 weeks and 6 days ago
Are you aware how easy it is to do a personal installation of GNU Parallel: http://git.savannah.gnu.org/cgit/parallel.git/tree/README
unixmonkey8046 · 270 weeks and 6 days ago
Yes, but I'm not willing to do that on a production server.
Ztyx · 270 weeks and 5 days ago

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands



Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: