defragment files

find ~ -maxdepth 20 -type f -size -16M -print > t; for ((i=$(wc -l < t); i>0; i--)) do a=$(sed -n ${i}p < t); mv "$a" /dev/shm/d; mv /dev/shm/d "$a"; echo $i; done; echo DONE; rm t
Thanks to flatcap for optimizing this command. This command takes advantage of the ext4 filesystem's resistance to fragmentation. By using this command, files that were previously fragmented will be copied / deleted / pasted essentially giving the filesystem another chance at saving the file contiguously. ( unlike FAT / NTFS, the *nix filesystem always try to save a file without fragmenting it ) My command only effects the home directory and only those files with your R/W (read / write ) permissions. There are two issues with this command: 1. it really won't help, it works, but linux doesn't suffer much (if any ) fragmentation and even fragmented files have fast I/O 2. it doesn't discriminate between fragmented and non-fragmented files, so a large ~/ directory with no fragments will take almost as long as an equally sized fragmented ~/ directory The benefits i managed to work into the command: 1. it only defragments files under 16mb, because a large file with fragments isn't as noticeable as a small file that's fragmented, and copy/ delete/ paste of large files would take too long 2. it gives a nice countdown in the terminal so you know how far how much progress is being made and just like other defragmenters you can stop at any time ( use ctrl+c ) 3. fast! i can defrag my ~/ directory in 11 seconds thanks to the ramdrive powering the command's temporary storage bottom line: 1. its only an experiment, safe ( i've used it several times for testing ), but probably not very effective ( unless you somehow have a fragmentation problem on linux ). might be a placebo for recent windows converts looking for a defrag utility on linux and won't accept no for an answer 2. it's my first commandlinefu command
Sample Output

By: LinuxMan
2010-07-07 04:29:22

These Might Interest You

  • This command defragment the SQLite databases found in the home folder of the current Windows user. This is usefull to speed up Firefox startup. The executable sqlite3.exe must be located in PATH or in the current folder. In a script use: for /f "delims==" %%a in (' dir "%USERPROFILE%\*.sqlite" /s/b ') do echo vacuum;|"sqlite3.exe" "%%a" Show Sample Output

    for /f "delims==" %a in (' dir "%USERPROFILE%\*.sqlite" /s/b ') do echo vacuum;|"sqlite3.exe" "%a"
    vutcovici · 2010-01-18 20:56:00 2
  • Yeah, there are many ways to do that. Doing with sed by using a for loop is my favourite, because these are two basic things in all *nix environments. Sed by default does not allow to save the output in the same files so we'll use mv to do that in batch along with the sed. Show Sample Output

    for files in $(ls -A directory_name); do sed 's/search/replaced/g' $files > $ && mv $ $files; done;
    bassu · 2009-05-07 20:13:07 6
  • Rather than typing out all 10 files, you can use brace expansion to do the trick for you. This is useful for backup files, numbered files, or any files with a repeating pattern. Gives more control than 'rm file*' as I might want to keep others around.

    rm file{1..10}
    atoponce · 2009-03-02 14:42:05 2
  • A quick find command to identify all TAR files in a given path, extract a list of files contained within the tar, then search for a given string in the filelist. Returns to the user as a list of TAR files found (enclosed in []) followed by any matching files that exist in that archive. TAR can easily be swapped for JAR if required. Show Sample Output

    find . -type f -name "*.tar" -printf [%f]\\n -exec tar -tf {} \; | grep -iE "[\[]|<filename>"
    andrewtayloruk · 2011-01-06 13:01:38 0

What Others Think

I'm impressed, but why is the chmod there at all? Why not keep the file's original permissions?
kaedenn · 411 weeks and 4 days ago
I'm intrigued to know what you're doing that's SO time dependent. ext4 is pretty fast, so unless you have millions of files, or you read these files millions of times, I doubt you'd ever notice the problem of fragmentation. As for the script... First get rid of sudo. You're performing this operation in your own home directory. Next, if you mv a file onto /dev/shm (another filesystem) and you mv it back you will get a new file. By mv'ing the file, you don't need to worry about chmod (or rm). Big security note, though. If someone else has created a file in /dev/shm called 'd', your script will replace all your files with it. Next, I've used wc (word count) which is faster than grep. I only use it once, too, to initialise a for loop. Then, instead of sed reading/altering/writing the file list every loop, I get it to give me the n'th line. Finally, I added "rm t" to clean up the temporary file. For speed, you could put the file in /dev/shm too :-) Here's my updated version: find ~ -maxdepth 20 -type f -size -16M -print > t; for ((i=$(wc -l < t); i>0; i--)) do a=$(sed -n ${i}p < t); mv "$a" /dev/shm/d; mv /dev/shm/d "$a"; echo $i; done; echo DONE; rm t 74 fewer chars. List of commands: find, wc, sed, mv, rm, echo
flatcap · 411 weeks and 4 days ago
Thank you flatcap, you've managed to squeeze my command into an even smaller space and remove the junk from my command ( left over from my big 2kb command, i was so concerned with reducing the size of to under 255 characters, i failed to realize that some code from the original script wasn't necessary anymore ). after testing your script, i would like to know if i can replace my command with yours. @kaedenn the chmod was left over from my original script that used "dd" and attempted to work on files aoutside of the home directory
LinuxMan · 411 weeks and 3 days ago
No problem, I enjoyed the challenge :-) I didn't do much testing, but I'm /fairly/ certain it'll do the same as your script. Be careful before using it on your home directory.
flatcap · 411 weeks and 3 days ago
You could use mktemp to bypass the potential /dev/shm/d file issue. The temp file will be created when the var is set. d=$(mktemp --tmpdir=/dev/shm); find ~ -maxdepth 20 -type f -size -16M -print > t; for ((i=$(wc -l < t); i>0; i--)) do a=$(sed -n ${i}p < t); mv "$a" /dev/shm/$d; mv /dev/shm/$d "$a"; echo $i; done; echo DONE; rm t; rm $d
Vilemirth · 237 weeks and 3 days ago
Sorry. Screwed that up. You don't need the path for $d, as it's already there. d=$(mktemp --tmpdir=/dev/shm); find ~ -maxdepth 20 -type f -size -16M -print > t; for ((i=$(wc -l < t); i>0; i--)) do a=$(sed -n ${i}p < t); mv "$a" "$d"; mv "$d" "$a"; echo $i; done; echo DONE; rm t; rm $d
Vilemirth · 237 weeks and 3 days ago

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this? is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.


Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: