Commands by 2chg (2)

  • Removes all lines between the lines containing "" and "", including these lines itself Backdrop: Sometimes when working with XML files without an graphical editor, large comment-/annotation-blocks taper the readability to walk through the file. I like to create a copy of such documents without these annotations. As the documentation itself is in documentation tags inside the annotation tags an therefore graphical editors tend to put the annotation tags in their own lines, this command removes all documentations within annotation-tags. Show Sample Output


    -1
    awk "/<xsd:annotation>/{h=1};!h;/<\/xsd:annotation>/{h=0}" annotatedSchema.xsd
    2chg · 2011-07-15 07:17:17 8
  • Improvement of the command "Find Duplicate Files (based on size first, then MD5 hash)" when searching for duplicate files in a directory containing a subversion working copy. This way the (multiple dupicates) in the meta-information directories are ignored. Can easily be adopted for other VCS as well. For CVS i.e. change ".svn" into ".csv": find -type d -name ".csv" -prune -o -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type d -name ".csv" -prune -o -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate Show Sample Output


    2
    find -type d -name ".svn" -prune -o -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type d -name ".svn" -prune -o -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate
    2chg · 2010-01-28 09:45:29 6

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Get info about a GitHub user
In this example 'git' is the user name and the output format is YAML but you can change this to XML or JSON, eg: curl http://github.com/api/v1/json/usernamehere

Ask for a password, the passwd-style
You can ask repeatedly for a non-blank password using this function: function read_password() { while [ ! -n "$USER_PASSWORD" ]; do read -s -p"Password: " USER_PASSWORD if [ ! -n "$USER_PASSWORD" ]; then echo "ERROR: You must specify a valid password, please try again" fi echo done } Also you can set a time out (in seconds) to write the password read -t 10 -s -p"Password: " USER_PASSWORD_VARIABLE if [ ! $? -eq 0 ]; then echo "Time out!" fi

Convert AVI to iPhone MP4
Fir3net.com

statistics in one line
In this example, file contains five columns where first column is text. Variance is calculated for columns 2 - 5 by using perl module Statistics::Descriptive. There are many more statistical functions available in the module.

list all opened ports on host
in loop, until the last port (65535), list all opened ports on host. in the sample I used localhost, but you can replace with any host to test.

Lists unambigously names of all xml elements used in files in current directory
This set of commands was very convenient for me when I was preparing some xml files for typesetting a book. I wanted to check what styles I had to prepare but coudn't remember all tags that I used. This one saved me from error-prone browsing of all my files. It should be also useful if one tries to process xml files with xsl, when using own xml application.

List open files that have no links to them on the filesystem
I have come across a situation in the past where someone has unlinked a file by running an 'rm' command against it while it was still being written to by a running process. The problem manifested itself when a 'df' command showed a filesystem at 100%, but this did not match the total value of a 'du -sk *'. When this happens, the process continues to write to the file but you can no longer see the file on the filesystem. Stopping and starting the process will, more often than not, get rid of the unlinked file, however this is not always possible on a live server. When you are in this situation you can use the 'lsof' command above to get the PID of the process that owns the file (in the sample output this is 23521). Run the following command to see a sym-link to the file (marked as deleted): $ cd /proc/23521/fd && ls -l Truncate the sym-link to regain your disk space: $ > /proc/23521/fd/3 I should point out that this is pretty brutal and *could* potentially destabilise your system depending on what process the file belongs to that you are truncating.

Email yourself after a job is done
This is a two part command that comes in really handy if you're running commands that take longer than you're willing to wait. The commands are separated by the semicolon(;) The first command is whatever you're attempting to do. The second commands emails you after the job completes.

Substitute audio track of video file using mencoder
Creates a new video file with video stream copied from input file and a different audio stream

Prepare a commandlinefu command.
This command will format your alias or function to a single line, trimming duplicate white space and newlines and inserting delimiter semi-colons, so it continues to work on a single line.


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: