commandlinefu.com is the place to record those command-line gems that you return to again and again.
Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.
If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/
You can sign-in using OpenID credentials, or register a traditional username and password.
First-time OpenID users will be automatically assigned a username which can be changed after signing in.
Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.
Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):
Subscribe to the feed for:
Sort netflow packet capture by unique connections excluding source port.
on some distro's you have to replace "BogoMIPS" with "bogomips".
Remove duplicate line in a text file.
displays a list of all file extensions in current directory and how many files there are of each type of extension in ascending order (case insensitive)
* Find all file sizes and file names from the current directory down (replace "." with a target directory as needed).
* sort the file sizes in numeric order
* List only the duplicated file sizes
* drop the file sizes so there are simply a list of files (retain order)
* calculate md5sums on all of the files
* replace the first instance of two spaces (md5sum output) with a \0
* drop the unique md5sums so only duplicate files remain listed
* Use AWK to aggregate identical files on one line.
* Remove the blank line from the beginning (This was done more efficiently by putting another "IF" into the AWK command, but then the whole line exceeded the 255 char limit).
>>>> Each output line contains the md5sum and then all of the files that have that identical md5sum. All fields are \0 delimited. All records are \n delimited.
When trying to find an error in a hosted project it's interesting to find out how the source is organized: Are there .inc files? Or .php files only? Or .xml files that probably contain translated texts?
Analyze an Apache access log for the time period with most activity and display the hit count, requesting IP and the timestamp. May help detect a brute force dos attack.
cut can handle files as well. No neet for a cat.
Same as the rest, but handle IPv6 short IPs. Also, sort in the order that you're probably looking for.
The first sort is necessary for ips in a list to be actually unique.
Show's per IP of how many requests they did to the Apache webserver
dumpfile is a CSV file, which its 1st field is a phone number in format CC+10 digits
Empty lines are deleted, before the output in format "prefix,ocurrences"
Counts of messages by recipient, with frozen messages excluded.
This commands queries the delicious api then runs the xml through xml2, grabs the urls cuts out the first two columns, passes through uniq to remove duplicates if any, and then goes into linkchecker who checks the links. the links go the blacklist in ~/.linkchecker/blacklist. please see the manual pages for further info peeps. I took me a few days to figure this one out. I how you enjoy it. Also don't run these api more then once a few seconds you can get banned by delicious see their site for info. ~updated for no recursive
top 10 of access log
parallel can be installed on your central node and can be used to run a command multiple times.
In this example, multiple ssh connections are used to run commands. (-j is the number of jobs to run at the same time). The result can then be piped to commands to perform the "reduce" stage. (sort then uniq in this example).
This example assumes "keyless ssh login" has been set up between the central node and all machines in the cluster.
bashreduce may also do what you want.
make usable on OSX with filenames containing spaces. note: will still break if filenames contain newlines... possible, but who does that?!