Commands using awk (1,418)

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Automatic ssh Session Logger
A wrapper around ssh to automatically provide logging and session handling. This function runs ssh, which runs screen, which runs script. . The logs and the screen session are stored on the server. This means you can leave a session running and re-attach to it later, or from another machine. . . Requirements: * Log sessions on a remote server * Transparent - nothing extra to type * No installation - nothing to copy to the server beforehand . Features: * Function wrapper delegating to ssh - so nothing to remember - uses .ssh/config as expected - passes your command line option to ssh * Self-contained: no scripts to install on the server * Uses screen(1), so is: - detachable - re-attachable - shareable * Records session using script(1) * Configurable log file location, which may contain variables or whitespace L="$HOME" # local variable L="\$HOME" # server variable L="some space" . Limitations: * Log dir/file may not contain '~' (which would require eval on the server) . . The sessions are named by the local user connecting to the server. Therefore if you detach and re-run the same command you will reconnect to your original session. If you want to connect/share another's session simply run: $ USER=bob ssh root@server . The command above is stripped down to an absolute minimum. A fully expanded and annotated version is available as a Gist (git pastebin): https://gist.github.com/flatcap/3c42326abeb1197ee714 . If you want to add timing info to script, change the command to: $ ssh(){ L="\$HOME/logs/$(date +%F_%H:%M)-$USER";/usr/bin/ssh -t "$@" "mkdir -p \"${L%/*}\";screen -xRRS $USER script --timing=\"$L-timing\" -f \"$L\"";}

Making scripts runs on backgourd and logging output
Save all output to a log.

Loops over files, runs a command, dumps output to a file
In this case I'm selecting all php files in a dir, then echoing the filename and piping it to ~/temp/errors.txt. Then I'm running my alias for PHPCS (WordPress flags in my alias), then piping the PHPCS output to grep and looking for GET. Then I'm piping that output to the same file as above. This gets a list of files and under each file the GET security errors for that file. Extrapolate this to run any command on any list of files and pipe the output to a file. Remove the >> ~/temp/errors.txt to get output to the screen rather than to a file.

Upgrade all perl modules via CPAN

loop over a set of items that contain spaces
If you want to operate on a set of items in Bash, and at least one of them contains spaces, the `for` loop isn't going to work the way you might expect. For example, if the current dir has two files, named "file" and "file 2", this would loop 3 times (once each for "file", "file", and "2"): $ for ITEM in `ls`; do echo "$ITEM"; done Instead, use a while loop with `read`: $ ls | while read ITEM; do echo "$ITEM"; done

List Threads by Pid along with Thread Start Time
This command will list all threads started by a particular pid along with the start time of each thread. This is very valuable when diagnosing thread leaks.

show framebuffer console modes to use in grub vga option
look at /boot/grub/menu.lst for somethig like: ## additional options to use with the default boot option, but not with the ## alternatives ## e.g. defoptions=vga=791 resume=/dev/hda5 ## defoptions=vga=795 # defoptions=vga=873 ## altoption boot targets option ## multiple altoptions lines are allowed ## e.g. altoptions=(extra menu suffix) extra boot options ## altoptions=(recovery) single # altoptions=(verbose mode) vga=775 debug # altoptions=(console mode) vga=ask # altoptions=(graphic mode) quiet splash # altoptions=(recovery mode) single vga=(decimal value) is framebuffer mode

Swap a file or dir with quick resotre
This lets you replace a file or directory and quickly revert if something goes wrong. For example, the current version of a website's files are in public_html. Put a new version of the site in public_html~ and execute the command. The names are swapped. If anything goes wrong, execute it again (up arrow or !!).

sort ip by count quickly with awk from apache logs
creates associative array from apache logs, assumes "combined" log format or similar. replace awk column to suit needs. bandwidth per ip is also useful. have fun. I haven't found a more efficient way to do this as yet. sorry, FIXED TYPO: log file should obviously go after awk, which then pipes into sort.

untar undo


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: