Commands tagged unique (4)

  • uniq without pre-sorting

    Reads stdin, and outputs each line only once - without sorting ahead of time. This does use more memory than your system's sort utility.


    4
    perl -ne 'print if !$a{$_}++'
    doherty · 2011-02-17 02:18:44 8
  • View advanced Sort options, Quick Reference Help Alias

    Once you get into advanced/optimized scripts, functions, or cli usage, you will use the sort command alot. The options are difficult to master/memorize however, and when you use sort commands as much as I do (some examples below), it's useful to have the help available with a simple alias. I love this alias as I never seem to remember all the options for sort, and I use sort like crazy (much better than uniq for example). # Sorts by file permissions find . -maxdepth 1 -printf '%.5m %10M %p\n' | sort -k1 -r -g -bS 20% 00761 drwxrw---x ./tmp 00755 drwxr-xr-x . 00701 drwx-----x ./askapache-m 00644 -rw-r--r-- ./.htaccess # Shows uniq history fast history 1000 | sed 's/^[0-9 ]*//' | sort -fubdS 50% exec bash -lxv export TERM=putty-256color Taken from my http://www.askapache.com/linux-unix/bash_profile-functions-advanced-shell.html Show Sample Output


    3
    alias sorth='sort --help|sed -n "/^ *-[^-]/s/^ *\(-[^ ]* -[^ ]*\) *\(.*\)/\1:\2/p"|column -ts":"'
    AskApache · 2010-06-10 21:30:31 10
  • Unique sort

    Uniq command is mostly used in combination with sort command, as uniq removes duplicates only from a sorted file. i.e In order for uniq to work, all the duplicate entries should be in the adjacent lines. Show Sample Output


    0
    sort namesd.txt | uniq
    ankush108 · 2012-06-26 19:22:34 7
  • Show duplicate lines in a file

    The following displays only the entries that are duplicates. Show Sample Output


    0
    sort namesd.txt | uniq ?cd
    ankush108 · 2012-06-26 19:23:58 4

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Clear all Windows Event Log entries (cygwin)
Efficiently clear all Windows Event log entries from within a Cygwin terminal. Uses "cygstart" to launch a hidden "PowerShell" session passing a Powershell command to loop through and clear all Windows Event Log entries. Very useful for troubleshooting and debugging. The command should in theory elevate you session if needed. One liner is based on the PowerShell command: $ wevtutil el | foreach { wevtutil cl $_ }

recursively change file name from uppercase to lowercase (or viceversa)
easier way to recursively change files to lowercase using rename instead

Check your unread Gmail from the command line
Checks the Gmail ATOM feed for your account, parses it and outputs a list of unread messages. For some reason sed gets stuck on OS X, so here's a Perl version for the Mac: $ curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | tr -d '\n' | awk -F '' '{for (i=2; i

delete multiple files with spaces in filenames (with confirmation)
ls -Q will show the filenames in quotes. xargs -p rm will print all the filenames piped from ls -Q and ask for confirmation before deleting the files. without the -Q switch, if we have spaces in names, then the files won't be deleted.

Gzip files older than 10 days matching *
Useful for a cron job that runs nightly, gzipping or alternatively deleting files from a specific directory that are older than 10 days (in this case)

Grab the first 3 octets of your ip addresses
For machines that have many ip blocks spanning different Class C's, this will show which ones.

Get IP from hostname

Purge configuration files of removed packages on debian based systems
also search with aptitude search '~c'

Scan your LAN for unauthorized IPs
populate the auth.hosts file with a list of IP addresses that are authorized to be in use and when you run this command it will return the addresses that are pingable and not in the authorized list. Can be combined with the "Command line Twitter" command to tweet unauthorized access.

Stop Flash from tracking everything you do.
Brute force way to block all LSO cookies on a Linux system with the non-free Flash browser plugin. Works just fine for my needs. Enjoy.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: