Commands using uniq (252)

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Generate a 18 character password, print the password and sha512 salted hash
Generate a 18 character password from character set a-zA-Z0-9 from /dev/urandom, pipe the output to Python which prints the password on standard out and in crypt sha512 form.

Restart Xen XAPI
Restarts the XAPI service on the host, mostly used by Xen Center. It does not affect any running VMs, just the Xen client tools that may be connected. On my list as XAPI frustratingly keeps running out of memory and getting killed off.

Which processes are listening on a specific port (e.g. port 80)
swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

Split a large file, without wasting disk space
It's common to want to split up large files and the usual method is to use split(1). If you have a 10GiB file, you'll need 10GiB of free space. Then the OS has to read 10GiB and write 10GiB (usually on the same filesystem). This takes AGES. . The command uses a set of loop block devices to create fake chunks, but without making any changes to the file. This means the file splitting is nearly instantaneous. The example creates a 1GiB file, then splits it into 16 x 64MiB chunks (/dev/loop0 .. loop15). . Note: This isn't a drop-in replacement for using split. The results are block devices. tar and zip won't do what you expect when given block devices. . These commands will work: $ hexdump /dev/loop4 . $ gzip -9 < /dev/loop6 > part6.gz . $ cat /dev/loop10 > /media/usb/part10.bin

find process associated with a port
e.g. fuser 25/tcp (see which pid is listening on smtp)

Rebuild a Delimited File with a Unique Delimiter
Useful for CSV files. In the command, the file in question is comma delimited but contains double quoted fields containing commas and contains no @ symbols (as confirmed with http://www.commandlinefu.com/commands/view/9998/delimiter-hunting). This command converts the delimiting commas to @s while preserving the commas in the fields using the "uniqueString" to mark the ends of lines.

Stat each file in a directory
Possible simplification of egrep-awk-sort with find and -exec with xargs.

Convert seconds to [DD:][HH:]MM:SS
Converts any number of seconds into days, hours, minutes and seconds. sec2dhms() { declare -i SS="$1" D=$(( SS / 86400 )) H=$(( SS % 86400 / 3600 )) M=$(( SS % 3600 / 60 )) S=$(( SS % 60 )) [ "$D" -gt 0 ] && echo -n "${D}:" [ "$H" -gt 0 ] && printf "%02g:" "$H" printf "%02g:%02g\n" "$M" "$S" }

github push-ing behind draconian proxies!
If you are behind a restrictive proxy/firewall that blocks port 22 connections but allows SSL on 443 (like most do) then you can still push changes to your github repository. Your .ssh/config file should contain: Host * ForwardX11 no TCPKeepAlive yes ProtocolKeepAlives 30 ProxyCommand /usr/local/bin/proxytunnel -v -p -d %h:443 Host User git Hostname ssh.github.com ChallengeResponseAuthentication yes IdentityFile ~/.ssh/id_rsa IdentitiesOnly yes Basically proxytunnel "tunnels" your ssh connection through port 443. You could also use corkscrew or some other tunneling program that is available in your distro's repository. PS: I generally use "github.com" as the SSH-HOST so that urls of the kind git@github.com:USER/REPO.git work transparently :) You

Configuring proxy client on terminal without leaving password on screen or in bash_history
Prompts the user for username and password, that are then exported to http_proxy for use by wget, yum etc Default user, webproxy and port are used. Using this script prevent the cleartext user and pass being in your bash_history and on-screen


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: