Delimiter Hunting

Search in decimal rather than hex. od dumps the character list, cut to remove offsets, sort -u gives the used characters. seq gives the comparison list, but we need this sorted alphabetically for comm, which does the filtering. I drop to perl to convert back to characters (is there a better way?) and then use od to dump them in a print-safe format.

Sample Output

0000000  \0 001  \n   j   k  \v  \f   z   {   |   }   ~ 177  \r 016 017
0000020 020 021 022 023 002 024 025 026 027 030 031 032 033 034 035 003
0000040 036 037       !   "   #   $   %   &   ' 004   (   )   *   +   ,
0000060   -   .   /   0   1 005   2   3   4   5   6   7   8   9   :   ;
0000100 006   <   =   >   ?   @   A   B   C   D   E  \a   F   G   H   I
0000120   J   K   L   M   N   O  \b   P   Q   R   S   T   U   V   W   X
0000140   Y  \t   Z   [   \   ]   ^   _   `   a   b   c
0000154

By: bazzargh

2012-01-09 01:32:20

comm cut od perl seq sort bash

2 Alternatives + Submit Alt

Delimiter Hunting

Scan a file and print out a list of ASCII characters that are not used in the file which can then be safely used to delimit fields. Useful when needing to convert CSV files using "," to a single character delimiter. Piping it into less at the end (which could be redundant) stops the command characters being interpreted by the terminal.
This is sample output - yours may be different.
0

for i in `seq 0 9` A B C D E F; do for j in `seq 0 9` A B C D E F; do HEX=\$\'\\x${i}${j}\'; if ! eval grep -qF "$HEX" file; then eval echo $HEX \\x${i}${j}; fi; done; done 2> /dev/null | less

moogmusic · 2012-01-05 10:09:07 49
Delimiter Hunting

Here's a perl version that only considers printable characters. Change the regex /[[:print:]]/ to look for different sets of delimiter characters.
This is sample output - yours may be different.
0

perl -e '$f = join("", <>); for (0..127) {$_ = chr($_); if (/[[:print:]]/) {print if index($f, $_) < 0}} print "\n"'

putnamhill · 2012-01-05 23:38:06 28

What Others Think

The coding help from experts really wise for the students and the beginners as well. Through the help on ">https://floridanewstimes.com/the-importance-of-using-custom-writing-services-for-better-grades/286800/ blog there should be the people who able to improve their knowledge about. Thank you so much for the great update that is just tremendous to all.

AudreyJ · 172 weeks and 2 days ago

rahimhh21 · 118 weeks and 3 days ago

Perfecthomepugs · 109 weeks and 1 day ago

The sports autofocus conveys sharp, bright 3D images from 20 yards to infinity. No refocusing is needed. opticstrends review

kenwood · 86 weeks and 6 days ago

The number on which less cash is contributed, that number is chosen. This opened number is given to the bookie and this number arrives at everybody. satta matka

kenwood · 86 weeks ago

Suppressor Shops may offer package deals that include both a suppressor and compatible accessories. Suppressor Silencer Shop

kenwood · 79 weeks and 4 days ago

pugpuppies95 · 71 weeks and 3 days ago

yeneba6574 · 24 weeks and 5 days ago

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Similar Commands

Rebuild a Delimited File with a Unique Delimiter

Sort using kth column using : delimiter

Binary clock

strips the first field of each line where the delimiter is the first ascii character

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)