Efficiently extract lines between markers

sed -n '/START/,${/STOP/q;p}'
GNU Sed can 'address' between two regex, but it continues parsing through to the end of the file. This slight alteration causes it to terminate reading the input file once the STOP match is made. In my example I have included an extra '/START/d' as my 'start' marker line contains the 'stop' string (I'm extracting data between 'resets' and using the time stamp as the 'start'). My previous coding using grep is slightly faster near the end of the file, but overall (extracting all the reset cycles in turn) the new SED method is quicker and a lot neater.
Sample Output 
Match near start of file
--
$ time (sed -n '/,166499.248,/,/!./{/166499/d;/RTKCOMMAND/q;/#BESTPOS/p}' ../UUT1.ASC > temp_time_file)

real    0m0.093s
user    0m0.030s
sys     0m0.030s

$  time (sed -n '/,166499.248,/,/#RTKCOMMAND/{/#BESTPOS/p}' ../UUT1.ASC > temp_time_file)

real    0m2.015s
user    0m0.484s
sys     0m0.061s

$ time (grep -A 2500 -m 1 ",166499.248," ../UUT1.ASC | grep -A 2500 -m  1 '\#RTKCOMMAND' | grep '\#BESTPOS' > temp_time_file)

real    0m0.171s
user    0m0.122s
sys     0m0.060s
--

Match near end of file
--
$ time (sed -n '/,230399.072,/,/#RTKCOMMAND/{/#BESTPOS/p}' ../UUT1.ASC > temp_time_file)

real    0m2.000s
user    0m0.390s
sys     0m0.046s

$ time (sed -n '/,230399.072,/,/!./{/,230399.072,/d;/RTKCOMMAND/q;/#BESTPOS/p}' ../UUT1.ASC > temp_time_file)

real    0m2.015s
user    0m0.468s
sys     0m0.076s

$ time (grep -A 2500 -m 1 ",230399.072," ../UUT1.ASC | grep -A 2500 -m  1 '\#RTKCOMMAND' | grep '\#BESTPOS' > temp_time_file)

real    0m1.703s
user    0m0.137s
sys     0m0.076s
--

3
By: mungewell
2009-06-19 15:27:36
sed sed grep
1
Submit An Alternative

What Others Think

if you want the line with the STOP marker output as well you can use sed -n '/START/,/!./{p;/STOP/q}'
mungewell · 866 weeks ago

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Similar Commands

Extract specific lines from a text file using Stream Editor (sed)

extract a certain number of lines from a file and dump them to another file

Every Nth line position # (SED)

Extract specific lines from a text file using Stream Editor (sed)

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: