[svn:perlfaq] r10472 - perlfaq/trunk
[email protected] Thu, 3 Jan 2008 12:26:45 -0800 (PST)
| Newsgroups | perl.cvs.perlfaq |
|---|---|
| Message-ID | <[email protected]> |
Author: comdog
Date: Thu Jan 3 12:26:45 2008
New Revision: 10472
Modified:
perlfaq/trunk/glossary.pod
perlfaq/trunk/perlfaq1.pod
perlfaq/trunk/perlfaq2.pod
perlfaq/trunk/perlfaq3.pod
perlfaq/trunk/perlfaq4.pod
perlfaq/trunk/perlfaq5.pod
perlfaq/trunk/perlfaq6.pod
perlfaq/trunk/perlfaq7.pod
perlfaq/trunk/perlfaq8.pod
perlfaq/trunk/perlfaq9.pod
Log:
* all pods: updated copyright statement to 2008
* perlfaq6: Can I use Perl regular expressions to match balanced text?
+ rewrote answer using new Perl5.10 recursive regex features
Modified: perlfaq/trunk/glossary.pod
==============================================================================
--- perlfaq/trunk/glossary.pod (original)
+++ perlfaq/trunk/glossary.pod Thu Jan 3 12:26:45 2008
@@ -288,7 +288,6 @@
to read in an entire file in one step
-
=head2 SWIG
An interface for allowing Perl to use C/C++ libraries and code.
@@ -302,7 +301,7 @@
=head2 Topaz
An old, exploratory project by Chip Salzenberg to rewrite Perl in C++.
-This project predates the Perl6 project and is now dead.
+This project predates the Perl 6 project and is now dead.
=head2 TPC
Modified: perlfaq/trunk/perlfaq1.pod
==============================================================================
--- perlfaq/trunk/perlfaq1.pod (original)
+++ perlfaq/trunk/perlfaq1.pod Thu Jan 3 12:26:45 2008
@@ -408,7 +408,7 @@
=head1 AUTHOR AND COPYRIGHT
-Copyright (c) 1997-2007 Tom Christiansen, Nathan Torkington, and
+Copyright (c) 1997-2008 Tom Christiansen, Nathan Torkington, and
other authors as noted. All rights reserved.
This documentation is free; you can redistribute it and/or modify it
Modified: perlfaq/trunk/perlfaq2.pod
==============================================================================
--- perlfaq/trunk/perlfaq2.pod (original)
+++ perlfaq/trunk/perlfaq2.pod Thu Jan 3 12:26:45 2008
@@ -537,7 +537,7 @@
=head1 AUTHOR AND COPYRIGHT
-Copyright (c) 1997-2007 Tom Christiansen, Nathan Torkington, and
+Copyright (c) 1997-2008 Tom Christiansen, Nathan Torkington, and
other authors as noted. All rights reserved.
This documentation is free; you can redistribute it and/or modify it
Modified: perlfaq/trunk/perlfaq3.pod
==============================================================================
--- perlfaq/trunk/perlfaq3.pod (original)
+++ perlfaq/trunk/perlfaq3.pod Thu Jan 3 12:26:45 2008
@@ -1041,7 +1041,7 @@
=head1 AUTHOR AND COPYRIGHT
-Copyright (c) 1997-2007 Tom Christiansen, Nathan Torkington, and
+Copyright (c) 1997-2008 Tom Christiansen, Nathan Torkington, and
other authors as noted. All rights reserved.
This documentation is free; you can redistribute it and/or modify it
Modified: perlfaq/trunk/perlfaq4.pod
==============================================================================
--- perlfaq/trunk/perlfaq4.pod (original)
+++ perlfaq/trunk/perlfaq4.pod Thu Jan 3 12:26:45 2008
@@ -2312,7 +2312,7 @@
=head1 AUTHOR AND COPYRIGHT
-Copyright (c) 1997-2007 Tom Christiansen, Nathan Torkington, and
+Copyright (c) 1997-2008 Tom Christiansen, Nathan Torkington, and
other authors as noted. All rights reserved.
This documentation is free; you can redistribute it and/or modify it
Modified: perlfaq/trunk/perlfaq5.pod
==============================================================================
--- perlfaq/trunk/perlfaq5.pod (original)
+++ perlfaq/trunk/perlfaq5.pod Thu Jan 3 12:26:45 2008
@@ -1314,7 +1314,7 @@
=head1 AUTHOR AND COPYRIGHT
-Copyright (c) 1997-2007 Tom Christiansen, Nathan Torkington, and
+Copyright (c) 1997-2008 Tom Christiansen, Nathan Torkington, and
other authors as noted. All rights reserved.
This documentation is free; you can redistribute it and/or modify it
Modified: perlfaq/trunk/perlfaq6.pod
==============================================================================
--- perlfaq/trunk/perlfaq6.pod (original)
+++ perlfaq/trunk/perlfaq6.pod Thu Jan 3 12:26:45 2008
@@ -459,33 +459,131 @@
=head2 Can I use Perl regular expressions to match balanced text?
X<regex, matching balanced test> X<regexp, matching balanced test>
-X<regular expression, matching balanced test>
+X<regular expression, matching balanced test> X<possessive> X<PARNO>
+X<Text::Balanced> X<Regexp::Common> X<backtracking> x<recursion>
-Historically, Perl regular expressions were not capable of matching
-balanced text. As of more recent versions of perl including 5.6.1
-experimental features have been added that make it possible to do this.
-Look at the documentation for the (??{ }) construct in recent perlre manual
-pages to see an example of matching balanced parentheses. Be sure to take
-special notice of the warnings present in the manual before making use
-of this feature.
-
-CPAN contains many modules that can be useful for matching text
-depending on the context. Damian Conway provides some useful
-patterns in Regexp::Common. The module Text::Balanced provides a
-general solution to this problem.
-
-One of the common applications of balanced text matching is working
-with XML and HTML. There are many modules available that support
-these needs. Two examples are HTML::Parser and XML::Parser. There
-are many others.
-
-An elaborate subroutine (for 7-bit ASCII only) to pull out balanced
-and possibly nested single chars, like C<`> and C<'>, C<{> and C<}>,
-or C<(> and C<)> can be found in
-http://www.cpan.org/authors/id/TOMC/scripts/pull_quotes.gz .
+(contributed by brian d foy)
-The C::Scan module from CPAN also contains such subs for internal use,
-but they are undocumented.
+Your first try should probably be the C<Text::Balanced> module, which
+is in the Perl standard library since Perl 5.8. It has a variety of
+functions to deal with tricky text. The C<Regexp::Common> module can
+also help by providing canned patterns you can use.
+
+As of Perl 5.10, you can match balanced text with regular expressions
+using recursive patterns. Before Perl 5.10, you had to resort to
+various tricks such as using Perl code in C<(??{})> sequences.
+
+Here's an example using a recursive regular expression. The goal is to
+capture all of the text within angle brackets, including the text in
+nested angle brackets. This sample text has two "major" groups: a
+group with one level of nesting and a group with two levels of
+nesting. There are five total groups in angle brackets:
+
+ I have some <brackets in <nested brackets> > and
+ <another group <nested once <nested twice> > >
+ and that's it.
+
+The regular expression to match the balanced text uses two new (to
+Perl 5.10) regular expression features. These are covered in L<perlre>
+and this example is a modified version of one in that documentation.
+
+First, adding the new possesive C<+> to any quantifier finds the
+longest match and does not backtrack. That's important since you want
+to handle any angle brackets through the recursion, not backtracking.
+The group C<< [^<>]++ >> finds one or more non-angle brackets without
+backtracking.
+
+Second, the new C<(?PARNO)> refers to the sub-pattern in the
+particular capture buffer given by C<PARNO>. In the following regex,
+the first capture buffer finds (and remembers) the balanced text, and
+you need that same pattern within the first buffer to get past the
+nested text. That's the recursive part. The C<(?1)> uses the pattern
+in the outer capture buffer as an independent part of the regex.
+
+Putting it all together, you have:
+
+ #!/usr/local/bin/perl5.10.0
+
+ my $string =<<"HERE";
+ I have some <brackets in <nested brackets> > and
+ <another group <nested once <nested twice> > >
+ and that's it.
+ HERE
+
+ my @groups = $string =~ m/
+ ( # start of capture buffer 1
+ < # match an opening angle bracket
+ (?:
+ [^<>]++ # one or more non angle brackets, non backtracking
+ |
+ (?1) # found < or >, so recurse to capture buffer 1
+ )*
+ > # match a closing angle bracket
+ ) # end of capture buffer 1
+ /xg;
+
+ $" = "\n\t";
+ print "Found:\n\t@groups\n";
+
+The output shows that Perl found the two major groups:
+
+ Found:
+ <brackets in <nested brackets> >
+ <another group <nested once <nested twice> > >
+
+With a little extra work, you can get the all of the groups in angle
+brackets even if they are in other angle brackets too. Each time you
+get a balanced match, remove its outer delimiter (that's the one you
+just matched so don't match it again) and add it to a queue of strings
+to process. Keep doing that until you get no matches:
+
+ #!/usr/local/bin/perl5.10.0
+
+ my @queue =<<"HERE";
+ I have some <brackets in <nested brackets> > and
+ <another group <nested once <nested twice> > >
+ and that's it.
+ HERE
+
+ my $regex = qr/
+ ( # start of bracket 1
+ < # match an opening angle bracket
+ (?:
+ [^<>]++ # one or more non angle brackets, non backtracking
+ |
+ (?1) # recurse to bracket 1
+ )*
+ > # match a closing angle bracket
+ ) # end of bracket 1
+ /x;
+
+ $" = "\n\t";
+
+ while( @queue )
+ {
+ my $string = shift @queue;
+
+ my @groups = $string =~ m/$regex/g;
+ print "Found:\n\t@groups\n\n" if @groups;
+
+ unshift @queue, map { s/^<//; s/>$//; $_ } @groups;
+ }
+
+The output shows all of the groups. The outermost matches show up
+first and the nested matches so up later:
+
+ Found:
+ <brackets in <nested brackets> >
+ <another group <nested once <nested twice> > >
+
+ Found:
+ <nested brackets>
+
+ Found:
+ <nested once <nested twice> >
+
+ Found:
+ <nested twice>
=head2 What does it mean that regexes are greedy? How can I get around it?
X<greedy> X<greediness>
@@ -994,7 +1092,7 @@
=head1 AUTHOR AND COPYRIGHT
-Copyright (c) 1997-2007 Tom Christiansen, Nathan Torkington, and
+Copyright (c) 1997-2008 Tom Christiansen, Nathan Torkington, and
other authors as noted. All rights reserved.
This documentation is free; you can redistribute it and/or modify it
Modified: perlfaq/trunk/perlfaq7.pod
==============================================================================
--- perlfaq/trunk/perlfaq7.pod (original)
+++ perlfaq/trunk/perlfaq7.pod Thu Jan 3 12:26:45 2008
@@ -1019,7 +1019,7 @@
=head1 AUTHOR AND COPYRIGHT
-Copyright (c) 1997-2007 Tom Christiansen, Nathan Torkington, and
+Copyright (c) 1997-2008 Tom Christiansen, Nathan Torkington, and
other authors as noted. All rights reserved.
This documentation is free; you can redistribute it and/or modify it
Modified: perlfaq/trunk/perlfaq8.pod
==============================================================================
--- perlfaq/trunk/perlfaq8.pod (original)
+++ perlfaq/trunk/perlfaq8.pod Thu Jan 3 12:26:45 2008
@@ -1318,7 +1318,7 @@
=head1 AUTHOR AND COPYRIGHT
-Copyright (c) 1997-2007 Tom Christiansen, Nathan Torkington, and
+Copyright (c) 1997-2008 Tom Christiansen, Nathan Torkington, and
other authors as noted. All rights reserved.
This documentation is free; you can redistribute it and/or modify it
Modified: perlfaq/trunk/perlfaq9.pod
==============================================================================
--- perlfaq/trunk/perlfaq9.pod (original)
+++ perlfaq/trunk/perlfaq9.pod Thu Jan 3 12:26:45 2008
@@ -673,7 +673,7 @@
=head1 AUTHOR AND COPYRIGHT
-Copyright (c) 1997-2007 Tom Christiansen, Nathan Torkington, and
+Copyright (c) 1997-2008 Tom Christiansen, Nathan Torkington, and
other authors as noted. All rights reserved.
This documentation is free; you can redistribute it and/or modify it