Spirit vs C++11 regex/ECMAScript

Michael Powell <[email protected]>
Newsgroups gmane.comp.parsers.spirit.general
Message-ID <CAMEoF_GNBy4Tau1HiJXdagBLNsYBizfVYHzRjG8ohrW4yjQthg@mail.gmail.com> 
Hello,

I have been using regex (ECMAScript) thus far in order to verify
whether messaging framework addresses are correct, or at least
correctly syntaxed. Which, thus far, to a point, has been working.

However, I am now in the depths of IPv6 verification and am running
into what I think may be a buggy ECMAScript with matching embedded
square brackets. Currently, based on Microsoft Visual C++ 2015
(C++14?).

The basic pattern is essentially modeled after IPv6 RFCs:

#define XPO_IPV6_ADDR_PATTERN "(" XPO_IPV6_ADDR_SUFFIX_N(6) XPO_IPV6_LS32 ")" \
                              "|(" XPO_NON_CAP XPO_COL_2X
XPO_IPV6_ADDR_SUFFIX_N(5) XPO_IPV6_LS32 ")" \
        "|(" XPO_NON_CAP "(" XPO_IPV6_H16 ")?" XPO_COL_2X
XPO_IPV6_ADDR_SUFFIX_N(4) XPO_IPV6_LS32 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(1) XPO_COL_2X
XPO_IPV6_ADDR_SUFFIX_N(3) XPO_IPV6_LS32 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(2) XPO_COL_2X
XPO_IPV6_ADDR_SUFFIX_N(2) XPO_IPV6_LS32 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(3) XPO_COL_2X
XPO_IPV6_ADDR_SUFFIX      XPO_IPV6_LS32 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(4) XPO_COL_2X
             XPO_IPV6_LS32 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(5) XPO_COL_2X
             XPO_IPV6_H16 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(6) XPO_COL_2X ")"

With the port being:

... XPO_COL_1X "(" XPO_PORT_PATTERN ")"

#define XPO_PORT_PATTERN "[" XPO_DIG "]{1}|[" XPO_DIG_X(1) "]{1}["
XPO_DIG "]{1,4}"

The core XPO_IPV6_ADDR_PATTERN pattern is matching very well across
the full range of possible combinations. However, when I introduce the
possibility of square brackets "enclosing" the address, I get a
failure to match. Which pattern is:

#define XPO_UNA_N(U, N) "[" U "]{" #N "}"

#define XPO_SQUARE_BRACKET_L XPO_UNA_N("\\[", 1)
#define XPO_SQUARE_BRACKET_R XPO_UNA_N("\\]", 1)

#define XPO_SQUARE_BRACKETED(P) XPO_SQUARE_BRACKET_L P XPO_SQUARE_BRACKET_R

#define XPO_IPV6_ADDR_PORT_PATTERN
XPO_SQUARE_BRACKETED(XPO_IPV6_ADDR_PATTERN) XPO_COL_1X "("
XPO_PORT_PATTERN ")"

This fails straight out of the gate, even for this: "[::]:12345", for
instance. Even simplifying the pattern fails:

#define XPO_IPV6_SIMPLE_PORT_PATTERN XPO_SQUARE_BRACKET_L
"[^\\[^\\].]+" XPO_SQUARE_BRACKET_R XPO_COL_1X "(" XPO_PORT_PATTERN
")"

Reading a couple of blogs on SO leads me to believe that I may need to
"graduate" the approach I have been taking to a full fledged parser,
perhaps an LL(INF) such as Spirit. Key characteristics I am interested
in are not only parsing, but also extracting the address part, port,
and so on; which at least from prior experience with Spirit, I think
is possible?

I have worked with Spirit in the past, versions prior to X3. I haven't
looked closely at X3 yet; would it be possible to utilize this without
incurring many other in the way of dependencies from neighboring Boost
repos? In other words, the last thing I want is to incur a full Boost
dependency, if I can at all avoid doing so.

If I did follow a Spirit based approach, I would probably decompose
several micro grammars in much the same way that I decompose the
macros I use to build my regex patterns. I do not have a repo publicly
available, but I will share the header file I am using to build my
patterns for comment.

Thoughts?

Thanks, and cheers,

Michael Powell

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

_______________________________________________
Spirit-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/spirit-general
regex_elements.hpp (text/x-c++hdr, 6.6 KB) 
#ifndef KMSG_TRANSPORTS_REGEX_ELEMENTS_HPP
#define KMSG_TRANSPORTS_REGEX_ELEMENTS_HPP

#include <regex>

#define XPO_NON_CAP "?:"
#define XPO_ESC "\\"
#define XPO_DOT XPO_ESC "."
#define XPO_COL XPO_ESC ":"
#define XPO_DIG XPO_ESC "d"

#define XPO_UNA_N(U, N) "[" U "]{" #N "}"

#define XPO_SQUARE_BRACKET_L XPO_UNA_N("\\[", 1)
#define XPO_SQUARE_BRACKET_R XPO_UNA_N("\\]", 1)

#define XPO_COL_1X "[" XPO_COL "]{1}"
#define XPO_COL_2X "[" XPO_COL "]{2}"

/* If you feed the macros actual variable names and not literal constants, that can be a problem.
There is only so much the macros can do and remain useful as code generating substitutions. */
#define XPO_DIG_XY(x, y) #x "-" #y
#define XPO_DIG_X(x) XPO_DIG_XY(x, 9)
#define XPO_DIG_Y(y) XPO_DIG_XY(0, y)
#define XPO_LET XPO_DIG_XY(a, z) XPO_DIG_XY(A, Z)
#define XPO_LET_DIG XPO_LET XPO_DIG
#define XPO_LET_DIG_HYP XPO_LET_DIG "-"

// TODO: TBD: may need/want devise a pattern that comprehends ommission of leading (higher order) zeros:
#define XPO_HEX "[" XPO_DIG XPO_DIG_XY(a, f) XPO_DIG_XY(A, F) "]"
#define XPO_IPV6_H16 XPO_HEX "{1,4}"

#define XPO_SQUARE_BRACKETED(P) XPO_SQUARE_BRACKET_L P XPO_SQUARE_BRACKET_R

#define XPO_LABEL_PATTERN "[" XPO_LET "]{1}(" XPO_NON_CAP "[" XPO_LET_DIG_HYP "]+[" XPO_LET_DIG "]{1})?"

#define XPO_DEC_OCTET_PATTERN \
    "[" XPO_DIG "]|[" XPO_DIG_X(1) "][" XPO_DIG "]|[1][" XPO_DIG "]{2}" \
    "|[2][" XPO_DIG_XY(0, 4) "][" XPO_DIG "]|25[" XPO_DIG_XY(0, 5) "]"

#define XPO_PORT_PATTERN "[" XPO_DIG "]{1}|[" XPO_DIG_X(1) "]{1}[" XPO_DIG "]{1,4}"
//#define XPO_PORT_PATTERN "[" XPO_DIG "]{1}|[" XPO_DIG_X(1) "]{1}[" XPO_DIG "]{1,4}"

#define XPO_DOMAIN_ADDR_PATTERN "(" XPO_NON_CAP XPO_LABEL_PATTERN ")" \
"|(" XPO_NON_CAP "(" XPO_NON_CAP "(" XPO_NON_CAP XPO_LABEL_PATTERN ")[" XPO_DOT "])+(" XPO_NON_CAP XPO_LABEL_PATTERN "))"

#define XPO_DOMAIN_ADDR_PORT_PATTERN "(" XPO_DOMAIN_ADDR_PATTERN ")" XPO_COL_1X "(" XPO_PORT_PATTERN ")"

// TODO: TBD: perhaps I do not need ? or the group name <name> after all? or that Microsoft C++ regex does not support named groups as such?
// TODO: TBD: if that's desirable, consider Boost.Regex? can I find just that library?
#define XPO_DOMAIN_ADDR_PORT_GROUPS_PATTERN "(" XPO_DOMAIN_ADDR_PATTERN ")" XPO_COL_1X "(" XPO_PORT_PATTERN ")"

#define XPO_IPV4_ADDR_PATTERN "(" XPO_NON_CAP "(" XPO_NON_CAP XPO_DEC_OCTET_PATTERN ")[" XPO_DOT "]){3}(" XPO_NON_CAP XPO_DEC_OCTET_PATTERN ")"

// TODO: TBD: what is left to do I think is to capture some groups, or try to...
#define XPO_IPV4_ADDR_PORT_PATTERN "(" XPO_IPV4_ADDR_PATTERN ")" XPO_COL_1X "(" XPO_PORT_PATTERN ")"

#define XPO_IPV4_ADDR_PORT_FULL_PATTERN "((" XPO_NON_CAP XPO_DOMAIN_ADDR_PATTERN ")|(" XPO_NON_CAP XPO_IPV4_ADDR_PATTERN "))" XPO_COL_1X "(" XPO_PORT_PATTERN ")"

// Let's make an exception for avoiding regex punctuation such as brackets in this instance.
#define XPO_WHITESPACE "[" XPO_ESC "s]*"

#define XPO_IPV6_ADDR_PREFIX_0N(N) "(" XPO_NON_CAP "(" XPO_NON_CAP XPO_IPV6_H16 XPO_COL_1X "){0," #N "}" XPO_IPV6_H16 ")?"

#define XPO_IPV6_ADDR_SUFFIX "(" XPO_NON_CAP XPO_IPV6_H16 XPO_COL_1X ")"

#define XPO_IPV6_ADDR_SUFFIX_N(N) XPO_IPV6_ADDR_SUFFIX "{" #N "}"

#define XPO_IPV6_LS32 "(" XPO_NON_CAP "(" XPO_NON_CAP XPO_IPV6_H16 XPO_COL_1X XPO_IPV6_H16 ")|" XPO_IPV4_ADDR_PATTERN ")"

// TODO: TBD: greediness of the parser may be an issue? but we will start here...

/* Transliterated almost verbatim from the grammar:
http://tools.ietf.org/html/draft-main-ipaddr-text-rep-00#section-3.2
If I read the grammar correctly, what was described in the leading verbiage
is just a property of what is expressed in the grammar, I believe. That to
say, there is nothing special we need to do to comprehend "elision", for
example; it just is a natural byproduct of the grammar productions. So much
so that formatting herein is specific and follows the grammar very closely. */

/* TODO: TBD: possible future directions include:
1. Using the embedde IPv4 representation when IPv4-mapped addresses are desirable:
http://tools.ietf.org/html/draft-main-ipaddr-text-rep-00#section-4.2
*/

#define XPO_IPV6_ADDR_PATTERN "(" XPO_IPV6_ADDR_SUFFIX_N(6) XPO_IPV6_LS32 ")" \
                              "|(" XPO_NON_CAP XPO_COL_2X XPO_IPV6_ADDR_SUFFIX_N(5) XPO_IPV6_LS32 ")" \
        "|(" XPO_NON_CAP "(" XPO_IPV6_H16 ")?" XPO_COL_2X XPO_IPV6_ADDR_SUFFIX_N(4) XPO_IPV6_LS32 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(1) XPO_COL_2X XPO_IPV6_ADDR_SUFFIX_N(3) XPO_IPV6_LS32 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(2) XPO_COL_2X XPO_IPV6_ADDR_SUFFIX_N(2) XPO_IPV6_LS32 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(3) XPO_COL_2X XPO_IPV6_ADDR_SUFFIX      XPO_IPV6_LS32 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(4) XPO_COL_2X                           XPO_IPV6_LS32 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(5) XPO_COL_2X                           XPO_IPV6_H16 ")" \
   "|(" XPO_NON_CAP XPO_IPV6_ADDR_PREFIX_0N(6) XPO_COL_2X ")"

///* Unlike the full IPv4 patterns and composites leading up to this, we must capture the IPv6
//pattern here and now, so as not to confuse the brackets with the adsorbing production. */
//#define XPO_IPV6_ADDR_SQUARE_BRACKETED_PATTERN XPO_SQUARE_BRACKETED(XPO_IPV6_ADDR_PATTERN)

#define XPO_IPV6_SIMPLE_PORT_PATTERN XPO_SQUARE_BRACKET_L "[^\\[^\\].]+" XPO_SQUARE_BRACKET_R XPO_COL_1X "(" XPO_PORT_PATTERN ")"

#define XPO_IPV6_ADDR_PORT_PATTERN XPO_SQUARE_BRACKETED(XPO_IPV6_ADDR_PATTERN) XPO_COL_1X "(" XPO_PORT_PATTERN ")"

// TODO: TBD: does the square bracket ("[]") syntax count for both addresses? or just for IPv6? for now assuming IPv6 only...
// TODO: TBD: which makes matching the pattern for group extraction a bit tricky, as we want to group INSIDE the "[]"
// TODO: TBD: but yet we ALSO want to provide the DOMAIN group, so... we expect possibly 4 groups here:
// TODO: TBD: 1. the parent, 2. domain addr (optional), 3. IPv6 (optional), and 4. port
// TODO: TBD: in summary, we want to be careful NOT to capture inappropriate groups
#define XPO_IPV6_ADDR_PORT_FULL_PATTERN "(" XPO_NON_CAP "(" XPO_DOMAIN_ADDR_PATTERN ")" \
    "|(" XPO_NON_CAP XPO_SQUARE_BRACKETED(XPO_IPV6_ADDR_PATTERN) "))" XPO_COL_1X "(" XPO_PORT_PATTERN ")"

/*
TODO: TBD: enclose in (optional?) square brackets when adding port number?
https://tools.ietf.org/html/draft-main-ipaddr-text-rep-00#section-4.3
*/

/*
Essential Correction for IPv6 ABNF and URI Comparison in RFC 3261 (10/08)
http://www.ietf.org/rfc/rfc5954.txt
IP Version 6 Addressing Architecture (06/02)
http://tools.ietf.org/html/rfc4291
take aways include:
1. prefix length? i.e. addr/<prefix-length>
*/

#endif // KMSG_TRANSPORTS_REGEX_ELEMENTS_HPP
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.