Re: IPv6 address parsing

On Sat, Dec 16, 2017 at 6:27 PM, Michael Powell <[email protected]> wrote:
> On Sat, Dec 16, 2017 at 1:08 PM, Michael Powell <[email protected]> wrote:
>> On Fri, Dec 15, 2017 at 1:59 PM, Michael Powell <[email protected]> wrote:
>>> On Fri, Dec 15, 2017 at 12:22 PM, Michael Powell <[email protected]> wrote:
>>>> On Fri, Dec 15, 2017 at 11:31 AM, Michael Powell <[email protected]> wrote:
>>>>> On Thu, Dec 7, 2017 at 1:21 PM, Michael Powell <[email protected]> wrote:
>>>>>> Again, I am moving beyond IPv4 into IPv6. Do I need to re-post that
>>>>>> grammar attempt? Maybe I do at this point.
>>>>>>
>>>>>> _dot = ".";
>>>>>> _col = char_(':');
>>>>>> _col_x2 = _col >> _col;
>>>>>>
>>>>>> // _dec_octet = uint_parser<std::uint8_t, 10, 1, 3>();
>>>>>>
>>>>>> qi::rule<Iterator> _dec_octet = ... ; // Approach TBD; probably
>>>>>> running with the IPv4 proposal discussed in earlier thread(s).
>>>>>>
>>>>>> qi::rule<Iterator> _ipv4_addr = _dec_octet >> repeat(3)[_dot >> _dec_octet];
>>>>>> // or, qi::rule<Iterator> _ipv4_addr = raw[_dec_octet >> _dot >>
>>>>>> _dec_octet >> _dot >> _dec_octet >> _dot >> _dec_octet];
>>>>>>
>>>>>> qi::rule<Iterator> _h16 = raw[repeat(1, 4)[hex]];
>>>>>> // Or (?): _h16 = hex >> -(hex >> -(hex >> -hex));
>>>>>> qi::rule<Iterator> _h16_col = _h16 >> _col;
>>>>>> qi::rule<Iterator> _ls32 = _h16_col >> _h16 | _ipv4_addr;
>>>>>>
>>>>>> qi::rule<Iterator, std::string()> _ipv6_addr
>>>>>>     = repeat(6)[_h16_col] >> _ls32
>>>>>>     | _col_x2 >> repeat(5)[_h16_col] >> _ls32
>>>>>>     | -(_h16) >> _col_x2 >> repeat(4)[_h16_col] >> _ls32
>>>>>>     | -(_h16_col >> _h16) >> _col_x2 >> repeat(3)[_h16_col] >> _ls32
>>>>>>     | -(repeat(1, 2)[_h16_col] >> _h16) >> _col_x2 >>
>>>>>> repeat(2)[_h16_col] >> _ls32
>>>>>>     | -(repeat(1, 3)[_h16_col] >> _h16) >> _col_x2 >> _h16_col >> _ls32
>>>>>>     | -(repeat(1, 4)[_h16_col] >> _h16) >> _col_x2 >> _ls32
>>>>>>     | -(repeat(1, 5)[_h16_col] >> _h16) >> _col_x2 >> _h16
>>>>>>     | -(repeat(1, 6)[_h16_col] >> _h16) >> _col_x2
>>>>>>     ;
>>>>>
>>>>> I am testing the grammar now, and it seems like to me that the
>>>>> optional and/or repeat are not working correctly.
>>>>>
>>>>> For test case: prefix = pf_max_4|pf_count_1, suffix = pf_none,
>>>>> appendix = bo_hex_2, which yields address: "6a80:9024::c7e3:4150".
>>>>>
>>>>> Should be matching the alternative, from the RFC grammar:
>>>>>   / [*4(h16 ":") h16] "::"            ls32
>>>>>
>>>>> Or from the Qi grammar:
>>>>>      | -(repeat(1, 4)[_h16_col] >> _h16) >> _col_x2 >> _ls32
>>>>>
>>>>> But gets rejected. Did not parse, or rejected the entire string.
>>>>>
>>>>> Perhaps repeat does not work quite as advertised?
>>>>>
>>>>> Thoughts? Suggestions?
>>>>
>>>>
>>>> qi::rule<Iterator, std::string()> _ipv6_addr
>>>>     = repeat(6)[_h16_col] >> _ls32
>>>>     | _col_x2 >> repeat(5)[_h16_col] >> _ls32
>>>>     | -(-_h16) >> _col_x2 >> repeat(4)[_h16_col] >> _ls32
>>>>     | -(-_h16_col >> _h16) >> _col_x2 >> repeat(3)[_h16_col] >> _ls32
>>>>     | -(-repeat(1, 2)[_h16_col] >> _h16) >> _col_x2 >>
>>>> repeat(2)[_h16_col] >> _ls32
>>>>     | -(-repeat(1, 3)[_h16_col] >> _h16) >> _col_x2 >> _h16_col >> _ls32
>>>>     | -(-repeat(1, 4)[_h16_col] >> _h16) >> _col_x2 >> _ls32
>>>>     | -(-repeat(1, 5)[_h16_col] >> _h16) >> _col_x2 >> _h16
>>>>     | -(-repeat(1, 6)[_h16_col] >> _h16) >> _col_x2
>>>>     ;
>>>>
>>>> I did identify that possibly the prefix phrase needs to be built from
>>>> optional elements within itself.
>>>>
>>>> However, still not parsing.
>>>>
>>>> I am concerned that either Qi optional, repeat, etc, perhaps even
>>>> alternatives, are not working quite right in this instance.
>>>>
>>>> Do I need to separate the alternatives here into "micro" grammars
>>>> within themselves for this to possibly work?
>>>
>>> Thus far, separating the more complex grammar into micro-grammars
>>> seems to be the correct answer, the one that will work.
>>
>> I removed the repeats, and I think I've isolated an issue that nested
>> optional operators do not work. i.e. -(-(-(...) ...) ...) >> ...
>>
>> Which does not match the phrase I expect it to.
>
> Moving on, a little...
>
> I now have a test case "8d5d:5be71c85:7c0e:d86f:540b:d0ec:f177:2316"
> which should not be valid. I expect it to be invalid and rejected.
> However, the grammar
>
> _col = char_(':');
>
> _h16 = raw[
>   hex >> hex >> hex >> hex
>     | hex >> hex >> hex
>     | hex >> hex
>     | hex
>   ];
> //// I was using repeat(...), but I have concerns whether REPEAT
> (and/or OPTIONAL) are working properly, not least of all in this
> context, never mind concerns over nested repeat, optional, etc.
> //_h16 = repeat(1, 4)[hex];
>
> _h16_col = _h16 >> _col;
>
> _ls32 = _h16_col >> _h16;
>
> _ipv6_addr = lexeme[
>     _h16_col >> _h16_col >> _h16_col >> _h16_col >> _h16_col >>
> _h16_col >> _ls32
> ];
>
> _start = _ipv6_addr >> eoi;
>
> Which leads me to believe that boost::spirit::hex
> (boost::spirit::hex_type) is excessively greedy. Would it be more
> appropriate if I simply said this: char_("0123456789abcdef"),
> notwithstanding case?
>
> I am also chewing on the test case parameters, whether I should drive
> the test data to yield "8d5d:5be7::1c85:7c0e:d86f:540b:d0ec:f177:2316"
> instead, but this is beyond the scope of my grammar concerns.
>
> Any thoughts? Insights?


Instead of "hex", which appears to be too greedy, even after calling
for either 1) repeat(1,4)[hex], or spelling it out in sequenced
alternatives, perhaps a raw[uint_parser<uint8_t, 16, 1, 4>()] would be
better?

I don't care about parsing the integer out of it, only that it was a
1-4 hex characters. No more than 4, however.

> Thanks!
>
>>> Not sure why the broader parent grammar doesn't work, however. Apart
>>> from "greediness" suspicions, that is; the grammar is what it is, no?
>>>
>>>>> Thank you!
>>>>>
>>>>>> qi::rule<Iterator> _start = _ipv6_addr >> eoi;
>>>>>>
>>>>>> Q: Notwithstanding concern over raw IPv4, does the overall Qi-based
>>>>>> IPv6 grammar look reasonable? If not, why not?
>>>>>>
>>>>>> I'd like to land in a Fusion data structure something like (string >>
>>>>>> uint16_t >> eoi), but if I can land here, that's acceptable, too:
>>>>>> (string >> -(uint8_t >> uint8_t >>
>>>>>> uint8_t >> uint8_t) >> uint16_t >> eoi).
>>>>>>
>>>>>> From what I can determine, then, raw[...] is the difference between
>>>>>> parsed types like uint8_t and string?
>>>>>>
>>>>>> ref: http://www.ietf.org/rfc/rfc5954.txt, section 4.1 Resolution for
>>>>>> Extra Colon in IPv4-Mapped IPv6 Address
>>>>>>
>>>>>> Constructive feedback appreciated.
>>>>>>
>>>>>> Untested ATM, to be fair.
>>>>>>
>>>>>> On Thu, Dec 7, 2017 at 1:09 PM, Seth <[email protected]> wrote:
>>>>>>> On 07-12-17 18:21, Michael Powell wrote:
>>>>>>>> Well, you remembered incorrectly.
>>>>>>>
>>>>>>> I was asking.
>>>>>>>
>>>>>>>> If there's a doable intermediate step of parsed individual octets, so
>>>>>>>> be it; no problem.
>>>>>>>
>>>>>>> Please per-use all the parts of my answer.
>>>>>>>
>>>>>>> On 07-12-17 17:13, Seth wrote:
>>>>>>>> Maybe you'll be more convinced when it's demonstrated (I think I did
>>>>>>>> this before):
>>>>>>>>
>>>>>>>> https://wandbox.org/permlink/M12U5PEYcB8vnxIs
>>>>>>>>
>>>>>>>> It's really just ~3 lines of code changed:
>>>>>>>>
>>>>>>>>     using ipv4_address = std::string; // much less useful representation
>>>>>>>>     // ....
>>>>>>>>     addr_  = raw[ octet_ >> '.' >> octet_ >> '.' >> octet_ >> '.' >>
>>>>>>>> octet_ ];
>>>>>>>>
>>>>>>>> Note now that test fails:
>>>>>>>
>>>>>>> It's all there.
>>>>>>>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot