Re: How to adapt an AST involving forward declared meta data

On Thu, Nov 1, 2018 at 10:05 AM Michael Powell <[email protected]> wrote:
>
> On Wed, Oct 31, 2018 at 10:15 PM Michael Powell <[email protected]> wrote:
> >
> > Hello,
> >
> > Like the subject says, I've got some structs in my AST that have to be
> > forward declared.
>
> Okay, I've worked through the basic AST concepts I think I need for
> this problem. However, the VS2017 C2079 error is not going away, not
> without modifying the approach:
>
> One possible way is to "introduce" pointers into the AST. Question
> along these lines, how good is Spirit Qi at working with AST,
> pointers, etc? Smart pointers preferred, I think, if possible.

I'm wondering if this possibly has something to do with recursive
descent, boost::recursive_wrappers, etc? Perhaps I should even be type
defining some of these variants, etc?

Another question popped out of my background reading, whether I should
be aiming for X3; but that's a whole other can of worms I think. I'm
trying to keep the AST as minimalist as I possibly can.

I can try to pull together another web compiler example around  this,
but the scope has increased beyond the "simple" version parser,
obviously.

Thoughts?

> struct bool_t {
>     std::string val;
> };
>
> struct str_t {
>     std::string quoted_text;
> };
>
> struct full_id_t {
>     std::string full_id;
> };
>
> struct int_t {
>     std::string val;
> };
>
> struct float_t {
>     std::string val;
> };
>
> // TODO: TBD: we may need/want to dissect this one still further...
> i.e. to ident, message/enum-name, etc.
> struct element_type_t {
>     std::string name;
> };
>
> // TODO: TBD: let's not get too fancy with the inheritance, ...
> // TODO: TBD: however, scanning the other types, we could potentially
> do more of it, strategically, here and there
> struct msg_type_t : element_type_t {};
> struct enum_type_t : element_type_t {};
>
> struct package_t {
>     std::string full_id;
> };
>
> struct const_t {
>     std::variant<full_id_t, int_t, float_t, str_t, bool_t> val;
> };
>
> struct syntax_t {
>     std::string val;
> };
>
> struct import_modifier_t {
>     std::string val;
> };
>
> struct import_t {
>     std::optional<import_modifier_t> mod;
>     std::string target_name;
> };
>
> struct option_t {
>     std::string name;
>     const_t val;
> };
>
> struct label_t {
>     std::string val;
> };
>
> struct type_t {
>     std::variant<std::string, msg_type_t, enum_type_t> val;
> };
>
> // TODO: TBD: could potentially get more meta-dissected based on the
> specification:
> struct field_opt_t {
>     std::string name;
>     const_t val;
> };
>
> struct field_t {
>     label_t label;
>     type_t type;
>     std::string name;
>     int number;
>     std::vector<field_opt_t> opts;
> };
>
> // TODO: TBD: msg_body_t must be forward declared ...
> struct msg_body_t;
>
> struct group_t {
>     label_t label;
>     std::string name;
>     int number;
>     msg_body_t body;
> };
>
> struct oneof_field_t {
>     type_t type;
>     std::string name;
>     int number;
>     std::optional<std::vector<field_opt_t>> opts;
> };
>
> struct oneof_t {
>     std::string name;
>     std::vector<oneof_field_t> choices;
> };
>
> struct key_type_t {
>     std::string val;
> };
>
> struct map_field_t {
>     key_type_t key_type;
>     type_t type;
>     std::string name;
>     int number;
>     std::optional<std::vector<field_opt_t>> opts;
> };
>
> struct range_t {
>     int min;
>     std::optional<std::variant<int, std::string>> max;
> };
>
> struct extensions_t {
>     std::vector<range_t> ranges;
> };
>
> struct reserved_t {
>     std::variant<std::vector<range_t>, std::vector<std::string>> val;
> };
>
> struct enum_val_opt_t {
>     std::string name;
>     const_t val;
> };
>
> struct enum_field_t {
>     std::string name;
>     int ordinal;
>     std::optional<std::vector<enum_val_opt_t>> opt;
> };
>
> struct enum_body_t {
>     std::vector<std::variant<option_t, enum_field_t>> items;
> };
>
> struct enum_t {
>     std::string name;
>     enum_body_t body;
> };
>
> struct msg_t {
>     std::string name;
>     // TODO: TBD: msg_body_t must be forward declared ...
>     msg_body_t body;
> };
>
> struct msg_body_t {
>     // TODO: TBD: which references msg_t and group_t ...
>     std::vector<std::variant<field_t, enum_t, msg_t, extensions_t,
> group_t, option_t, oneof_t, map_field_t, reserved_t>> content;
> };
>
> struct extend_t {
>     std::string msg_type;
>     std::vector<std::variant<field_t, group_t>> content;
> };
>
> struct top_level_def_t {
>     std::variant<msg_t, enum_t, extend_t> content;
> };
>
> struct proto_t {
>     syntax_t syntax;
>     std::vector<std::variant<import_t, package_t, option_t,
> top_level_def_t>> content;
> };
>
> And the Fusion adapted structs:
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::msg_type_t, name)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::enum_type_t, name)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::bool_t, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::str_t, quoted_text)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::full_id_t, full_id)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::int_t, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::float_t, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::package_t, full_id)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::const_t, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::syntax_t, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::import_modifier_t, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::import_t, mod, target_name)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::label_t, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::type_t, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::field_opt_t, name, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::field_t, label, type, name,
> number, opts)
>
> // TODO: TBD: trying to resolve the C2079 forward declaration issue...
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::group_t, label, name, number, body)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::oneof_field_t, type, name,
> number, opts)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::oneof_t, name, choices)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::key_type_t, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::map_field_t, key_type, type,
> name, number, opts)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::range_t, min, max)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::extensions_t, ranges)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::reserved_t, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::enum_val_opt_t, name, val)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::enum_field_t, name, ordinal, opt)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::enum_body_t, items)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::enum_t, name, body)
>
> // TODO: TBD: ditto group_t, solving the C2079 forward declaration issue...
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::msg_t, name, body)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::msg_body_t, content)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::extend_t, msg_type, content)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::top_level_def_t, content)
>
> BOOST_FUSION_ADAPT_STRUCT(NS_PROTO_AST_H::proto_t, syntax, content)
>
> > // Must forward declare for Group to be happy as a struct.
> > struct msg_body_t;
> >
> > struct group_t {
> >     label_t label;
> >     std::string name;
> >     int number;
> >     msg_body_t body;
> > };
> >
> > struct msg_body_t {
> >     // TODO: TBD: ...
> > };
> >
> > Specifically, Group can contain a Message Body.
> >
> > group = label "group" groupName "=" fieldNumber messageBody
> >
> > However, Message Body can also reference a Group.
> >
> > messageBody = "{" { field | enum | message | extend | extensions | group
> >     | option | oneof | mapField | reserved | emptyStatement } "}"
> >
> > In terms of the C++ AST itself, I do not know of a way to model this
> > and at least <messageBody/> not be forward declared.
> >
> > However, Spirit struct adaptation is balking at this. Literally,
> >
> > 1>g:\source\kingdom
> > software\kingdom.ortools\standard\src\kingdom.ortools.sat.params.generator\proto_ast.h(100):
> > error C2079: 'kingdom::google::protobuf::proto2::ast::group_t::body'
> > uses undefined struct
> > 'kingdom::google::protobuf::proto2::ast::msg_body_t'
> >
> > I have not completely worked through the AST, but I am getting close.
> > So I am cautiously optimistic that once I cross the <messageBody/>
> > bridge, this error will go away.
> >
> > However, in the meantime, I thought I'd present the question, whether
> > forward declared AST posed any difficulties to Spirit Qi struct
> > adaptation to anyone's knowledge.
> >
> > Best regards,
> >
> > Michael Powell
> >
> > https://developers.google.com/protocol-buffers/docs/reference/proto2-spec