Re: Exposing some type details in the introspection XML?

Hi

On Mon, Jul 27, 2020 at 10:35 AM Matthew Bromley-Barratt <
[email protected]> wrote:

> > >
> > > > And I mean:
> > >> >
> a(sa(sa(sa(sgya{sv})a{sv})a(sa(sga{sv})a{sv})a(sgya{sv})a{sv})a{sv}) ?
> > >> > that's just so delicious! Just think about how much more awesome
> > >> > this signature string could be if there were structures.
> > >>
> > >> Somewhere, something went horribly wrong...
> > >>
> > >
> > > Feels like quite an intimidating task to define a new IDL for DBus.
> > > And I am not convinced the various bindings / code generators will
> > > follow, because parsing it will be an additional pain, whereas XML,
> > > well, it's there and it's easy to add new attributes/elements for
> > > existing code & interfaces. Perhaps the "delicious" giant a{sv} is
> > > indeed a better proposal, since it fits nicely. But to me it's just a
> > > dbus marshalled version of the XML, and we are back to similar
> > > problems/considerations regarding the schema. It may be even less
> > > convenient, less flexible, beside being humanly unreadable, and pretty
> > shallow.
> > >
> > > In the past (commit f3549401113b "spec: Allow <annotation> in <arg>
> > > elements in introspection XML", or commit dc12fac5f8a36d), we have
> > > modified the introspection schema slightly, without bumping version
> > > etc. Did it bother anyone? Can we further add elements or attributes
> > > here and there, without touching the existing ones?
> > >
> > > Although I can't find it explicitly in the spec, I suppose the
> > > introspection XML is free to introduce extra/new xml namespaces
> > > already in a compatible manner, correct?
> > >
> > > Clearly, it would be inconvenient if both client & servers would have
> > > to handle different introspection formats/versions in the future.
> > >
> > >
> > So I toyed with this idea a bit in zbus, and would like to get some
> feedback
> > before I go further. Eventually, I can either try to implement the
> proposal in
> > sd-bus or gdbus. I would like to invite the maintainers of go and python
> > bindings to give feedback, and eventually give a try at implementing it,
> if they
> > are interested.
> >
> > As we can't extend the current XML without risk of breaking parsers, and
> we
> > would have to open the door to other formats eventually (custom binding
> > formats too, perhaps), let's start with a new Introspect() method:
> >
> >   <interface name="org.freedesktop.DBus.Introspectable2">
> >     <method name="Introspect">
> >       <arg name="format" type="s" direction="in"/>
> >       <arg name="options" type="a{sv}" direction="in"/>
> >       <arg type="ay" direction="out"/>
> >     </method>
> >   </interface>
> >
> > That should be straightforward, a new interface Introspectable2 is added,
> > with a new method to introspect, that takes a format as a string (for
> example
> > "dbus.v1"), our idiomatic "a{sv}" dict for options (I think we could have
> > common options, such as "dbus.depth", "dbus.only-nodes" etc, and format
> > specific options with for example "fmt.comments").
> >
> > The returned value is changed to "ay", so it can return anything
> (although
> > most likely utf8 strings).
> >
> > I would like to extend the current XML, let's call it "dbus.v2" format
> for
> > example for the above method, or v2 in short. The goal is that a v2
> parser
> > should be compatible with current v1 XML documents. It should also be
> fairly
> > easy to extend a v1 parser to support v2. Or to generate v2 progressively
> > from v1. In other words, I tried to extend the v1 format in some
> compatible
> > manners, so both v1 and v2 can coexist and share code.
> >
> > - Comments
> >
> > I propose to have <!-- --> comments to be associated with the next
> element.
> > Ex:
> >
> >   <!-- This D-Bus interface is implemented by the
> >        /org/freedesktop/PolicyKit1/Authority object on the well-known
> name
> >        org.freedesktop.PolicyKit1 on the system message bus. -->
> >   <interface name="org.freedesktop.PolicyKit1.Authority"
> >
> > We could standardize on the markdown syntax too.
> >
> > - New elements
> >
> > Since a well-formed XML document should have only one top-level element,
> > I propose to allow various new type elements to the top-level <node/>.
> > Alternatively, we could have a new <introspect> root element, and have a
> > single child <node> there.
> >
> > - New type names
> >
> > Some proposed elements introduce new type names. I propose to change a
> > bit the way the "type" attributes are usually parsed and interpreted. If
> the
> > type starts with a capital, then the alphanumeric characters that follow
> form
> > a type name (that references a type element). Additionally, "[T]" is
> added for
> > array types. No dictionary syntax is necessary. Complex types can be
> > represented with <structure> types.
> >
> > - <error> element
> >
> > This element declares an error domain. It must have a domain name, and a
> > type, and list the various errors as childrens. Ex:
> >
> >   <!-- Errors that can be returned by various method calls. -->
> >   <error domain="org.freedesktop.PolicyKit1.Error" type="s">
> >     <item name="Failed"/>
> >     <item name="Cancelled"/>
> >     <item name="NotSupported"/>
> >     <item name="NotAuthorized"/>
> >     <item name="CancellationIdNotUnique"/>
> >   </error>
> >
> > the <interface> gains a new attribute
> > error="org.freedesktop.PolicyKit1.Error" to associate the interface
> methods
> > with the error domain.
> >
> > - <enum> type element
> >
> > Declares an enumeration. It must have a name and a basic type attribute.
> > Each item has an optional name and a value. (the name is optional, it
> can be
> > generated by the binding, and we can have enum of strings without
> > repeating name/value), ex:
> >
> >   <!-- An enumeration for granting implicit authorizations -->
> >   <enum name="ImplicitAuthorization" type="u">
> >     <item name="NotAuthorized" value="0"/>
> >     <item name="AuthenticationRequired" value="1"/>
> >     <item name="AdministratorAuthenticationRequired" value="2"/>
> >     <item name="AuthenticationRequiredRetained" value="3"/>
> >     <item name="AdministratorAuthenticationRequiredRetained" value="4"/>
> >     <item name="Authorized" value="5"/>
> >   </enum>
> >
> >   <!-- The type of the subject. -->
> >   <enum name="SubjectKind" type="s">
> >     <item value="unix-process"/>
> >     <item value="unix-session"/>
> >     <item value="system-bus-name"/>
> >   </enum>
> >
> > - <dictenum> type element
> >
> > Declares a dictionary, by listing the well-known keys (and if the set of
> > accepted keys is strict), and associated types. If the key type is "v",
> the items
> > should have an associated type too.
> >
> > Ex:
> >
> >   <!-- Somebody's details. -->
> >   <dictenum name="PersonDetails" type="ss">
> >     <item value="firstname"/>
> >     <item value="lastname"/>
> >   </dictenum>
> >
> >   <!-- Details about the identity. -->
> >   <dictenum name="IdentityDetails" type="sv">
> >     <item value="uid" type="u"/>
> >     <item value="gid" type="u"/>
> >   </dictenum>
> >
> > - <struct> type element
> >
> > Declares a struct, with field name and type. Ex:
> >
> >   <!-- This struct describes a temporary authorization. -->
> >   <struct name="TemporaryAuthorization">
> >     <item name="id" type="s"/>
> >     <item name="action_id" type="s"/>
> >     <item name="subject" type="Subject"/>
> >     <item name="time_obtained" type="t"/>
> >     <item name="time_expires" type="t"/>
> >   </struct>
> >
> > With the above, it becomes possible to generate much nicer high-level
> > bindings. With zbus, we go from v1
> > https://gitlab.freedesktop.org/snippets/1117 to
> > https://gitlab.freedesktop.org/snippets/1118 (the v2 XML:
> > https://gitlab.freedesktop.org/snippets/1119).
> >
> > What do you think?
> >
> > thanks
> >
> > --
> > Marc-Andr? Lureau
>
> I certainly see value in dBus supporting the things you've suggested,
> (structs, etc), but I can also see long term value in implementing them by
> adopting something else instead of extending the existing IDL. I can't help
> but feel that, the more the existing IDL schema language and tooling for
> dBus is evolved, the more it will resemble some other existing standard and
> its associated tooling, whilst not being able to benefit from them.
>
> At some point in the future it is possible that it will become widely
> recognised that a "big bang" is required, that a switch to some existing
> standard is imperative. If and when that happens, that would be quite a lot
> of work. However, it may be substantially less work overall than had
> already gone into patching up the existing IDL and code generator over the
> years between now and then. So there may be some benefit in thinking what
> the long term feature set for dBus needs to be, and considering whether or
> not a big bang now would be the shortest path to delivering those.
>
> As to what that standard could be, well, there's not many really good ones
> to pick from. One candidate I feel is Google protocol buffers. dBus could
> offer the use of Google Protocol Buffers as an alternative to the current
> IDL / marshalling, providing a standard .proto defining anything that dBus
> needed to be standardised. A lot of what is being discussed could be easily
> covered off by expanding that standard .proto file, with a minimal of code
> changes required to make use of that. There's already a huge number of code
> generators for every language under the sun (one of gpb's big strengths),
> and any improvements and additions directly benefit dBus. In other regards
> gbp is only just suitable - dBus would be depending using the oneof message
> type to be able to receive typed messages.
>
> Are there any spare bits in the existing interface to allow a dBus object
> to support 2 different marshalling formats (e.g. the existing one and, say,
> GPB)? That would allow for a period of transition.
>

Thanks for the feedback! Although there is a high-level overlap between
different IPCs, replacing DBus with grpc for both the IDL and wire format
is just completely out of scope here :)

Rather, I would like to address a fairly obvious & simple shortcoming of
the current XML: the lack of type details. It's not like replacing it with
something else. It's an evolution.

Fwiw, I don't know protobuf in detail, but I think we would have a hard
time trying to retrofit it in DBus. Unless someone deeply familiar with
both IPCs steps up with a strong proposal, I don't think it will ever
happen. (honestly, I am not sure it makes sense when you start looking at
details)


> Some time ago I floated the idea of extending the existing IDL with value
> and size constraints, generating little enthusiasm! But if there were a
> move to add to the IDL, rather than switch to something else, it would be
> excellent to be able to introduce those too. Unfortunately, GPB doesn't
> allow for size and value constraints, but I'm about to suggest that to that
> community, see if they like it.
>

I think the proposal above covers some of it, with the <enum/> elements
that should list acceptable values (strictly or not, with a 'strict'
attribute).

-- 
Marc-André Lureau

_______________________________________________
dbus mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dbus