Re: Reflection in game engine (c++)
Adrian Stephens <[email protected]> Mon, 18 Mar 2013 16:30:45 -0700
| Newsgroups | gmane.games.devel.sweng |
|---|---|
| Message-ID | <[email protected]> |
--===============1403331153==
Content-Type: multipart/alternative; boundary="Apple-Mail=_47BE95EA-B4BE-4E24-B7B2-90BF2251051B"
--Apple-Mail=_47BE95EA-B4BE-4E24-B7B2-90BF2251051B
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=iso-8859-1
I'm glad we're still talking about this - I think it's important and I =
wanted to add my own observations:
The macro/template approach to reflection only applies to one side of =
the data/code interface. If all you need to do is serialize runtime =
data to be read back in by runtime code, then you're ok; but if you want =
to access that metadata from an external tool, you either have to =
recompile the tool or plugin when anything changes, or write something =
that parses that code.
If you're going to have to write a parser anyway, it seems to me you =
might as well use it for both code and data views. Which means you can =
use a more natural syntax for your structure definitions that even =
non-programmers can use.
To map this back into your C++ code, you can pretty trivially write out =
header files containing typedefs that correspond to the metadata, and =
generate whatever you need to allow the runtime to interpret serialized =
data. This also address the separation of code and assets: we don't =
want artists having to sync the code, and this keeps the data =
definitions with the assets that they define.
This is what I do, anyway. I have a data language, which defines types, =
and which can also be used to define data, and all data is stored using =
these definitions. This includes bitmaps, sounds, models, etc. The =
data language is separated from the data representation so this doesn't =
imply that bitmaps are stored in a script-like array of colors - but it =
does mean that tools can use the reflection system on more classes of =
data than just small gameplay entities. And when distinctions like that =
are removed, and all things are treated the same way, lots of clever =
things just fall out.
Some other random thoughts:
As far as I can see, all C++ needs to allow full reflection is some sort =
of fieldsof(T) operator that enumerates the fields of a structure. e.g. =
(typed straight into email, so probably bollocks - and not =
representative of my actual system!)
struct metadata {
enum TYPE {INT, PTR, ARRAY, STRUCT} type;
metadata(TYPE &_type) : type(_type) {}
};
template<typename A, typename... B> struct md_fields : md<A>, =
md_fields<B> {};
template<typename T> struct md : metadata, md_fields<fieldsof(T)> { md() =
: metadata(STRUCT) {} };
template<typename T> struct md<T*> : metadata { md<T> t; md() : =
metadata(PTR) {} };
template<typename T, int N> struct md<T[N]> : metadata { int n; md<T> t; =
md() : metadata(ARRAY), n(N) {} };
template<> struct md<int> : metadata { md() : metadata(INT) {} };
template<typename T> metadata *get_md(const T &t) { static md<T> m; =
return &m; }
Adrian Stephens
On Mar 18, 2013, at 10:45 AM, Douglas Cox wrote:
> That sounds pretty similar to the goals of the engine that we're =
working on -- basically faster iteration times for everyone (this =
includes engine programmers as well).
>=20
> All of our code (including the compiler) is written in a C#-like =
language, so we have perfect reflection from that. Attributes are =
pretty much required for providing additional information to property =
editors for things like valid ranges of a field or notifications of a =
property change (most of which can go away in Final builds).
>=20
> Another thing to consider may be allowing for data derivation. This =
can save quite a bit of work when the need for copies of objects come up =
that only need one or two properties changed. If you pick a =
serialization format for the editable data that knows about the data =
(fieldtype + fieldname + value), it makes it a lot easier to get this =
working.
>=20
> In the past, we've used Macros and they worked fine. They're a bit =
more annoying to extend with optional parameters and you have to repeat =
the field/property name, but that's not terrible since you can generally =
get it down to compile-time errors if you get something wrong (other =
than forgetting to add a new field).
>=20
> -Doug
>=20
>=20
>=20
> On Fri, Mar 15, 2013 at 3:45 AM, Tinco Andringa <[email protected]> wrote:
> I think the point of the discussion is to not just assume C++ isn't
> going to do it. As far as I understand the goal is to have the ability
> to develop tools that aid programmers and artist in iterating rapidly.
> This means for artists that they can swap artwork in at runtime or
> quickly boot the game from a serialized state with new artwork, not
> just on his/her own computer but also over a network to perhaps a
> console. For programmers this may mean that they can view the game
> state easily, launch the game in various states easily, swap in/out
> scripts easily, maybe even change code without fully recompiling or
> even restarting the game.
>=20
> Am I missing something important?
>=20
> On Fri, Mar 15, 2013 at 8:33 AM, Massimo Del Zotto =
<[email protected]> wrote:
> > I have limited experience but I have been very impressed by the =
separation
> > provided by scripting languages.
> > Considering the goals:
> >
> > My goal is to have fast iteration times, both artists and =
programmers
> >
> > It appears to me C++ isn't going to do it. Especially on the artist =
side of
> > the thing.
> >
> > By the way, I would like to have an insight on how this reflection
> > information is to be used. If it is used to simulate duck typing I'd =
use a
> > tool which supports duck typing natively.
> >
> > Massimo
> >
> > _______________________________________________
> > Sweng-Gamedev mailing list
> > [email protected]
> > =
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.co=
m
> >
> _______________________________________________
> Sweng-Gamedev mailing list
> [email protected]
> =
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.co=
m
>=20
> _______________________________________________
> Sweng-Gamedev mailing list
> [email protected]
> =
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.co=
m
--Apple-Mail=_47BE95EA-B4BE-4E24-B7B2-90BF2251051B
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=iso-8859-1
<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div><br></div><div>I'm glad we're still talking about this - I think =
it's important and I wanted to add my own =
observations:</div><div><br></div><div>The macro/template approach to =
reflection only applies to one side of the data/code interface. If =
all you need to do is serialize runtime data to be read back in by =
runtime code, then you're ok; but if you want to access that metadata =
from an external tool, you either have to recompile the tool or plugin =
when anything changes, or write something that parses that =
code.</div><div><br></div><div>If you're going to have to write a parser =
anyway, it seems to me you might as well use it for both code and =
data views. Which means you can use a more natural syntax for your =
structure definitions that even non-programmers can =
use.</div><div><br></div><div>To map this back into your C++ code, you =
can pretty trivially write out header files containing typedefs that =
correspond to the metadata, and generate whatever you need to allow the =
runtime to interpret serialized data. This also address =
the separation of code and assets: we don't want artists having to =
sync the code, and this keeps the data definitions with the assets that =
they define.</div><div><br></div><div>This is what I do, anyway. I =
have a data language, which defines types, and which can also be used to =
define data, and all data is stored using these definitions. This =
includes bitmaps, sounds, models, etc. The data language is =
separated from the data representation so this doesn't imply that =
bitmaps are stored in a script-like array of colors - but it does mean =
that tools can use the reflection system on more classes of data than =
just small gameplay entities. And when distinctions like that are =
removed, and all things are treated the same way, lots of clever things =
just fall out.</div><div><br></div><div>Some other random =
thoughts:</div><div>As far as I can see, all C++ needs to allow full =
reflection is some sort of fieldsof(T) operator that enumerates the =
fields of a structure. e.g. (typed straight into email, so =
probably bollocks - and not representative of my actual =
system!)</div><div><br></div><div>struct metadata {</div><div><span =
class=3D"Apple-tab-span" style=3D"white-space:pre"> </span>enum TYPE =
{INT, PTR, ARRAY, STRUCT} type;</div><div><span class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span>metadata(TYPE &_type) : =
type(_type) {}</div><div>};</div><div>template<typename A, =
typename... B> struct md_fields : md<A>, md_fields<B> =
{};</div><div>template<typename T> struct md =
: metadata, md_fields<fieldsof(T)> { md() : =
metadata(STRUCT) {} };</div><div>template<typename T> struct =
md<T*> : metadata { md<T> t; md() : metadata(PTR) =
{} };</div><div>template<typename T, int N> struct md<T[N]> =
: metadata { int n; md<T> t; md() : metadata(ARRAY), n(N) {} =
};</div><div>template<> struct md<int> : metadata { md() : =
metadata(INT) {} };</div><div><br></div><div>template<typename T> =
metadata *get_md(const T &t) { static md<T> m; return &m; =
}</div><div><br></div><div><br></div><div><br></div><div>Adrian =
Stephens</div><br><div><div>On Mar 18, 2013, at 10:45 AM, Douglas Cox =
wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite"><div dir=3D"ltr">That sounds pretty similar to the goals =
of the engine that we're working on -- basically faster iteration times =
for everyone (this includes engine programmers as =
well).<div><br></div><div>All of our code (including the compiler) is =
written in a C#-like language, so we have perfect reflection from that. =
Attributes are pretty much required for providing additional =
information to property editors for things like valid ranges of a field =
or notifications of a property change (most of which can go away in =
Final builds).</div>
<div><br></div><div style=3D"">Another thing to consider may be allowing =
for data derivation. This can save quite a bit of work when the =
need for copies of objects come up that only need one or two properties =
changed. If you pick a serialization format for the editable data =
that knows about the data (fieldtype + fieldname + value), it makes it a =
lot easier to get this working.</div>
<div style=3D""><br></div><div style=3D"">In the past, we've used Macros =
and they worked fine. They're a bit more annoying to extend with =
optional parameters and you have to repeat the field/property name, but =
that's not terrible since you can generally get it down to compile-time =
errors if you get something wrong (other than forgetting to add a new =
field).</div>
<div style=3D""><br></div><div style=3D"">-Doug</div><div =
style=3D""><br></div></div><div class=3D"gmail_extra"><br><br><div =
class=3D"gmail_quote">On Fri, Mar 15, 2013 at 3:45 AM, Tinco Andringa =
<span dir=3D"ltr"><<a href=3D"mailto:[email protected]" =
target=3D"_blank">[email protected]</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex">I think the point of =
the discussion is to not just assume C++ isn't<br>
going to do it. As far as I understand the goal is to have the =
ability<br>
to develop tools that aid programmers and artist in iterating =
rapidly.<br>
This means for artists that they can swap artwork in at runtime or<br>
quickly boot the game from a serialized state with new artwork, not<br>
just on his/her own computer but also over a network to perhaps a<br>
console. For programmers this may mean that they can view the game<br>
state easily, launch the game in various states easily, swap in/out<br>
scripts easily, maybe even change code without fully recompiling or<br>
even restarting the game.<br>
<br>
Am I missing something important?<br>
<div class=3D"HOEnZb"><div class=3D"h5"><br>
On Fri, Mar 15, 2013 at 8:33 AM, Massimo Del Zotto <<a =
href=3D"mailto:[email protected]">[email protected]</a>> =
wrote:<br>
> I have limited experience but I have been very impressed by the =
separation<br>
> provided by scripting languages.<br>
> Considering the goals:<br>
><br>
> My goal is to have fast iteration times, both artists and =
programmers<br>
><br>
> It appears to me C++ isn't going to do it. Especially on the artist =
side of<br>
> the thing.<br>
><br>
> By the way, I would like to have an insight on how this =
reflection<br>
> information is to be used. If it is used to simulate duck typing =
I'd use a<br>
> tool which supports duck typing natively.<br>
><br>
> Massimo<br>
><br>
</div></div><div class=3D"HOEnZb"><div class=3D"h5">> =
_______________________________________________<br>
> Sweng-Gamedev mailing list<br>
> <a =
href=3D"mailto:[email protected]">Sweng-Gamedev@lists.=
midnightryder.com</a><br>
> <a =
href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnight=
ryder.com" =
target=3D"_blank">http://lists.midnightryder.com/listinfo.cgi/sweng-gamede=
v-midnightryder.com</a><br>
><br>
_______________________________________________<br>
Sweng-Gamedev mailing list<br>
<a =
href=3D"mailto:[email protected]">Sweng-Gamedev@lists.=
midnightryder.com</a><br>
<a =
href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnight=
ryder.com" =
target=3D"_blank">http://lists.midnightryder.com/listinfo.cgi/sweng-gamede=
v-midnightryder.com</a><br>
</div></div></blockquote></div><br></div>
_______________________________________________<br>Sweng-Gamedev mailing =
list<br><a =
href=3D"mailto:[email protected]">Sweng-Gamedev@lists.=
midnightryder.com</a><br>http://lists.midnightryder.com/listinfo.cgi/sweng=
-gamedev-midnightryder.com<br></blockquote></div><br></body></html>=
--Apple-Mail=_47BE95EA-B4BE-4E24-B7B2-90BF2251051B--
--===============1403331153==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Sweng-Gamedev mailing list
[email protected]
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
--===============1403331153==--