Re: Sweng-Gamedev Digest, Vol 71, Issue 18
Gabriel Sassone <[email protected]> Fri, 29 Mar 2013 17:11:39 -0700
| Newsgroups | gmane.games.devel.sweng |
|---|---|
| Message-ID | <CAFcPrsFpi2Xk4OGHqeACXxDJCgpKs+LWF-dk=he16-dab=ScNw@mail.gmail.com> |
--===============1296267095==
Content-Type: multipart/alternative; boundary=089e0112c00e5b257d04d9193be4
--089e0112c00e5b257d04d9193be4
Content-Type: text/plain; charset=ISO-8859-1
Thanks for all the reply guys, much appreciated.
The links you wrote are a lot helpful to have a deeper insight about this
challenge!
Still the solution to this problem really depends on the scope of the
project you're working on.
So far the main separation is between the reflection API and the way to
feed it, in one of the 5 methods.
I will continue experimenting with it, even though already having setting
up a simple api with fields and types showed me the power of this approach.
Sure there are limitations, and with complex memory layout you can bump in
some problems (virtual or multiple inheritance, even though are not my
case), but gives you a solid framework to start looking at things done with
less code, less errors and concentrating more on algorithms more that data
conversion!
I was thinking also about pointers to object and memory allocation: when
you are serializing/deserializing a pointer to an object, what is your
strategy?
You want to have a building order of objects, taking track also of the
instances of an object and the memory allocator you take the memory from.
What are your experiences on that?
Thanks again guys,
Gab
> Message: 1
> Date: Tue, 19 Mar 2013 16:40:14 -0400
> From: Douglas Cox <[email protected]>
> To: [email protected]
> Subject: Re: [Sweng-Gamedev] Reflection in game engine (c++)
> Message-ID:
> <CA+i4re6_-sfYRWHKdOFyO-SbRTk=
> [email protected]>
> Content-Type: text/plain; charset="iso-8859-1"
>
> "That system continues to work with dlc and patches, but if your patch
> needs to change a type, you'd probably be sending new data files using that
> type anyway."
>
> You'd think so, but we've had a few examples of this already in the last
> few years. Only a few new instances needed to override a default value.
>
> "We usually read these files on a separate thread, and it's comforting not
> to be making lots of allocs that might fragment memory if interleaved with
> allocations that might happen on other threads."
>
> This seems like it would happen regardless unless your main 'runtime'
> allocations are coming from a different memory pool.
>
> I probably should have mentioned that I was also talking about overriding
> values on 'prefabs' with this same system. So when we override just a few
> position/rotations it is saving considerably more than flattening the
> entire prefab into the level file for instance.
>
> Sorry I should have made this a separate thread.
>
> -Doug
>
>
> On Tue, Mar 19, 2013 at 4:02 PM, Adrian Stephens <[email protected]
> >wrote:
>
> >
> > We flatten our data, so reading data in-game is just a question of
> > allocating enough space (from the different flavors of memory - normal,
> > video, sound, etc) reading the data and patching the results. During
> > development there is the possibility of reading some out-of-date types,
> and
> > those need to be converted during the patch-up, which means they do take
> > extra memory. The shipped code shouldn't have to do that.
> >
> > That system continues to work with dlc and patches, but if your patch
> > needs to change a type, you'd probably be sending new data files using
> that
> > type anyway.
> >
> > We usually read these files on a separate thread, and it's comforting not
> > to be making lots of allocs that might fragment memory if interleaved
> with
> > allocations that might happen on other threads.
> >
> > Regarding storage space - if you compress your binary files, as we do,
> any
> > often-used default values are likely to compress away very easily; and
> > since our data is flattened and compressed during development, there's no
> > additional code to worry about.
> >
> > Adrian
> >
> > On Mar 19, 2013, at 9:09 AM, Douglas Cox wrote:
> >
> > One issue that we were thinking about after coming up with a file format
> > using reflection that allows for data-derivation, versioning, etc., and
> > that basically looks like "field name + field type + value" (only
> > overriding/saving values that were changed) was: "Is it worth trying to
> > optimize or flatten this for final builds?"
> >
> > At first we were just going to write a different serializer and write out
> > a flattened set of in-order values thinking that our types would never
> > change. But with DLC and patches, this is not the case. We may release
> a
> > patch that changes a type (adds or removes a field) and existing
> > (non-patched) data files would fail to load.
> >
> > Another reason to not flatten is that it is quite possible that the final
> > data in the 'overrides only' format would be much less storage than the
> > flattened format. And of course not having to write additional code
> that's
> > only run in final builds is a plus.
> >
> > Anyone else have thoughts on this or do it differently?
> >
> > --
> > And as far as the debate over where to put the type info, all I can say
> is
> > that having the compiler generate it and having 1 file (a single ".x"
> > instead of ".h" and ".cpp" and ".inl") for code has been great. I
> > definitely do not miss bouncing back and forth between .h and .cpp files.
> > So if you do start wandering into the land of making your own language
> for
> > just reflection, you might see what other benefits you can get from doing
> > more in it than just that.
> > --
> >
> > -Doug
> >
> >
> > On Tue, Mar 19, 2013 at 12:07 AM, Mike Shaver <[email protected]
> >wrote:
> >
> >> I wanted to just have one place that things were declared, and for that
> >> to be with real code rather in a separate "please don't edit this no
> matter
> >> how much VC++ tells you this is the definition" file.
> >>
> >> My point is really that it's over-constraining the problem to choose one
> >> format that is both fast+compact enough for runtime use, and
> >> flexible+straightforward enough to bend into all the parts of the tool
> >> pipeline.
> >>
> >> It's the work of 5 lines of Python to go from JSON to XML or CSV or .ini
> >> files or whatever. That someone has linked the format parser into a
> dozen
> >> different software environments is indeed an advantage that I would take
> >> seriously. I generally don't know what my tools are going to be until
> I've
> >> written them, so free option value is pretty nice.
> >>
> >> I've done the "IDL to headers and packed metadata files" thing, and it
> >> was pretty annoying (and error-prone) to wire it into even a half-dozen
> >> straight-C tools. I may be overreacting to that experience, but mutatis
> >> mutandis I very much prefer one tiny exporter and then letting tools
> >> transcode however they want.
> >>
> >> Mike
> >>
> >> phone-typed
> >>
> >> On Mar 18, 2013, at 7:58 PM, Adrian Stephens <[email protected]>
> >> wrote:
> >>
> >>
> >> So you're writing macros to turn your C structures into data, building
> >> them into a library, linking it with your exporter and running the
> exporter
> >> to generate a script-format of the structures. Which, then has to be
> >> parsed by your tools. I honestly didn't mean that to sound so negative,
> >> but wouldn't it have been easier the other way round - start with a
> simple
> >> domain-specific language to define your types and spit out C header
> files?
> >>
> >> The PHP thing works because you're proposing outputting JSON, which
> >> somebody presumably has already linked into it. If that sort of thing
> is
> >> difficult you could certainly output JSON at the same time you output C
> >> header files, or I suppose you could probably even use JSON *as* your
> >> script format.
> >>
> >>
> >> On Mar 18, 2013, at 5:26 PM, Mike Shaver wrote:
> >>
> >> When I did this for a non-game app, I defined it with self-aware,
> >> strong-AI macros, and compiled the resulting data into a shared library.
> >> The app (game) linked against the library, as did one important tool:
> the
> >> exporter. It was a wee little program that dumped the compiled metadata
> to
> >> an interchange format. Today I would choose JSON or XML. The build
> process
> >> would regenerate metadata.json whenever metadata.so changed, and
> extending
> >> it to multiple libraries/output files was straightforward.
> >>
> >> Runtime reflection formats have many constraints that tool-interchange
> >> formats don't, and tool formats often want things the runtime abhors.
> Put
> >> the line-numbers in the output and remix with git-blame output to tell
> you
> >> who last changed an entity? Sure. Web explorer of the resources, without
> >> linking the parser library into PHP...I'll be over here.
> >>
> >> I would similarly not use Collada files at runtime.
> >>
> >> phone-typed
> >>
> >>
> >> On Mar 18, 2013, at 4:30 PM, Adrian Stephens <[email protected]>
> >> wrote:
> >>
> >>
> >> I'm glad we're still talking about this - I think it's important and I
> >> wanted to add my own observations:
> >>
> >> The macro/template approach to reflection only applies to one side of
> the
> >> data/code interface. If all you need to do is serialize runtime data
> to be
> >> read back in by runtime code, then you're ok; but if you want to access
> >> that metadata from an external tool, you either have to recompile the
> tool
> >> or plugin when anything changes, or write something that parses that
> code.
> >>
> >> If you're going to have to write a parser anyway, it seems to me you
> >> might as well use it for both code and data views. Which means you can
> use
> >> a more natural syntax for your structure definitions that even
> >> non-programmers can use.
> >>
> >> To map this back into your C++ code, you can pretty trivially write out
> >> header files containing typedefs that correspond to the metadata, and
> >> generate whatever you need to allow the runtime to interpret serialized
> >> data. This also address the separation of code and assets: we don't
> want
> >> artists having to sync the code, and this keeps the data definitions
> with
> >> the assets that they define.
> >>
> >> This is what I do, anyway. I have a data language, which defines types,
> >> and which can also be used to define data, and all data is stored using
> >> these definitions. This includes bitmaps, sounds, models, etc. The
> data
> >> language is separated from the data representation so this doesn't imply
> >> that bitmaps are stored in a script-like array of colors - but it does
> mean
> >> that tools can use the reflection system on more classes of data than
> just
> >> small gameplay entities. And when distinctions like that are removed,
> and
> >> all things are treated the same way, lots of clever things just fall
> out.
> >>
> >> Some other random thoughts:
> >> As far as I can see, all C++ needs to allow full reflection is some sort
> >> of fieldsof(T) operator that enumerates the fields of a structure. e.g.
> >> (typed straight into email, so probably bollocks - and not
> representative
> >> of my actual system!)
> >>
> >> struct metadata {
> >> enum TYPE {INT, PTR, ARRAY, STRUCT} type;
> >> metadata(TYPE &_type) : type(_type) {}
> >> };
> >> template<typename A, typename... B> struct md_fields : md<A>,
> >> md_fields<B> {};
> >> template<typename T> struct md : metadata, md_fields<fieldsof(T)> { md()
> >> : metadata(STRUCT) {} };
> >> template<typename T> struct md<T*> : metadata { md<T> t; md() :
> >> metadata(PTR) {} };
> >> template<typename T, int N> struct md<T[N]> : metadata { int n; md<T> t;
> >> md() : metadata(ARRAY), n(N) {} };
> >> template<> struct md<int> : metadata { md() : metadata(INT) {} };
> >>
> >> template<typename T> metadata *get_md(const T &t) { static md<T> m;
> >> return &m; }
> >>
> >>
> >>
> >> Adrian Stephens
> >>
> >> On Mar 18, 2013, at 10:45 AM, Douglas Cox wrote:
> >>
> >> That sounds pretty similar to the goals of the engine that we're working
> >> on -- basically faster iteration times for everyone (this includes
> engine
> >> programmers as well).
> >>
> >> All of our code (including the compiler) is written in a C#-like
> >> language, so we have perfect reflection from that. Attributes are
> pretty
> >> much required for providing additional information to property editors
> for
> >> things like valid ranges of a field or notifications of a property
> change
> >> (most of which can go away in Final builds).
> >>
> >> Another thing to consider may be allowing for data derivation. This can
> >> save quite a bit of work when the need for copies of objects come up
> that
> >> only need one or two properties changed. If you pick a serialization
> >> format for the editable data that knows about the data (fieldtype +
> >> fieldname + value), it makes it a lot easier to get this working.
> >>
> >> In the past, we've used Macros and they worked fine. They're a bit more
> >> annoying to extend with optional parameters and you have to repeat the
> >> field/property name, but that's not terrible since you can generally
> get it
> >> down to compile-time errors if you get something wrong (other than
> >> forgetting to add a new field).
> >>
> >> -Doug
> >>
> >>
> >>
> >> On Fri, Mar 15, 2013 at 3:45 AM, Tinco Andringa <[email protected]> wrote:
> >>
> >>> I think the point of the discussion is to not just assume C++ isn't
> >>> going to do it. As far as I understand the goal is to have the ability
> >>> to develop tools that aid programmers and artist in iterating rapidly.
> >>> This means for artists that they can swap artwork in at runtime or
> >>> quickly boot the game from a serialized state with new artwork, not
> >>> just on his/her own computer but also over a network to perhaps a
> >>> console. For programmers this may mean that they can view the game
> >>> state easily, launch the game in various states easily, swap in/out
> >>> scripts easily, maybe even change code without fully recompiling or
> >>> even restarting the game.
> >>>
> >>> Am I missing something important?
> >>>
> >>> On Fri, Mar 15, 2013 at 8:33 AM, Massimo Del Zotto <
> [email protected]>
> >>> wrote:
> >>> > I have limited experience but I have been very impressed by the
> >>> separation
> >>> > provided by scripting languages.
> >>> > Considering the goals:
> >>> >
> >>> > My goal is to have fast iteration times, both artists and programmers
> >>> >
> >>> > It appears to me C++ isn't going to do it. Especially on the artist
> >>> side of
> >>> > the thing.
> >>> >
> >>> > By the way, I would like to have an insight on how this reflection
> >>> > information is to be used. If it is used to simulate duck typing I'd
> >>> use a
> >>> > tool which supports duck typing natively.
> >>> >
> >>> > Massimo
> >>> >
> >>> > _______________________________________________
> >>> > Sweng-Gamedev mailing list
> >>> > [email protected]
> >>> >
> >>>
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
> >>> >
> >>> _______________________________________________
> >>> Sweng-Gamedev mailing list
> >>> [email protected]
> >>>
> >>>
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
> >>>
> >>
> >> _______________________________________________
> >> Sweng-Gamedev mailing list
> >> [email protected]
> >>
> >>
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
> >>
> >>
> >> _______________________________________________
> >> Sweng-Gamedev mailing list
> >> [email protected]
> >>
> >>
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
> >>
> >> _______________________________________________
> >> Sweng-Gamedev mailing list
> >> [email protected]
> >>
> >>
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
> >>
> >>
> >> _______________________________________________
> >> Sweng-Gamedev mailing list
> >> [email protected]
> >>
> >>
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
> >>
> >>
> >> _______________________________________________
> >> Sweng-Gamedev mailing list
> >> [email protected]
> >>
> >>
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
> >>
> >>
> > _______________________________________________
> > Sweng-Gamedev mailing list
> > [email protected]
> >
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
> >
> >
> >
> > _______________________________________________
> > Sweng-Gamedev mailing list
> > [email protected]
> >
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.midnightryder.com/private.cgi/sweng-gamedev-midnightryder.com/attachments/20130319/357fd431/attachment.htm
> >
>
> ------------------------------
>
> _______________________________________________
> Sweng-Gamedev mailing list
> [email protected]
> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>
>
> End of Sweng-Gamedev Digest, Vol 71, Issue 18
> *********************************************
>
--089e0112c00e5b257d04d9193be4
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Thanks for all the reply guys, much appreciated.<br>The links you wrote are=
a lot helpful to have a deeper insight about this challenge!<div><br></div=
><div>Still the solution to this problem really depends on the scope of the=
project you're working on.</div>
<div>So far the main separation is between the reflection API and the way t=
o feed it, in one of the 5 methods.</div><div><br></div><div>I will continu=
e experimenting with it, even though already having setting up a simple api=
with fields and types showed me the power of this approach.</div>
<div>Sure there are limitations, and with complex memory layout you can bum=
p in some problems (virtual or multiple inheritance, even though are not my=
case), but gives you a solid framework to start looking at things done wit=
h less code, less errors and concentrating more on algorithms more that dat=
a conversion!</div>
<div><br></div><div>I was thinking also about pointers to object and memory=
allocation: when you are serializing/deserializing a pointer to an object,=
what is your strategy?<br>You want to have a building order of objects, ta=
king track also of the instances of an object and the memory allocator you =
take the memory from.<br>
<br>What are your experiences on that?</div><div><br></div><div>Thanks agai=
n guys,</div><div>=A0 =A0 Gab</div><div><br><div class=3D"gmail_quote"><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c=
cc solid;padding-left:1ex">
<br>
Message: 1<br>
Date: Tue, 19 Mar 2013 16:40:14 -0400<br>
From: Douglas Cox <<a href=3D"mailto:[email protected]" target=3D"_blank"=
>[email protected]</a>><br>
To: <a href=3D"mailto:[email protected]" target=3D"_blank">sw=
[email protected]</a><br>
Subject: Re: [Sweng-Gamedev] Reflection in game engine (c++)<br>
Message-ID:<br>
=A0 =A0 =A0 =A0 <CA+i4re6_-sfYRWHKdOFyO-SbRTk=3D<a href=3D"mailto:FBUBmW=
A1mCsWZoyes7%[email protected]" target=3D"_blank">FBUBmWA1mCsWZoyes7+dpA=
@mail.gmail.com</a>><br>
Content-Type: text/plain; charset=3D"iso-8859-1"<br>
<br>
"That system continues to work with dlc and patches, but if your patch=
<br>
needs to change a type, you'd probably be sending new data files using =
that<br>
type anyway."<br>
<br>
You'd think so, but we've had a few examples of this already in the=
last<br>
few years. =A0Only a few new instances needed to override a default value.<=
br>
<br>
"We usually read these files on a separate thread, and it's comfor=
ting not<br>
to be making lots of allocs that might fragment memory if interleaved with<=
br>
allocations that might happen on other threads."<br>
<br>
This seems like it would happen regardless unless your main 'runtime=
9;<br>
allocations are coming from a different memory pool.<br>
<br>
I probably should have mentioned that I was also talking about overriding<b=
r>
values on 'prefabs' with this same system. =A0So when we override j=
ust a few<br>
position/rotations it is saving considerably more than flattening the<br>
entire prefab into the level file for instance.<br>
<br>
Sorry I should have made this a separate thread.<br>
<br>
-Doug<br>
<br>
<br>
On Tue, Mar 19, 2013 at 4:02 PM, Adrian Stephens <<a href=3D"mailto:adri=
[email protected]" target=3D"_blank">[email protected]</a>>wrote:<=
br>
<br>
><br>
> We flatten our data, so reading data in-game is just a question of<br>
> allocating enough space (from the different flavors of memory - normal=
,<br>
> video, sound, etc) reading the data and patching the results. =A0Durin=
g<br>
> development there is the possibility of reading some out-of-date types=
, and<br>
> those need to be converted =A0during the patch-up, which means they do=
take<br>
> extra memory. =A0The shipped code shouldn't have to do that.<br>
><br>
> That system continues to work with dlc and patches, but if your patch<=
br>
> needs to change a type, you'd probably be sending new data files u=
sing that<br>
> type anyway.<br>
><br>
> We usually read these files on a separate thread, and it's comfort=
ing not<br>
> to be making lots of allocs that might fragment memory if interleaved =
with<br>
> allocations that might happen on other threads.<br>
><br>
> Regarding storage space - if you compress your binary files, as we do,=
any<br>
> often-used default values are likely to compress away very easily; and=
<br>
> since our data is flattened and compressed during development, there&#=
39;s no<br>
> additional code to worry about.<br>
><br>
> Adrian<br>
><br>
> On Mar 19, 2013, at 9:09 AM, Douglas Cox wrote:<br>
><br>
> One issue that we were thinking about after coming up with a file form=
at<br>
> using reflection that allows for data-derivation, versioning, etc., an=
d<br>
> that basically looks like "field name + field type + value" =
(only<br>
> overriding/saving values that were changed) was: "Is it worth try=
ing to<br>
> optimize or flatten this for final builds?"<br>
><br>
> At first we were just going to write a different serializer and write =
out<br>
> a flattened set of in-order values thinking that our types would never=
<br>
> change. =A0But with DLC and patches, this is not the case. =A0We may r=
elease a<br>
> patch that changes a type (adds or removes a field) and existing<br>
> (non-patched) data files would fail to load.<br>
><br>
> Another reason to not flatten is that it is quite possible that the fi=
nal<br>
> data in the 'overrides only' format would be much less storage=
than the<br>
> flattened format. =A0And of course not having to write additional code=
that's<br>
> only run in final builds is a plus.<br>
><br>
> Anyone else have thoughts on this or do it differently?<br>
><br>
> --<br>
> And as far as the debate over where to put the type info, all I can sa=
y is<br>
> that having the compiler generate it and having 1 file (a single "=
;.x"<br>
> instead of ".h" and ".cpp" and ".inl") f=
or code has been great. =A0I<br>
> definitely do not miss bouncing back and forth between .h and .cpp fil=
es.<br>
> =A0So if you do start wandering into the land of making your own langu=
age for<br>
> just reflection, you might see what other benefits you can get from do=
ing<br>
> more in it than just that.<br>
> --<br>
><br>
> -Doug<br>
><br>
><br>
> On Tue, Mar 19, 2013 at 12:07 AM, Mike Shaver <<a href=3D"mailto:mi=
[email protected]" target=3D"_blank">[email protected]</a>>wrote:<=
br>
><br>
>> I wanted to just have one place that things were declared, and for=
that<br>
>> to be with real code rather in a separate "please don't e=
dit this no matter<br>
>> how much VC++ tells you this is the definition" file.<br>
>><br>
>> My point is really that it's over-constraining the problem to =
choose one<br>
>> format that is both fast+compact enough for runtime use, and<br>
>> flexible+straightforward enough to bend into all the parts of the =
tool<br>
>> pipeline.<br>
>><br>
>> It's the work of 5 lines of Python to go from JSON to XML or C=
SV or .ini<br>
>> files or whatever. That someone has linked the format parser into =
a dozen<br>
>> different software environments is indeed an advantage that I woul=
d take<br>
>> seriously. I generally don't know what my tools are going to b=
e until I've<br>
>> written them, so free option value is pretty nice.<br>
>><br>
>> I've done the "IDL to headers and packed metadata files&q=
uot; thing, and it<br>
>> was pretty annoying (and error-prone) to wire it into even a half-=
dozen<br>
>> straight-C tools. I may be overreacting to that experience, but mu=
tatis<br>
>> mutandis I very much prefer one tiny exporter and then letting too=
ls<br>
>> transcode however they want.<br>
>><br>
>> Mike<br>
>><br>
>> phone-typed<br>
>><br>
>> On Mar 18, 2013, at 7:58 PM, Adrian Stephens <<a href=3D"mailto=
:[email protected]" target=3D"_blank">[email protected]</a>><b=
r>
>> wrote:<br>
>><br>
>><br>
>> So you're writing macros to turn your C structures into data, =
building<br>
>> them into a library, linking it with your exporter and running the=
exporter<br>
>> to generate a script-format of the structures. =A0Which, then has =
to be<br>
>> parsed by your tools. =A0I honestly didn't mean that to sound =
so negative,<br>
>> but wouldn't it have been easier the other way round - start w=
ith a simple<br>
>> domain-specific language to define your types and spit out C heade=
r files?<br>
>><br>
>> The PHP thing works because you're proposing outputting JSON, =
which<br>
>> somebody presumably has already linked into it. =A0If that sort of=
thing is<br>
>> difficult you could certainly output JSON at the same time you out=
put C<br>
>> header files, or I suppose you could probably even use JSON *as* y=
our<br>
>> script format.<br>
>><br>
>><br>
>> On Mar 18, 2013, at 5:26 PM, Mike Shaver wrote:<br>
>><br>
>> When I did this for a non-game app, I defined it with self-aware,<=
br>
>> strong-AI macros, and compiled the resulting data into a shared li=
brary.<br>
>> The app (game) linked against the library, as did one important to=
ol: the<br>
>> exporter. It was a wee little program that dumped the compiled met=
adata to<br>
>> an interchange format. Today I would choose JSON or XML. The build=
process<br>
>> would regenerate metadata.json whenever metadata.so changed, and e=
xtending<br>
>> it to multiple libraries/output files was straightforward.<br>
>><br>
>> Runtime reflection formats have many constraints that tool-interch=
ange<br>
>> formats don't, and tool formats often want things the runtime =
abhors. Put<br>
>> the line-numbers in the output and remix with git-blame output to =
tell you<br>
>> who last changed an entity? Sure. Web explorer of the resources, w=
ithout<br>
>> linking the parser library into PHP...I'll be over here.<br>
>><br>
>> I would similarly not use Collada files at runtime.<br>
>><br>
>> phone-typed<br>
>><br>
>><br>
>> On Mar 18, 2013, at 4:30 PM, Adrian Stephens <<a href=3D"mailto=
:[email protected]" target=3D"_blank">[email protected]</a>><b=
r>
>> wrote:<br>
>><br>
>><br>
>> I'm glad we're still talking about this - I think it's=
important and I<br>
>> wanted to add my own observations:<br>
>><br>
>> The macro/template approach to reflection only applies to one side=
of the<br>
>> data/code interface. =A0If all you need to do is serialize runtime=
data to be<br>
>> read back in by runtime code, then you're ok; but if you want =
to access<br>
>> that metadata from an external tool, you either have to recompile =
the tool<br>
>> or plugin when anything changes, or write something that parses th=
at code.<br>
>><br>
>> If you're going to have to write a parser anyway, =A0it seems =
to me you<br>
>> might as well use it for both code and data views. =A0Which means =
you can use<br>
>> a more natural syntax for your structure definitions that even<br>
>> non-programmers can use.<br>
>><br>
>> To map this back into your C++ code, you can pretty trivially writ=
e out<br>
>> header files containing typedefs that correspond to the metadata, =
and<br>
>> generate whatever you need to allow the runtime to interpret seria=
lized<br>
>> data. =A0This also address the separation of code and assets: we d=
on't want<br>
>> artists having to sync the code, and this keeps the data definitio=
ns with<br>
>> the assets that they define.<br>
>><br>
>> This is what I do, anyway. =A0I have a data language, which define=
s types,<br>
>> and which can also be used to define data, and all data is stored =
using<br>
>> these definitions. =A0This includes bitmaps, sounds, models, etc. =
=A0The data<br>
>> language is separated from the data representation so this doesn&#=
39;t imply<br>
>> that bitmaps are stored in a script-like array of colors - but it =
does mean<br>
>> that tools can use the reflection system on more classes of data t=
han just<br>
>> small gameplay entities. =A0And when distinctions like that are re=
moved, and<br>
>> all things are treated the same way, lots of clever things just fa=
ll out.<br>
>><br>
>> Some other random thoughts:<br>
>> As far as I can see, all C++ needs to allow full reflection is som=
e sort<br>
>> of fieldsof(T) operator that enumerates the fields of a structure.=
=A0e.g.<br>
>> (typed straight into email, so probably bollocks - and not represe=
ntative<br>
>> of my actual system!)<br>
>><br>
>> struct metadata {<br>
>> enum TYPE {INT, PTR, ARRAY, STRUCT} type;<br>
>> metadata(TYPE &_type) : type(_type) {}<br>
>> };<br>
>> template<typename A, typename... B> struct md_fields : md<=
;A>,<br>
>> md_fields<B> {};<br>
>> template<typename T> struct md : metadata, md_fields<fiel=
dsof(T)> { md()<br>
>> : metadata(STRUCT) {} };<br>
>> template<typename T> struct md<T*> : metadata { md<=
T> t; md() :<br>
>> metadata(PTR) {} };<br>
>> template<typename T, int N> struct md<T[N]> : metadata=
{ int n; md<T> t;<br>
>> md() : metadata(ARRAY), n(N) {} };<br>
>> template<> struct md<int> : metadata { md() : metadata=
(INT) {} };<br>
>><br>
>> template<typename T> metadata *get_md(const T &t) { stat=
ic md<T> m;<br>
>> return &m; }<br>
>><br>
>><br>
>><br>
>> Adrian Stephens<br>
>><br>
>> On Mar 18, 2013, at 10:45 AM, Douglas Cox wrote:<br>
>><br>
>> That sounds pretty similar to the goals of the engine that we'=
re working<br>
>> on -- basically faster iteration times for everyone (this includes=
engine<br>
>> programmers as well).<br>
>><br>
>> All of our code (including the compiler) is written in a C#-like<b=
r>
>> language, so we have perfect reflection from that. =A0Attributes a=
re pretty<br>
>> much required for providing additional information to property edi=
tors for<br>
>> things like valid ranges of a field or notifications of a property=
change<br>
>> (most of which can go away in Final builds).<br>
>><br>
>> Another thing to consider may be allowing for data derivation. =A0=
This can<br>
>> save quite a bit of work when the need for copies of objects come =
up that<br>
>> only need one or two properties changed. =A0If you pick a serializ=
ation<br>
>> format for the editable data that knows about the data (fieldtype =
+<br>
>> fieldname + value), it makes it a lot easier to get this working.<=
br>
>><br>
>> In the past, we've used Macros and they worked fine. =A0They&#=
39;re a bit more<br>
>> annoying to extend with optional parameters and you have to repeat=
the<br>
>> field/property name, but that's not terrible since you can gen=
erally get it<br>
>> down to compile-time errors if you get something wrong (other than=
<br>
>> forgetting to add a new field).<br>
>><br>
>> -Doug<br>
>><br>
>><br>
>><br>
>> On Fri, Mar 15, 2013 at 3:45 AM, Tinco Andringa <<a href=3D"mai=
lto:[email protected]" target=3D"_blank">[email protected]</a>> wrote:<br>
>><br>
>>> I think the point of the discussion is to not just assume C++ =
isn't<br>
>>> going to do it. As far as I understand the goal is to have the=
ability<br>
>>> to develop tools that aid programmers and artist in iterating =
rapidly.<br>
>>> This means for artists that they can swap artwork in at runtim=
e or<br>
>>> quickly boot the game from a serialized state with new artwork=
, not<br>
>>> just on his/her own computer but also over a network to perhap=
s a<br>
>>> console. For programmers this may mean that they can view the =
game<br>
>>> state easily, launch the game in various states easily, swap i=
n/out<br>
>>> scripts easily, maybe even change code without fully recompili=
ng or<br>
>>> even restarting the game.<br>
>>><br>
>>> Am I missing something important?<br>
>>><br>
>>> On Fri, Mar 15, 2013 at 8:33 AM, Massimo Del Zotto <<a href=
=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>=
><br>
>>> wrote:<br>
>>> > I have limited experience but I have been very impressed =
by the<br>
>>> separation<br>
>>> > provided by scripting languages.<br>
>>> > Considering the goals:<br>
>>> ><br>
>>> > My goal is to have fast iteration times, both artists and=
programmers<br>
>>> ><br>
>>> > It appears to me C++ isn't going to do it. Especially=
on the artist<br>
>>> side of<br>
>>> > the thing.<br>
>>> ><br>
>>> > By the way, I would like to have an insight on how this r=
eflection<br>
>>> > information is to be used. If it is used to simulate duck=
typing I'd<br>
>>> use a<br>
>>> > tool which supports duck typing natively.<br>
>>> ><br>
>>> > Massimo<br>
>>> ><br>
>>> > _______________________________________________<br>
>>> > Sweng-Gamedev mailing list<br>
>>> > <a href=3D"mailto:[email protected]" =
target=3D"_blank">[email protected]</a><br>
>>> ><br>
>>> <a href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-g=
amedev-midnightryder.com" target=3D"_blank">http://lists.midnightryder.com/=
listinfo.cgi/sweng-gamedev-midnightryder.com</a><br>
>>> ><br>
>>> _______________________________________________<br>
>>> Sweng-Gamedev mailing list<br>
>>> <a href=3D"mailto:[email protected]" targe=
t=3D"_blank">[email protected]</a><br>
>>><br>
>>> <a href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-g=
amedev-midnightryder.com" target=3D"_blank">http://lists.midnightryder.com/=
listinfo.cgi/sweng-gamedev-midnightryder.com</a><br>
>>><br>
>><br>
>> _______________________________________________<br>
>> Sweng-Gamedev mailing list<br>
>> <a href=3D"mailto:[email protected]" target=3D=
"_blank">[email protected]</a><br>
>><br>
>> <a href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-gamed=
ev-midnightryder.com" target=3D"_blank">http://lists.midnightryder.com/list=
info.cgi/sweng-gamedev-midnightryder.com</a><br>
>><br>
>><br>
>> _______________________________________________<br>
>> Sweng-Gamedev mailing list<br>
>> <a href=3D"mailto:[email protected]" target=3D=
"_blank">[email protected]</a><br>
>><br>
>> <a href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-gamed=
ev-midnightryder.com" target=3D"_blank">http://lists.midnightryder.com/list=
info.cgi/sweng-gamedev-midnightryder.com</a><br>
>><br>
>> _______________________________________________<br>
>> Sweng-Gamedev mailing list<br>
>> <a href=3D"mailto:[email protected]" target=3D=
"_blank">[email protected]</a><br>
>><br>
>> <a href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-gamed=
ev-midnightryder.com" target=3D"_blank">http://lists.midnightryder.com/list=
info.cgi/sweng-gamedev-midnightryder.com</a><br>
>><br>
>><br>
>> _______________________________________________<br>
>> Sweng-Gamedev mailing list<br>
>> <a href=3D"mailto:[email protected]" target=3D=
"_blank">[email protected]</a><br>
>><br>
>> <a href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-gamed=
ev-midnightryder.com" target=3D"_blank">http://lists.midnightryder.com/list=
info.cgi/sweng-gamedev-midnightryder.com</a><br>
>><br>
>><br>
>> _______________________________________________<br>
>> Sweng-Gamedev mailing list<br>
>> <a href=3D"mailto:[email protected]" target=3D=
"_blank">[email protected]</a><br>
>><br>
>> <a href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-gamed=
ev-midnightryder.com" target=3D"_blank">http://lists.midnightryder.com/list=
info.cgi/sweng-gamedev-midnightryder.com</a><br>
>><br>
>><br>
> _______________________________________________<br>
> Sweng-Gamedev mailing list<br>
> <a href=3D"mailto:[email protected]" target=3D"_bl=
ank">[email protected]</a><br>
> <a href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-m=
idnightryder.com" target=3D"_blank">http://lists.midnightryder.com/listinfo=
.cgi/sweng-gamedev-midnightryder.com</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Sweng-Gamedev mailing list<br>
> <a href=3D"mailto:[email protected]" target=3D"_bl=
ank">[email protected]</a><br>
> <a href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-m=
idnightryder.com" target=3D"_blank">http://lists.midnightryder.com/listinfo=
.cgi/sweng-gamedev-midnightryder.com</a><br>
><br>
><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href=3D"http://lists.midnightryder.com/private.cgi/sweng-gamede=
v-midnightryder.com/attachments/20130319/357fd431/attachment.htm" target=3D=
"_blank">http://lists.midnightryder.com/private.cgi/sweng-gamedev-midnightr=
yder.com/attachments/20130319/357fd431/attachment.htm</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
Sweng-Gamedev mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">=
[email protected]</a><br>
<a href=3D"http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnig=
htryder.com" target=3D"_blank">http://lists.midnightryder.com/listinfo.cgi/=
sweng-gamedev-midnightryder.com</a><br>
<br>
<br>
End of Sweng-Gamedev Digest, Vol 71, Issue 18<br>
*********************************************<br>
</blockquote></div><br></div>
--089e0112c00e5b257d04d9193be4--
--===============1296267095==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Sweng-Gamedev mailing list
[email protected]
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
--===============1296267095==--