Re: sparse components

Perhaps this summary may aid yours and others thinking:
1) Never forget the CPU loads data from memory transforms it (in Acton-speak) and writes the modified data out2) It doesn't transform just one input and produce one output - it processes streams of data whether or not you choose to see it that way3) The natural unit of program composition is not a conceptional 'OO' object but functions that transform an input stream to an output stream (stream or list)4) It sounds simple and obvious and it is - but decades of 'OO' teaching have obscured this fact in many peoples minds5) Functional decomposition leads to extremely reusable code without the awkwardness OO constructs can introduce6) By functional decomposition I mean your function only reads its input and writes its output and touches no hidden shared state inside7) If you view the input and output parameters that are connecting your functions as asynchronous streams rather than instant value parameters I would suggest writing concurrent code becomes much easier8) Such parameter streams can be implemented as message passing lockless queues (Forget STM) and functions can be invoked when they have parameters to process by kicking them to a pool of worker threads9) Push your iterations on the parameter streams down to the lowest level function they can go and you get the 'Action: 'Where there is one there is many'10) Notice how tuples of parameter stream data can be efficiently processed by SSE instructions etc11) Investigate what this means in respect to languages such as Erlang, Haskell, C#, F#, C++, C and understand why in lots of ways staying closer to the C subset of C++ in OO constructs is a good thing (Templates on such functions are great tools also)12) View your input parameters as immutable data structures13) Optimise your immutable data structures by performing copies only when necessary and providing garbage collection for them and transmission support for distributed computing (Understand how this maps to functional language compiler optimisations) - and how this can collapse to simple double buffer schemes in cases etc14) Notice how efficiently a component/aspect/state/whatever entity system can be implemented in tandem with this and how well this maps to relational database tables (and ignore object orientated databases from now on)15) Organise data into tables of actual data and sets of different indices into that data - and think in terms of queries, adds/removes and updates16) Just have structs of data with the only functions allowed being initialisers and those that can compute values from the data in an immutable fashion17) Read up on dataflow/flow based programming and model your programs by data flow and transforms on the data rather than ANY CLASS HIERARCHY AT ALL18) Put this into practice in your C++ projects by using only static functions of the form described utilising the grouping features of namespaces and then if a naturally easy to use object orientated hierarchy fits nicely on top of certain collections of functions then create OO objects that utilise your existing static functions.  Notice how this probably fits lots of the advice of good OO programming.
I hope that helps!
Cheers,
Warrick Buchanan

From: [email protected]
Date: Fri, 28 May 2010 16:43:30 +0100
To: [email protected]
Subject: Re: [Sweng-Gamedev] sparse components

I've read all the Mike Acton's stuff I can find. He's a really interesting read for performance, I especially liked his small bit on branch optimisation, went and implemented the stuff only to find that it's not just faster on Cell's but on CISC out-of-order CPUs as well. He's also very clued up on concurrency and I've definitely learned from him in that area. Without what he had written, I'd have had a couple of problems trying to perceive some of the other benefits to using my collected techniques.


My general work is almost like an attempt to bring a lot of "right thought" to the general coders rather than go for ultimate throughput, because I've found that optimising conceptually wrong code can be a nightmare of refactoring before you even get to doing any real performance tuning. I guess what I'm doing is trying to make it possible to enable the less clued up to write code that easier for performance guru's like Mike to make a bigger impact by having things laid out in a simple to grasp format.


On 28 May 2010 16:18, Javier Arevalo <[email protected]> wrote:


You might enjoy this:



http://www.insomniacgames.com/blogcast/blog/mike_acton/1500756



Sorry I can't elaborate much, but I can tell you you're not the only one thinking in that direction.



On 5/28/2010 16:44 PM, Richard Fabian wrote:


I'm working on some stuff at the moment, a kind of collection of techniques that I've started pulling together into a more coherent single way of thinking.



One of the techniques is something I've been calling Existence Based Processing, and it was born of my early implementations of component based development. I've now heard someone call it Push Processing, but can't find any references on Google, and wondered if anyone knew of the right search phrase.



In my component system, I oriented the component processing to a per component type rather than per entity, which meant I was keeping the I cache warm all the time. I was lucky, I hadn't noticed this benefit at the time, and now I do. Also, because components only existed for entities which needed them, there was no processing done for entities without that component. At the time I had simple finite state machines for AI that switched in and out which component the AI was using as a hierarchical FSM, but I've seen now that this rabbit hole goes deeper.





So, what's this stuff called normally, and also, why aren't we taught anything about it, and why aren't we using it?



I've seen speed ups in my own code due to rewriting small sections in this style, so can't fathom why we're not using it in general, or for that matter, taught it at all.



I've breathed even more life into this technique with my current work on writing up my experience with data oriented development by adding or at least clarifying what this sparse existence component technique can offer, including some work on making your components behave as states for an FSM, or even, because now I only have the concept of an implicit core entity, the ability to generate multiple entities from one when needed (such as level of detail change), and also collapse multiple entities down to one when the level of detail is no longer necessary.





A bit of background,: I was spurred on a bit by the "OOP is bad" talk into putting a bit more effort into what I was doing so that other people might benefit. The basis of my stuff is really a revisit to old computer science, relational databases to be specific, and leveraging what has been learned over the decades into something more useful for games. I've found that there are a number of parallels in database technology and current high performance game development, and I think it's because we're effectively in the same hardware situation, just scaled up a lot. I think of the old DB hard drives as our main memory, and their old small computers as being equivalent to our modern CPUs and parallel machines. That is, we're using powerful but small processing machines, committing transforms on the data set that is in a slow medium.





I'm currently doing general research to find out if there are other old science techniques that we've forgotten, or not repurposed, while also writing a proof of concept rogue-like in 3D. This does mean I have to write two whole games that are exactly the same, as best I can, just to prove any of my research, but I hope that doing so I may come across some other unexpected discoveries.








-- 

Javier Arevalo

http://www.iguanademos.com/Jare



_______________________________________________

Sweng-Gamedev mailing list

[email protected]

http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com



-- 
fabs();
Just because the world is full of people that think just like you, doesn't mean the other ones can't be right.

 		 	   		  
_________________________________________________________________
http://clk.atdmt.com/UKM/go/197222280/direct/01/
We want to hear all your funny, exciting and crazy Hotmail stories. Tell us now

_______________________________________________
Sweng-Gamedev mailing list
[email protected]
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com