Re: Architecting and programming a graphics engine in C++ & OpenGL

There were a few emails between me beginning to write my reply and finishing
it.

Peter - I actually recall a thread on this mailing list last year which
described a programming paradigm that shifted away from OOP toward a more
data driven approach and it piqued my interest considerably. I believe it
separated data and code in an attempt to allow transformation functions to
work on streams of data, I can't find it in my inbox at the moment though.
What you said is basically what I aim to do - create a proof of concept that
a little more advanced than taking an existing 3rd party tool and creating
something with it. Thanks also to Oladotun for giving me some links to read.

- Tommy

On Fri, Mar 18, 2011 at 10:40 AM, Tommy Brett <[email protected]> wrote:

> Thanks for the replies guys, I should clarify that yes a graphics
> programmer is what I would like to become, and if I turn out to be Pretty
> Good at it, then authoring my own game engine would be an exceptional
> challenge to overcome. How I'm going to get to that end goal, or even if the
> end goal is feasible is what I'm trying to ascertain. I understand your
> point Nicholas; the graphics engine I'd like to create might not even be
> rightfully called a 'graphics engine', but I need it to be something that
> would prove to an employer that I'm capable of doing more than stitching
> together some rendering algorithms I pulled out of a couple of GDC papers,
> and in doing so, would get me into a position where I'm able to learn from
> the pros.
>
> I've noticed that a lot of computer games degrees (of which I'm a graduate,
> albeit 5 years ago) include courses specifically designed to build you a
> portfolio, but almost all of them will involve a 3rd party tool to create
> the work. For example, "get 5 people together and create a 1 level prototype
> using Ogre, Unity or Torque". When I did my 'portfolio course' we actually
> built our own engine from scratch (of which I built the physics, AI and
> 'game' part of) and our Professor echoed the same sentiments as those in
> this email thread. But in that case we actually overcame the odds and
> produced a pretty good piece of work that I was able to extend through
> several other courses to include networking and more advanced modular AI...
> But again, the code was not something I'd be overly proud of now, when
> attempting to apply for a position that doesn't have the word 'junior' in
> it.
>
> From this email discussion I'm hoping to be able to get a clear picture of
> what I can do in the time that I have to create a piece of work I can show
> to an employer and say "I built this, the code is solid although simple in
> places, there's room for expansion but this proves without a doubt that I am
> not only capable of working with modern graphics engines, but I also know
> enough of the underlying low level concepts to have designed this". Which I
> think would give me an advantage over other graduates, and give me leverage
> for a paycheck that would justify leaving my current position.
>
> - Tommy
>
> On Fri, Mar 18, 2011 at 10:15 AM, Nicholas Aelick <[email protected]>wrote:
>
>> I would tend to agree with Megan. Building a graphics engine from scratch
>> for the learning experience is a little like building a computer by
>> soldering bits of wire together for the learning experience. You're unlikely
>> to get far, and even if you succeed, you're unlikely to end up with
>> something nicer then is already available.
>>
>> I would recommend looking into existing solutions (partially because any
>> place you work for will have you working with those, anyways). If you're
>> really serious about building a custom graphics engine, find one that's open
>> source, and see how they solved the problems you've been having.
>>
>> Nicholas
>>
>>
>> On Fri, Mar 18, 2011 at 10:03 AM, Megan Fox <[email protected]> wrote:
>>
>>> To take it in another way, I would generally recommend no one set out to
>>> write their own graphics engine from scratch without first using another.
>>> Simple example - OGRE.  Great jumping-off point, great community, etc.
>>> Build up a game around it, then burrow down and start changing OGRE to suit
>>> your needs, then burrow down further and tweak the graphics, and so on.
>>>
>>> There's very little reason to write a graphics engine yourself, unless
>>> you're addressing the faults of a pre-existing engine you tried and found
>>> insufficient.  Without that grounding, you're stumbling in the dark, and it
>>> will likely take quite some time and a hundred dead ends for you to get
>>> within spitting distance of even the bad current graphics engines.  You're
>>> also needlessly re-inventing the wheel, unless Graphics Programming is
>>> specifically what you're aiming at - there's a lot more to games programming
>>> than rendering, and your portfolio would need to be more than rendered
>>> planets to break in (ie. what happens on or with those planets, and why
>>> they're fun to walk around / fly around / whatever).
>>>
>>> ----
>>> Megan Fox
>>> http://www.shalinor.com/
>>> http://www.glassbottomgames.com/
>>>
>>>
>>>
>>> On Fri, Mar 18, 2011 at 7:58 AM, Rémi Papillié <[email protected]>wrote:
>>>
>>>> Hi Tommy,
>>>>
>>>> While this might not answer completely such a broad question, I would
>>>> recommend you look at this site:
>>>> http://gamearchitect.net/
>>>>
>>>> More specifically, some articles called 'An Anatomy of Despair'
>>>> explain the design of the author's engine. See for instance:
>>>>
>>>> http://gamearchitect.net/2008/04/19/an-anatomy-of-despair-orthogonal-views
>>>>
>>>> I would also recommend the excellent book 'Game Engine Architecture',
>>>> by Jason Gregory.
>>>>
>>>> Note, however, that these resources are mainly oriented towards game
>>>> engines as whole. AFAIK, there are not many resources on the specifics
>>>> of graphics engine design and architecture but, as you said, mainly a
>>>> lot of very technical articles one some technologies and
>>>> implementations.
>>>>
>>>> For what it's worth, here is the structure I currently use on my
>>>> personal engine. It works, but has not been confronted to a real prod,
>>>> so what follows is definitely not bullet-proof.
>>>>
>>>> The technical API (OpenGL/DirectX) is abstracted away in a thin layer.
>>>> I use this layer mainly for readibility purposes (e.g being able to
>>>> write texture->enableFiltering(BILINEAR) instead of two or three
>>>> cryptic OGL/DX calls), but with a bit more work, you can also use it
>>>> to make the engine cross-api (with many simple but error-prone
>>>> details, such as matrix transpositions).
>>>>
>>>> On top of that, two modules are present:
>>>> - The graphics scene contains an abstract description of graphical
>>>> elements that are part of the world. No shading algorithms here, just
>>>> plain buffers augmented with various information (transforms...). The
>>>> scene handles all issues about traversal and culling. Whenever a
>>>> render is asked, a render list is built by querying the spatial
>>>> partitioning structure, and then sent to the renderer (see below).
>>>> - The renderer is just the raw implementation of shading algorithms.
>>>> It receives a render list + a frustum and draws everything, using
>>>> various techniques (e.g forward or deferred shading).
>>>>
>>>> The render list basically contains packets in the form <transform,
>>>> mesh, material> (+ lights), but this is highly dependent on how the
>>>> rest is implemented.
>>>>
>>>> Hope this helps! I'll also be very happy to learn about other ways of
>>>> designing graphics engines, if someone else has good resources on the
>>>> subject ;)
>>>>
>>>> Rémi
>>>>
>>>> 2011/3/18 Tommy Brett <[email protected]>:
>>>> > Hi folks,
>>>> > I've been working as an AS3 programmer / animator for the last 3.5
>>>> years,
>>>> > and have the strongest desire to leave the marketing industry behind
>>>> and
>>>> > pursue my original intended career as a games programmer, which I
>>>> sidelined
>>>> > due to a very coincidental series of events. To this end, I've been
>>>> (slowly)
>>>> > building a graphics engine using C++, OpenGL and nVidia CG which
>>>> renders
>>>> > very large chunks of terrain using GPU Clipmaps. The idea is that
>>>> eventually
>>>> > it will render planets with atmosphere and real time shadows, or die
>>>> trying,
>>>> > and in any case will serve as my main portfolio piece when I
>>>> eventually
>>>> > start applying for positions. Because I've been outside of my
>>>> programming
>>>> > comfort zone for so long it's been slow going, but now that I've
>>>> finally got
>>>> > a firm grasp on shaders and the OpenGL pipeline it's time to look to
>>>> the
>>>> > future.
>>>> > This is where my problem lies; although there are many examples on the
>>>> 'net
>>>> > showing me how to animate the most beautiful normal mapped cobbled
>>>> cube, or
>>>> > tutorials on specific rendering / culling / lighting algorithms, I've
>>>> yet to
>>>> > find a resource beside Michael Abrash's Graphics Programming Black
>>>> Book
>>>> > (which sits on my desk, reminding me of what I should have been doing
>>>> all
>>>> > along) which goes into the subject of writing a graphics engine. See
>>>> > although I can write perfectly passable C++, and thanks to my AS3
>>>> > programming experience which has frequently required that I get my
>>>> projects
>>>> > right 'the first time', I'm fairly adept at planning and designing my
>>>> > classes before I begin... But from a C++ standpoint, I'm somewhat in
>>>> the
>>>> > dark. What are the best practices for building a graphics engine in
>>>> C++?
>>>> > Exactly how OOP should I go? How should I handle the use of multiple
>>>> shaders
>>>> > on chunks of geometry? There's just this giant multitude of questions,
>>>> and
>>>> > although I have a knack for over engineering things, I really want to
>>>> have a
>>>> > better picture of what I'm aiming for before I start, or I'll end up
>>>> with
>>>> > the monstrosity that is my terrain prototype (it runs, but it's so, so
>>>> > ugly).
>>>> > Currently my idea for a graphics engine involves something like an
>>>> engine
>>>> > class that takes render packets of data, has some sort of ability to
>>>> sort
>>>> > them into an order that best facilitates speedy rendering, and changes
>>>> its
>>>> > state depending on the information in the packets. E.g. one packet
>>>> might be
>>>> > a chunk of terrain, and could include information about what shaders
>>>> to use,
>>>> > how to determine its potential visible set, and the type of collision
>>>> > detection to use (though this deviates a little from the graphics
>>>> engine
>>>> > part). There might be an arbitrary number of terrain packets, followed
>>>> by
>>>> > some character models or buildings using a different type of culling
>>>> > algorithm. My worry aside from the obvious problem of the whole idea
>>>> > possibly being crap, is that it's too general. How could I possibly
>>>> create
>>>> > an engine that handles every possible rendering type, shouldn't I
>>>> perhaps
>>>> > start with the assertion that I'm building a terrain engine, and base
>>>> > everything around rendering sphere like chunks of geometry with
>>>> varying
>>>> > levels of detail? But on the same token, I don't just want to end up
>>>> with
>>>> > another terrain engine prototype as seen on youtube(tm).
>>>> > So I suppose what I'm looking for are book suggestions, or perhaps
>>>> > open-sourced graphics engines that I could look at and learn from, or
>>>> maybe
>>>> > I should just try and get a position as a junior-something-programmer
>>>> and
>>>> > learn by doing (the trouble is there aren't any such positions
>>>> available
>>>> > where I live, and relocation would be difficult to justify based upon
>>>> the
>>>> > possibly low salary expectations of a junior position, I'd like at
>>>> least
>>>> > some leverage). Ideas? Suggestions? Is this even the right place to
>>>> ask such
>>>> > broad questions? I'm all ears.
>>>> > - Tommy
>>>> > _______________________________________________
>>>> > Sweng-Gamedev mailing list
>>>> > [email protected]
>>>> >
>>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>>> >
>>>> >
>>>> _______________________________________________
>>>> Sweng-Gamedev mailing list
>>>> [email protected]
>>>>
>>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>>>
>>>
>>> _______________________________________________
>>> Sweng-Gamedev mailing list
>>> [email protected]
>>>
>>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>>
>>>
>>
>> _______________________________________________
>> Sweng-Gamedev mailing list
>> [email protected]
>>
>> http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com
>>
>>
>

_______________________________________________
Sweng-Gamedev mailing list
[email protected]
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com