Re: Thoughts about PyOpenGL, and other stuff.
matthew sitton <[email protected]> Tue, 20 Oct 2015 12:39:52 -0500
| Newsgroups | gmane.comp.python.opengl.devel |
|---|---|
| Message-ID | <CAL_9FWLBYG=HTU_PGfaX5HhSRLpA5SDf=1U=45NqscBW6Ngvgg@mail.gmail.com> |
--===============3446106225801429670== Content-Type: multipart/alternative; boundary=001a113adef0d7d9d005228cbf57 --001a113adef0d7d9d005228cbf57 Content-Type: text/plain; charset=UTF-8 So here a year later i thought I would give you an update on this. I really only started working on this again about a month ago, but i've made decent progress since. Still a lot of stuff to do, but i've made decent progress at reading the xml registry(only really need to do extensions now), and generating a binding. Didn't end up using your code though plus i've never been a fan of ElementTree fan. So I ended up making a wrapper around xml.parser.expat to do the parsing of the xml registry. I still have a lot of ideas and plans for this, mainly: - Wrapper layer that is mostly compatible with pyopengl. - Basic context creation code for various platforms (no late binding for gl calls) - Various run-time, and generation-time options to customize the binding depending on the needs of a project. The code is here: https://github.com/mdsitton/pyogl If you have any feedback I would be glad to hear it. Thanks, Matthew On Thu, Oct 2, 2014 at 10:41 PM, Mike C. Fletcher <[email protected]> wrote: > On 02/10/14 16:15, matthew sitton wrote: > > So i have been thinking about starting my own opengl binding for > > python, but i thought why not come here and post some of what i am > > wanting out of an opengl binding. > > Sounds fun, be aware that there are actually a *lot* of Python OpenGL > bindings, so you might want to survey them first to see if there is a > jump-start available. > > > Make "Modern OpenGL" a first class citizen. Mainly this would involve > > the following in my opinion: > > - Provide a math library (Something generally equivalent to GLM or > others) > > Math libraries are a hard one IMO. > > GLM is something like 20,000 lines of code, Numpy is huge (quarter of a > million LOC or so). A wrapper around GLM would be great, something > sip-based (or the like) that would let you use GLM vectors etc naturally > from Python would be cool. Thing is, that really sounds like a separate > project (both in terms of scope and focus) from PyOpenGL. Similarly, > while I have a library of code for using Numpy in my scenegraph it > doesn't seem like something that should be dropped into PyOpenGL. > > Basic problem in my mind is that any sufficiently > useful/clean/attractive math library likely should be its own project, > potentially with support in PyOpenGL for it (e.g. as Numpy is directly > supported). > > > - Support opengl profiles, core forward compatible, and compatibility > > based off of the xml registry. > > Sure, using the Registry is pretty much a given, I would think. I'm not > sure whether you'll see a huge advantage to the core vs. compatibility > profiles, I suppose memory usage and start-up might be better. > > > - This would also mean providing imports for opengl functionality > > levels, so if you import OpenGL 3.3 core, you would be returned a > > module that contains all of the required Features at that > > functionality level, and none of the removed features. > > Should be pretty straightforward to do that. The registry has all of the > information for generating those namespaces. However, currently the > manually-wrapped code is all tied into the current module-per-extension > structure. > > > I also think it would be interesting to experiment with reorganization > > of the code to provide an abstraction of ctypes so that a cffi backend > > could be introduced. This should allow PyOpenGL to run under pypy with > > improved performance. > > I believe there's actually a fork of PyOpenGL somewhere that did a > ctypes -> cffi translation. I don't know if there was any performance > improvements. I wouldn't get my hopes up *too* much as a *lot* of the > overhead in PyOpenGL is due to munging the parameters to functions to > make APIs "pythonic", that overhead is significantly reduced by the > OpenGL_accelerate module, which likely won't be useful under PyPy. > Something like Pyglet, which uses the C api directly should give you a > much better bang-for-the-buck improvement in a cffi translation (IIRC > they may have already done that translation too). > > > I have also been thinking about digging into the source, and figuring > > out how the whole thing works better. That way i could contribute to > > the project. I read through this document: > > http://pyopengl.sourceforge.net/documentation/development.html > > However i suspect that its outdated. > > Yup, it's definitely outdated, the whole raw-wrapper hierarchy is now > generated from the XML registry. You can start reading at > `src/xml_generate.py` which is the overall runner that regenerates the > automated wrappers. In particular, if you want to generate your own > wrappers you'll likely want to look at `src/xmlreg.py` which takes the > xml and generates an in-memory model of the APIs. The module > `src/codegenerator.py` is the stuff that actually spits out the modules > and packages. You should pretty easily be able to modify that to spit > out a different style of wrapper. > > I'd say just go ahead and play with generating a wrapper in a form you > feel is good. PyOpenGL tries to be newbie-friendly and does a *lot* of > stuff that really shouldn't be done if you are concerned about creating > a fast engine. Really, if you want to build a fast engine you likely > don't care much about the API you are using, so using a raw wrapper > (i.e. direct cffi or Cython access to unvarnished C-style API) is likely > what you want; you're going to build abstractions over it anyway, so you > don't need them at the API level; pyglet is the obvious example of how > to make that work. > > Things I'd suggest doing if you want a simple, clean, fast API: > > * no lists/tuples/incorrect-formats; demand a binary-compatible, > buffer-protocol-supporting (or native cffi/ctypes) data-type for all > pointer/array parameters > * no pointer management; just let the user crash if they forget to > keep the object pointer around, maybe provide a debug mode that uses > weakref-based logging to display which objects are being deallocated > to make it easier to debug crashes > * don't muck about with the low-level API unnecessarily, every time > you imply parameters or unpack values to be "friendly" you increase > the overhead > * drop GLX, GLU, GLUT, WGL > * don't provide context-checking or the like to try to catch errors, > just let the crashes happen > * use a gl-object with attribute lookup instead of actual modules, so > you only need to load those entry points that get used (note, you'll > get error reports that this makes IDEs unable to find names, sorry > about that) > > I believe you'll find that's *basically* pyglet's automated wrappers, or > SWIG's automated wrappers. I think there's even a Cython wrapper that > basically follows this approach. > > Anyway, go for it. I'm happy to give you assistance on how to use the > PyOpenGL/src tools to generate your own wrapper, and if something is > obviously better I'm happy to look at adopting it for PyOpenGL. If you > want to do something like adding a function to the OpenGL root namespace > (something like) get_feature('GL_VERSION_1_1',profile='core') just to > get started playing, that would be fine too. > > Have fun, > Mike > > -- > ________________________________________________ > Mike C. Fletcher > Designer, VR Plumber, Coder > http://www.vrplumber.com > http://blog.vrplumber.com > > > > ------------------------------------------------------------------------------ > Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer > Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer > > http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk > _______________________________________________ > PyOpenGL Homepage > http://pyopengl.sourceforge.net > _______________________________________________ > PyOpenGL-Devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/pyopengl-devel > --001a113adef0d7d9d005228cbf57 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><span style=3D"font-size:12.8px">So here a year later i th= ought I would give you an update on this. I really only started working on = this again about a month ago, but i've made decent progress since.</spa= n><div style=3D"font-size:12.8px"><br></div><div style=3D"font-size:12.8px"= >Still a lot of stuff to do, but i've made decent progress at reading t= he xml registry(only really need to do extensions now), and generating a bi= nding.</div><div style=3D"font-size:12.8px">Didn't end up using your co= de though plus i've never been a fan of ElementTree fan. So I ended up = making a wrapper around xml.parser.expat to do the parsing of the xml regis= try.</div><div style=3D"font-size:12.8px"><br></div><div style=3D"font-size= :12.8px">I still have a lot of ideas and plans for this, mainly:</div><div = style=3D"font-size:12.8px">- Wrapper layer that is mostly compatible with= =C2=A0<span class=3D"">pyopengl</span>.</div><div style=3D"font-size:12.8px= ">- Basic context creation code for various platforms (no late binding for = gl calls)</div><div style=3D"font-size:12.8px">- Various run-time, and gene= ration-time options to customize the binding depending on the needs of a pr= oject.</div><div style=3D"font-size:12.8px"><br></div><div style=3D"font-si= ze:12.8px">The code is here:=C2=A0<a href=3D"https://github.com/mdsitton/py= ogl" target=3D"_blank">https://github.com/mdsitton/pyogl</a></div><div styl= e=3D"font-size:12.8px"><br></div><div style=3D"font-size:12.8px">If you hav= e any feedback I would be glad to hear it.</div><div><br></div><div>Thanks,= Matthew</div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quot= e">On Thu, Oct 2, 2014 at 10:41 PM, Mike C. Fletcher <span dir=3D"ltr"><= <a href=3D"mailto:[email protected]" target=3D"_blank">mcfletch@vrplum= ber.com</a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D= "margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class= =3D"">On 02/10/14 16:15, matthew sitton wrote:<br> > So i have been thinking about starting my own opengl binding for<br> > python, but i thought why not come here and post some of what i am<br> > wanting out of an opengl binding.<br> <br> </span>Sounds fun, be aware that there are actually a *lot* of Python OpenG= L<br> bindings, so you might want to survey them first to see if there is a<br> jump-start available.<br> <span class=3D""><br> > Make "Modern OpenGL" a first class citizen. Mainly this woul= d involve<br> > the following in my opinion:<br> > - Provide a math library (Something generally equivalent to GLM or oth= ers)<br> <br> </span>Math libraries are a hard one IMO.<br> <br> GLM is something like 20,000 lines of code, Numpy is huge (quarter of a<br> million LOC or so). A wrapper around GLM would be great, something<br> sip-based (or the like) that would let you use GLM vectors etc naturally<br= > from Python would be cool. Thing is, that really sounds like a separate<br> project (both in terms of scope and focus) from PyOpenGL. Similarly,<br> while I have a library of code for using Numpy in my scenegraph it<br> doesn't seem like something that should be dropped into PyOpenGL.<br> <br> Basic problem in my mind is that any sufficiently<br> useful/clean/attractive math library likely should be its own project,<br> potentially with support in PyOpenGL for it (e.g. as Numpy is directly<br> supported).<br> <span class=3D""><br> > - Support opengl profiles, core forward compatible, and compatibility<= br> > based off of the xml registry.<br> <br> </span>Sure, using the Registry is pretty much a given, I would think. I= 9;m not<br> sure whether you'll see a huge advantage to the core vs. compatibility<= br> profiles, I suppose memory usage and start-up might be better.<br> <span class=3D""><br> > - This would also mean providing imports for opengl functionality<br> > levels, so if you import OpenGL 3.3 core, you would be returned a<br> > module that contains all of the required Features at that<br> > functionality level, and none of the removed features.<br> <br> </span>Should be pretty straightforward to do that. The registry has all of= the<br> information for generating those namespaces. However, currently the<br> manually-wrapped code is all tied into the current module-per-extension<br> structure.<br> <span class=3D""><br> > I also think it would be interesting to experiment with reorganization= <br> > of the code to provide an abstraction of ctypes so that a cffi backend= <br> > could be introduced. This should allow PyOpenGL to run under pypy with= <br> > improved performance.<br> <br> </span>I believe there's actually a fork of PyOpenGL somewhere that did= a<br> ctypes -> cffi translation. I don't know if there was any performanc= e<br> improvements. I wouldn't get my hopes up *too* much as a *lot* of the<b= r> overhead in PyOpenGL is due to munging the parameters to functions to<br> make APIs "pythonic", that overhead is significantly reduced by t= he<br> OpenGL_accelerate module, which likely won't be useful under PyPy.<br> Something like Pyglet, which uses the C api directly should give you a<br> much better bang-for-the-buck improvement in a cffi translation (IIRC<br> they may have already done that translation too).<br> <span class=3D""><br> > I have also been thinking about digging into the source, and figuring<= br> > out how the whole thing works better. That way i could contribute to<b= r> > the project. I read through this document:<br> > <a href=3D"http://pyopengl.sourceforge.net/documentation/development.h= tml" rel=3D"noreferrer" target=3D"_blank">http://pyopengl.sourceforge.net/d= ocumentation/development.html</a><br> > However i suspect that its outdated.<br> <br> </span>Yup, it's definitely outdated, the whole raw-wrapper hierarchy i= s now<br> generated from the XML registry. You can start reading at<br> `src/xml_generate.py` which is the overall runner that regenerates the<br> automated wrappers. In particular, if you want to generate your own<br> wrappers you'll likely want to look at `src/xmlreg.py` which takes the<= br> xml and generates an in-memory model of the APIs. The module<br> `src/codegenerator.py` is the stuff that actually spits out the modules<br> and packages. You should pretty easily be able to modify that to spit<br> out a different style of wrapper.<br> <br> I'd say just go ahead and play with generating a wrapper in a form you<= br> feel is good. PyOpenGL tries to be newbie-friendly and does a *lot* of<br> stuff that really shouldn't be done if you are concerned about creating= <br> a fast engine. Really, if you want to build a fast engine you likely<br> don't care much about the API you are using, so using a raw wrapper<br> (i.e. direct cffi or Cython access to unvarnished C-style API) is likely<br= > what you want; you're going to build abstractions over it anyway, so yo= u<br> don't need them at the API level; pyglet is the obvious example of how<= br> to make that work.<br> <br> Things I'd suggest doing if you want a simple, clean, fast API:<br> <br> =C2=A0 * no lists/tuples/incorrect-formats; demand a binary-compatible,<br> =C2=A0 =C2=A0 buffer-protocol-supporting (or native cffi/ctypes) data-type = for all<br> =C2=A0 =C2=A0 pointer/array parameters<br> =C2=A0 * no pointer management; just let the user crash if they forget to<b= r> =C2=A0 =C2=A0 keep the object pointer around, maybe provide a debug mode th= at uses<br> =C2=A0 =C2=A0 weakref-based logging to display which objects are being deal= located<br> =C2=A0 =C2=A0 to make it easier to debug crashes<br> =C2=A0 * don't muck about with the low-level API unnecessarily, every t= ime<br> =C2=A0 =C2=A0 you imply parameters or unpack values to be "friendly&qu= ot; you increase<br> =C2=A0 =C2=A0 the overhead<br> =C2=A0 * drop GLX, GLU, GLUT, WGL<br> =C2=A0 * don't provide context-checking or the like to try to catch err= ors,<br> =C2=A0 =C2=A0 just let the crashes happen<br> =C2=A0 * use a gl-object with attribute lookup instead of actual modules, s= o<br> =C2=A0 =C2=A0 you only need to load those entry points that get used (note,= you'll<br> =C2=A0 =C2=A0 get error reports that this makes IDEs unable to find names, = sorry<br> =C2=A0 =C2=A0 about that)<br> <br> I believe you'll find that's *basically* pyglet's automated wra= ppers, or<br> SWIG's automated wrappers. I think there's even a Cython wrapper th= at<br> basically follows this approach.<br> <br> Anyway, go for it. I'm happy to give you assistance on how to use the<b= r> PyOpenGL/src tools to generate your own wrapper, and if something is<br> obviously better I'm happy to look at adopting it for PyOpenGL. If you<= br> want to do something like adding a function to the OpenGL root namespace<br= > (something like) get_feature('GL_VERSION_1_1',profile=3D'core&#= 39;) just to<br> get started playing, that would be fine too.<br> <br> Have fun,<br> Mike<br> <br> --<br> ________________________________________________<br> =C2=A0 =C2=A0Mike C. Fletcher<br> =C2=A0 =C2=A0Designer, VR Plumber, Coder<br> =C2=A0 =C2=A0<a href=3D"http://www.vrplumber.com" rel=3D"noreferrer" target= =3D"_blank">http://www.vrplumber.com</a><br> =C2=A0 =C2=A0<a href=3D"http://blog.vrplumber.com" rel=3D"noreferrer" targe= t=3D"_blank">http://blog.vrplumber.com</a><br> <br> <br> ---------------------------------------------------------------------------= ---<br> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer<br> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports<br= > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper<br> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer<br> <a href=3D"http://pubads.g.doubleclick.net/gampad/clk?id=3D154622311&iu= =3D/4140/ostg.clktrk" rel=3D"noreferrer" target=3D"_blank">http://pubads.g.= doubleclick.net/gampad/clk?id=3D154622311&iu=3D/4140/ostg.clktrk</a><br= > _______________________________________________<br> PyOpenGL Homepage<br> <a href=3D"http://pyopengl.sourceforge.net" rel=3D"noreferrer" target=3D"_b= lank">http://pyopengl.sourceforge.net</a><br> _______________________________________________<br> PyOpenGL-Devel mailing list<br> <a href=3D"mailto:[email protected]">PyOpenGL-Devel@list= s.sourceforge.net</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/pyopengl-devel" rel= =3D"noreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/listi= nfo/pyopengl-devel</a><br> </blockquote></div><br></div> --001a113adef0d7d9d005228cbf57-- --===============3446106225801429670== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------------ --===============3446106225801429670== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ PyOpenGL Homepage http://pyopengl.sourceforge.net _______________________________________________ PyOpenGL-Devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/pyopengl-devel --===============3446106225801429670==--