Re: Rer: How I succeeded in building Mesa llvmpipe 64-bit on Windows

Jose,

Thank you for this. I will definitely have a closer look.
Eventually it will take a few days, as I'm out of office most time of next week.

Regards


-------- Original-Nachricht --------
> Datum: Tue, 30 Oct 2012 01:34:11 -0700 (PDT)
> Von: Jose Fonseca <[email protected]>
> An: [email protected]
> CC: Brian Paul <[email protected]>
> Betreff: Rer: [Mesa-users] How I succeeded in building Mesa llvmpipe 64-bit on Windows

> Charlie,
> 
> I suspect the reason why multi-threaded on windows is not good is our poor
> conditional variable implementation in
> src/gallium/auxiliary/os/os_thread.h .
> 
> The issue is that until Vista, Windows did not have native support for
> conditional variables : 
> http://msdn.microsoft.com/en-us/library/windows/desktop/ms682052(v=vs.85).aspx
> .
> 
> We need to support XP for now, so relying on Vista is not an option yet.
> Furthmore, our condition variable support is extremely poor -- the thread
> that waits on a thread keeps polling on the variable with a sleep.
> 
> So, if you are confortable with windows programming I suggest:
> 
> - modify src/gallium/auxiliary/os/os_thread.h to use Vista conditional
> variables, and see if that improves performance
> 
> - if so modify the os_thread.h code to detect Windows version, only using
> Windows conditional variables on vista or higher, and reverting to the old
> behavior on previous versions.
> 
> Jose
> 
> ----- Forwarded Message -----
> > From: Charlie Charsie <[email protected]>
> > To: Brian Paul <[email protected]>
> > CC: [email protected]
> > 
> > 
> > 
> > Dear Brian,
> > 
> > Yes, I think some of the information could be benefitial for the Mesa
> > documentation. I'd be also glad to contribute in a more detailed way
> > to the documentation of a Windows toolchain.
> > The current Mesa documentation avoids to be precise with respect to
> > supported (or at least known-to-work / best practice) toolchains /
> > compilers. I think more concrete input from the Mesa Devs on this is
> > a
> > prerequisite for improving the documentation.
> > 
> > > - Visual Studio 64 Bit doesn't like DLL exports by DEF file AND by
> > > declspec. Modify include/gl.h around ~ line 56: # define GLAPI
> > > /*declspec(dllexport)*/
> > 
> > About the symbol export on Visual Studio x64 environments, I'd please
> > you to take a look on this:
> >
> http://stackoverflow.com/questions/3572344/a-warning-with-building-64bit-dll
> > 
> > > - On some tested systems, performance was unequivocal slow. After
> > > disabling the LLVM threader (Env. var. LP_NUM_THREADS = 0),
> > > performance again was as expected with this setting.
> > 
> > No, those systems were up-to-date. As an example, a brand new Macbook
> > Pro Retina 15" showed that behaviour using Bootcamp/Windows 7 64Bit.
> > While CPU consumption was ~ 5%, simple OpenGL apps were almost
> > blocked. After disabling the threader, they were multiple times
> > faster.
> > The problem was reproducible with Mesa 8.0.2 LLVM 2.9, and Mesa 9.0
> > LLVM 3.1 - compiled as I posted earlier.
> > 
> > But also on other (Core-i5, Core-i7) systems, CPU consumption did not
> > go higher than 50 or 60 %. However, simple OpenGL 64-bit benchmark
> > has
> > shown that texture intense apps are already running twice to triple
> > as
> > fast as old MS OpenGL on Windows 64bit. Do you have an idea about why
> > the CPU is not fully busy?
> > 
> > > - VS 2010 might also work, but VS 2012 struggles with compiler
> > > keywords.
> > 
> > I restarted my Windows 8 VM with VS 2012. I remember LLVM 3.1 built
> > was fine.
> > 
> > If I now invoke Mesa build process, I get the following output - not
> > sure if it was the same two weeks ago, or if I did s.th. wrong...
> > 
> > 
> > Best,
> > MB
> > 
> > 
> > 
> > C:\git\mesa-9.0>set llvm=C:\git\llvm-3.1.install
> > 
> > C:\git\mesa-9.0>c:\Python27\scons.bat build=release llvm=yes
> > platform=
> > windows machine=x86_64 libgl-gdi
> > scons: Reading SConscript files ...
> > 
> > scons: *** Invalid value for option platform: .  Valid values are:
> > ('cygwin', 'darwin', 'freebsd', 'haiku', 'linux', 'sunos', 'windows')
> > File "C:\git\mesa-9.0\SConstruct", line 40, in <module>
> > 
> > C:\git\mesa-9.0>c:\Python27\scons.bat build=release llvm=yes
> > platform=windows machine=x86_64 libgl-gdi
> > scons: Reading SConscript files ...
> > scons: Found LLVM version 3.1svn
> > Checking for X11 (x11 xext xdamage xfixes)... no
> > Checking for XCB (x11-xcb xcb-glx >= 1.8.1)... no
> > Checking for XF86VIDMODE (xxf86vm)... no
> > Checking for DRM (libdrm >= 2.4.24)... no
> > Checking for DRM_INTEL (libdrm_intel >= 2.4.30)... no
> > Checking for DRM_RADEON (libdrm_radeon >= 2.4.31)... no
> > Checking for XORG (xorg-server >= 1.6.0)... no
> > Checking for KMS (libkms >= 2.4.24)... no
> > Checking for UDEV (libudev > 150)... no
> > scons: done reading SConscript files.
> > 
> > scons: warning: you do not seem to have the pywin32 extensions
> > installed;
> >          parallel (-j) builds may not work reliably with open Python
> > files.
> > File "c:\Python27\Scripts\scons.py", line 190, in <module>
> > scons: Building targets ...
> >    Generating src\mesa\main\dispatch.h ...
> > Traceback (most recent call last):
> >    File "src\mapi\glapi\gen\gl_table.py", line 28, in <module>
> >      import gl_XML
> >    File "C:\git\mesa-9.0\src\mapi\glapi\gen\gl_XML.py", line 28, in
> > <module>
> >      import libxml2
> > ImportError: No module named libxml2
> >    Compiling src\gallium\drivers\softpipe\sp_prim_vbuf.c ...
> >    Compiling src\gallium\drivers\softpipe\sp_setup.c ...
> > sp_setup.c
> > sp_prim_vbuf.c
> > scons: *** [src\mesa\main\dispatch.h] Error 1
> >    Compiling src\gallium\drivers\softpipe\sp_quad_blend.c ...
> > sp_quad_blend.c
> > src\gallium\drivers\softpipe\sp_setup.c(581) : warning C4244: '=' :
> > conversion from 'double' to 'float', possible loss of data
> > src\gallium\drivers\softpipe\sp_setup.c(587) : warning C4244: '=' :
> > conversion from 'double' to 'float', possible loss of data
> > src\gallium\drivers\softpipe\sp_setup.c(589) : warning C4244: '=' :
> > conversion from 'double' to 'float', possible loss of data
> > scons: building terminated because of errors.
> > 
> > scons: *** Found dependency cycle(s):
> >    Internal Error: no cycle found for node src\mapi\glapi\glprocs.h
> > (<SCons.Node.FS.File object at 0x000000000323AEB8>) in state pending
> >    Internal Error: no cycle found for node
> >    src\mapi\glapi\glapitable.h
> > (<SCons.Node.FS.File object at 0x000000000323A9B0>) in state pending
> >    Internal Error: no cycle found for node
> > src\mesa\main\remap_helper.h (<SCons.Node.FS.File object at
> > 0x00000000032431D0>) in state pending
> >    Internal Error: no cycle found for node src\mapi\glapi\glapitemp.h
> > (<SCons.Node.FS.File object at 0x000000000323ABE0>) in state pending
> > 
> > File "c:\Python27\scons-2.2.0\SCons\Taskmaster.py", line 1026, in
> > cleanup
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > -------- Original-Nachricht --------
> > > Datum: Mon, 29 Oct 2012 10:57:56 -0600
> > > Von: Brian Paul <[email protected]>
> > > An: Charlie Charsie <[email protected]>
> > > CC: [email protected]
> > > Betreff: Re: [Mesa-users] How I succeeded in building Mesa llvmpipe
> > > 64-bit on	Windows
> > 
> > > On 10/29/2012 10:20 AM, Charlie Charsie wrote:
> > > > Dear list,
> > > >
> > > > I successfully built llvmpipe on Windows with native Visual
> > > > Studio
> > > 2008sp1, and wanted to share my experiences.
> > >
> > > If you think that some of this information should be rolled into
> > > the
> > > Mesa documentation, please submit some patches to do so.
> > >
> > >
> > > > Most I did was 64-bit, however, it should work for 32-bit compile
> > > settings, also.
> > > >
> > > > Requirements:
> > > > - Python 2.7 (32-bit)
> > > > - SCons 2.2
> > > > - CMake
> > > > - MSYS, e.g. provided by MinGW. Including Flex&  Bison. MSYS
> > > > Console:
> > > "mingw-get install msys-flex msys-bison", and environment path
> > > modified
> > > respectively (e.g. "PATH=...;C:\MinGW\msys\1.0\bin;").
> > > >
> > > > MESA 8.0.2 with LLVM 2.9
> > > > ------------------------
> > > > Start with LLVM:
> > > > - LLVM required patch source code a bit in prior:
> > > http://llvm.org/bugs/show_bug.cgi?id=9976 , and
> > > http://llvm.org/bugs/attachment.cgi?id=6634
> > > > - Use Cmake to configure, runtime libraries linked with /MT
> > > > (statically)
> > > > - Define Install target in CMake and install binaries after
> > > > build.
> > > > - Set install path on environment: "SET LLVM=C:\MyLLVMInstall"
> > > >
> > > > MESA:
> > > > - Visual Studio 64 Bit doesn't like DLL exports by DEF file AND
> > > > by
> > > declspec. Modify include/gl.h around ~ line 56: # define GLAPI
> > > /*declspec(dllexport)*/
> > >
> > > I'd be hesitant to do that unconditionally in Mesa because it might
> > > break something else.  Maybe there should be some sort of #ifdef
> > > around that change.
> > >
> > >
> > > > - Open Visual Studio x64 command prompt
> > > > - Invoke build by: c:\python27\scons.bat build=release llvm=yes
> > > platform=windows machine=x86_64 libgl-gdi
> > > >
> > > >
> > > > MESA 9.0 with LLVM 3.1
> > > > ----------------------
> > > > - Works quite the same way without having to patch LLVM.
> > > >
> > > >
> > > > MESA 8.0.5 with LLVM 2.9
> > > > ------------------------
> > > > - Got a compile error in shaderobj.c (~280), after defining the
> > > > unsigned
> > > int in a different way, compile succeeds. However:
> > >
> > > Please post your patch for this so we can fix it.  But I've been
> > > building Mesa on Windows and don't recall seeing a problem there.
> > >
> > >
> > > > - DLL crashes after about 100 rendered frames all the time -
> > > > don't know
> > > why.
> > > >
> > > > I tried native MinGW
> > > > --------------------
> > > > - I installed 64 bit from http://tdm-gcc.tdragon.net/download
> > > (experimental)
> > > > - I invoked build process by calling: c:\python27\scons.bat
> > > build=release llvm=yes platform=windows toolchain=mingw
> > > machine=x86_64 libgl-gdi
> > > >
> > > > Result: Initially, SCons always tried to use the VC compiler,
> > > > after
> > > setting all up on a different VM, I ended in the well-known
> > > "command line too
> > > long" problem
> > >
> (http://scons.tigris.org/ds/viewMessage.do?dsForumId=1272&dsMessageId=883411)
> > > >
> > > > I see no solution to this without having the Mesa Developers to
> > > > take
> > > care of it in their built solution.
> > > >
> > > > I tried cross-compile using Ubuntu
> > > > ----------------------------------
> > > > - different way to tell mesa the LLVM root, try: "export
> > > LLVM=/home/myLLVMInstall" or "export LLVM_ROOT=/home/myLLVMInstall"
> > > > - I got compile errors, maybe related to
> > > https://bugs.freedesktop.org/show_bug.cgi?id=44618
> > > >
> > > >
> > > > FURTHER TIPS
> > > > ------------
> > > >
> > > > - So far no success in using the software rasterizer of mesa, DLL
> > > crashes on use
> > > > - OSMesa exports all symbols under windows - So far, I cannot use
> > > > it
> > > along with llvmpipe.
> > >
> > > Yeah, there's no support for OSMesa + llvmpipe.
> > >
> > >
> > > >
> > > > - On all tested systems, CPU resources have not been exhausted
> > > > properly
> > > by llvmpipe. However, performance much better compared to Microsoft
> > > software renderer.
> > > >
> > > > - On some tested systems, performance was unequivocal slow. After
> > > disabling the LLVM threader (Env. var. LP_NUM_THREADS = 0),
> > > performance again was
> > > as expected with this setting.
> > >
> > > Are those systems single core or what?
> > >
> > >
> > > > - VS 2010 might also work, but VS 2012 struggles with compiler
> > > > keywords.
> > >
> > > Can you be more specific?
> > >
> > >
> > > >
> > > > LINKS
> > > > -----
> > > >
> > > > Some links that might be helpful (not all emerged to be correct
> > > > though):
> > > >
> > >
> http://phoronix.com/forums/showthread.php?72780-LLVMpipe-On-Mesa-8-1-Performance
> > > >
> > >
> http://www.phoronix.com/scan.php?page=article&item=llvmpipe_snb_bulldozer&num=2
> > > > http://www.phoronix.com/scan.php?page=news_item&px=MTA5ODc
> > > > https://sites.google.com/site/buildllvmpipe/
> > > >
> > > >
> > > > Hope that helps.
> > > >
> > > > Best,
> > > > MB
> > > > _______________________________________________
> > > > mesa-users mailing list
> > > > [email protected]
> > > > http://lists.freedesktop.org/mailman/listinfo/mesa-users
> > >
> > > -Brian
> >