Re: Re[10]: : boehm port to Native Client

Hi Ivan,

We haven't landed all of the NaCl ARM changes into mono upstream yet, but
will be working on it.  Here is where the ARM work currently lives:
https://github.com/igotti-google/mono/tree/arm_nacl

You can see changes made to libgc/ if you want to merge these in:
https://github.com/igotti-google/mono/commits/arm_nacl/libgc

Specifically, it looks like maybe you don't have changes from this:
https://github.com/igotti-google/mono/commit/8fada1863f201e29516a5442f91000bfefc39417

-Elijah


On Sun, Sep 15, 2013 at 7:56 AM, Ivan Maidanski <ivmai-JGs/[email protected]> wrote:

>
> Hi Elijah,
>
> I found several commits from you to mono repository this year (8 months
> ago) including ARM support:
> *
> https://github.com/ivmai/bdwgc/commit/204bcc6ede64402ca3ff2b165fc47a63374c68b1
> *
> https://github.com/ivmai/bdwgc/commit/b328e8824e6ffa7cbd38c83e26433bee24bd3bb1
> *
> https://github.com/ivmai/bdwgc/commit/21f0df75f5bac02619fb6fa4bcf5b49b169aeb1a
>
> Anything to upstream?
> Thank you.
>
>
> Tue, 6 Mar 2012, 7:51 -08:00 from Elijah Taylor <[email protected]>:
>
>   That's correct that mono master doesn't have these changes. I'm working
> to get these changes in upstream mono so I can retire my fork.
>
> I'll let you know of further changes in the future but I would call it
> pretty stable at this point and don't expect it to change for some time.
>  On Mar 6, 2012 12:11 AM, "Ivan Maidanski" <ivmai-JGs/[email protected]<https://e.mail.ru/sentmsg?mailto=mailto%3aivmai-JGs/[email protected]>>
> wrote:
>
> Hi Elijah,
>
> Thank you for your reply.
> In future, please also CC to this ML on committing to mono/libgc.
> Unfortunately, i don't have time to check it on NaCl target.
>
> PS. I don't see anything else in mono master relevant to GC and NaCl in
> 2011.
> (for convenience, we have a branch "mono_libgc" which is a copy of
> mono/master - https://github.com/ivmai/bdwgc/tree/mono_libgc )
>
> 05 03 2012, 23:11 Elijah Taylor <[email protected]<https://e.mail.ru/sentmsg?mailto=mailto%[email protected]>
> >:
> > Hi Ivan,
> >
> > nacl_register_gc_hooks is defined in libnacl.a.  Unfortunately, that
> > library is really only meaningful with our newlib toolchain.  Thanks for
> > reminding me, I've filed a bug:
> > http://code.google.com/p/nativeclient/issues/detail?id=2635
> >
> > You may want to consider taking a couple more changes to that file (and
> > other parts of libgc that we changed) in order to get something that will
> > build and run with our glibc toolchain.  From September through January,
> a
> > lot of changes went into Mono to make things work cleanly with our glibc,
> > and a decent amount was getting libgc working in the different
> environment.
> >  In fact, I've pretty much dropped support for using this with our newlib
> > version.
> >
> > -Elijah
> >
> > On Mon, Mar 5, 2012 at 7:01 AM, Ivan Maidanski <ivmai-JGs/[email protected]<https://e.mail.ru/sentmsg?mailto=mailto%3aivmai-JGs/[email protected]>>
> wrote:
> >
> > > Hi Elijah,
> > >
> > > I've cherry-picked your latest commit regarding NaCl from mono/libgc to
> > > BDWGC master branch (with minor modifications):
> > >
> > >
> https://github.com/ivmai/bdwgc/commit/14f2760d584c18fc8a1f305f5ed0a6d13ff5918a
> > >
> > > I wonder where is nacl_register_gc_hooks defined?
> > >
> > > Thanks.
> > >
> > > 18 04 2011, 22:31 Elijah Taylor <[email protected]<https://e.mail.ru/sentmsg?mailto=mailto%[email protected]>
> >:
> > > > Hi Ivan,
> > > >
> > > > If I can add a couple of functions to our pthread implementation
> > > > (pthread_getattr_np and pthread_getattr_getstack) then we should be
> able
> > > to
> > > > just use the GC_LINUX_THREADS version and take off the
> "!defined(NACL)".
> > > >  I'll let you know of my progress when I have something to report.
> > > >
> > > > -Elijah
> > > >
> > > > On Sat, Apr 16, 2011 at 1:48 AM, Ivan Maidanski <ivmai-JGs/[email protected]<https://e.mail.ru/sentmsg?mailto=mailto%3aivmai-JGs/[email protected]>>
> wrote:
> > > >
> > > > > Hi Elijah,
> > > > >
> > > > > Could you also provide GC_get_stack_base() implementation for NaCl?
> > > (just
> > > > > for the completeness of the port)
> > > > >
> > > > > Thanks.
> > > > >
> > > > > Sat, 22 Jan 2011 16:23:32 -0800 Elijah Taylor <
> [email protected]<https://e.mail.ru/sentmsg?mailto=mailto%[email protected]>
> > > >:
> > > > >
> > > > > Hi Ivan,
> > > > >
> > > > > Sorry I haven't gotten back to you yet, I've been busy with other
> > > things
> > > > > this last week.  I'm planning on addressing the feedback you've
> given
> > > me so
> > > > > far in the next week, and I can send you a more detailed response
> to
> > > your
> > > > > other questions at that time.  Thanks for the help so far.
> > > > >
> > > > > -Elijah
> > > > >
> > > > >
> > > > > On Sat, Jan 22, 2011 at 9:09 AM, Ivan Maidanski <ivmai-JGs/[email protected]<https://e.mail.ru/sentmsg?mailto=mailto%3aivmai-JGs/[email protected]>
> <
> > > http://sentmsg?compose&To=ivmai-JGs/[email protected]>
> > > > > > wrote:
> > > > >
> > > > >> Hi Elijah,
> > > > >>
> > > > >> Any progress or comments?
> > > > >>
> > > > >> Sat, 15 Jan 2011 15:36:47 +0300 Ivan Maidanski <ivmai-JGs/[email protected]<https://e.mail.ru/sentmsg?mailto=mailto%3aivmai-JGs/[email protected]>
> <
> > > http://sentmsg?compose&To=ivmai-JGs/[email protected]>
> > > > >> >:
> > > > >>
> > > > >> > Hello Elijah,
> > > > >> >
> > > > >> > Fri, 14 Jan 2011 15:36:34 -0800 письмо от Elijah Taylor <
> > > > >> > [email protected]<https://e.mail.ru/sentmsg?mailto=mailto%[email protected]>
> <
> > > http://sentmsg?compose&[email protected]>(sentmsg?compose&To=
> > > > >> [email protected]<https://e.mail.ru/sentmsg?mailto=mailto%[email protected]>
> <
> > > http://sentmsg?compose&[email protected]>)
> > > > >> >:
> > > > >> >
> > > > >> > > Hi Ivan,
> > > > >> > >
> > > > >> > > Thanks for taking a look, I'm pleasantly surprised by the
> level of
> > > > >> detail
> > > > >> > > here.  Specific replies inline:
> > > > >> > >
> > > > >> > >
> > > > >> > > On Fri, Jan 14, 2011 at 2:50 PM, Ivan Maidanski <
> ivmai-JGs/[email protected] <https://e.mail.ru/sentmsg?mailto=mailto%3aivmai-JGs/[email protected]><
> > > http://sentmsg?compose&To=ivmai-JGs/[email protected]>
> > > > >> > (sentmsg?compose&To=ivmai-JGs/[email protected]<https://e.mail.ru/sentmsg?mailto=mailto%3aivmai-JGs/[email protected]>
> <
> > > http://sentmsg?compose&To=ivmai-JGs/[email protected]>)
> > > > >> > wrote:
> > > > >> > >
> > > > >> > > - the patch in  naclports repository contains a typo in a
> macro
> > > > >> definition
> > > > >> > > > (MAC_TYPE -> MACH_TYPE);
> > > > >> > >
> > > > >> > > Oops, will fix.
> > > > >> >
> > > > >> > Why not to leave MACH_TYPE as-is (e.g. "I386", etc.)? NaCl is a
> > > kind of
> > > > >> OS not
> > > > >> > a kind of machine hardware.
> > > > >> > Is eg. I386 defined for x86?
> > > > >> >
> > > > >> > Also, please add a mapping comment in gcconfig.h (around "Feel
> free
> > > to
> > > > >> add
> > > > >> > more clauses here").
> > > > >> >
> > > > >> > >
> > > > >> > > > - the patch in  naclports repository looks more suitable
> for gc
> > > v72
> > > > >> than
> > > > >> > > > that for mono/libgc;
> > > > >> > > >
> > > > >> > >
> > > > >> > > This patch is meant to be applied to vanilla gc6.8.  The
> > > mono/libgc
> > > > >> port is
> > > > >> > > already patched directly into mono's code repository.
> > > > >> >
> > > > >> > Yes, I only meant the vanilla gc6.8 patch contains some more
> code
> > > (eg,
> > > > >> for
> > > > >> > PARALLEL_MARK) not present in mono/libgc.
> > > > >> >
> > > > >> > > > - gc_pthread_redirects.h (which is a public one) should not
> test
> > > > >> NACL
> > > > >> > macro
> > > > >> > > > (or, at least, while less desirable, it should be prefixed
> with
> > > > >> GC_);
> > > > >> > > >
> > > > >> > >
> > > > >> > > Ok, makes sense, I think I didn't realize this was a public
> > > header.
> > > > >>  Though
> > > > >> > > that explains why I had to add a test for __native_client__
> > >  (which is
> > > > >> > > defined in our toolchain).  I'll fix this.
> > > > >> > >
> > > > >> > > > - it's not clear why you need to explicitly undef
> STACK_GRAN,
> > > > >> > USE_M[UN]MAP,
> > > > >> > > > etc. in gcconfig.h;
> > > > >> > > >
> > > > >> > >
> > > > >> > > I'll do some investigation, but IIRC these were needed at one
> > > point
> > > > >> for me.
> > > > >> > > There's a good chance these may be unnecessary and vestigial.
> > > > >> >
> > > > >> > There should none "undef" (no other target undefining them).
> > > > >> >
> > > > >> > If mmap is supported by NaCl then it might be possible to
> support
> > > > >> > USE_M[UN]MAP.
> > > > >> >
> > > > >> > > > - is MPROTECT_VDB supported or not?;
> > > > >> > >
> > > > >> > > Is MPROTECT_VDB equivalent to catching protection violations
> in
> > > the GC
> > > > >> code?
> > > > >> > > If so, then no, we don't support anything like that right now.
> > > > >>  Protection
> > > > >> > > violation in NaCl == instant death.
> > > > >> >
> > > > >> > Ok, so MPROTECT_VDB (i.e., incremental/generation collection)
> is not
> > > > >> > supported.
> > > > >> >
> > > > >> > > > - if you you want to port gc72 please use the recent CVS
> > > snapshot
> > > > >> (it
> > > > >> > would
> > > > >> > > > be easier to me to review and commit it);
> > > > >> > >
> > > > >> > > I've been grabbing source from
> > > > >> > >  http://www.hpl.hp.com/personal/Hans_Boehm/gc/gc_source/
> > > > >> > (http://www.hpl.hp.com/personal/Hans_Boehm/gc/gc_source/)  ...
> can
> > > you
> > > > >> point
> > > > >> > me
> > > > >> > > to where I should be getting the latest?  It's not immediately
> > > obvious
> > > > >> to
> > > > >> > > me.
> > > > >> >
> > > > >> > http://bdwgc.cvs.sourceforge.net/viewvc/bdwgc/bdwgc/
> > > > >> > (For convenience, I have a recent snapshot as a tarball which I
> use
> > > for
> > > > >> my
> > > > >> > project -
> > > > >> >
> > > http://www.ivmaisoft.com/_mirror/hpl/bdwgc-7_2alpha5-20110107.tar.bz2)
> > > > >> >
> > > > >> > > - not sure that HEURISTIC1 really works reliably there (in
> short,
> > > > >> HEURISTIC1
> > > > >> > > > means you treat stack pointer at GC_init call as stack
> bottom -
> > > is
> > > > >> it
> > > > >> > > > guaranteed that GC_init call is always done at higher stack
> > > > >> addresses than
> > > > >> > > > any other GC call);
> > > > >> > > >
> > > > >> > >
> > > > >> > > HEURISTIC2 will definitely not work for us as it wants to use
> a
> > > > >> segfault to
> > > > >> > > detect running over the stack.  I've set STACK_GRAN to 64K,
> so as
> > > long
> > > > >> as
> > > > >> > > the stack doesn't grow beyond that size before GC_init, we
> should
> > > be
> > > > >> ok,as
> > > > >> > > right?  The stack for the main thread right now in NaCl lives
> at a
> > > > >> fixed
> > > > >> > > address usually, but that isn't guaranteed for all future
> time,
> > > so I'd
> > > > >> > > prefer not to hard code magic numbers here.
> > > > >> >
> > > > >> > No, HEURISTIC2 won't work without signals, but there are other
> > > > >> alternatives:
> > > > >> > - if threads-support is on then is it possible to use
> > > > >> > USE_GET_STACKBASE_FOR_MAIN?;
> > > > >> > - is it possible to LINUX_STACKBOTTOM (if we are on real Linux)?
> > > > >> > - for NaCl on Cygwin, it might be possible to use
> > > > >> GC_get_[main_]stack_base
> > > > >> > based on __asm__ ("%fs:4").
> > > > >> >
> > > > >> > > > - is the GC port compilable (and working) on other
> (non-Linux)
> > > > >> platforms
> > > > >> > > > (eg., Cygwin);
> > > > >> > > >
> > > > >> > >
> > > > >> > > Native Client is meant to be portable, so it should run on any
> > > x86 or
> > > > >> x86-64
> > > > >> > > machine once it's built.  In terms of building, I haven't
> built
> > > this
> > > > >> gc port
> > > > >> > > personally on Mac or Windows, but I just checked our build bot
> > > logs
> > > > >> and they
> > > > >> > > seem to be building ok on Mac and in Cygwin.
> > > > >> >
> > > > >> > So, eg. DARWIN, GC_DARWIN_THREADS,  WIN32, CYGWIN,
> GC_WIN32_THREADS
> > > > >> won't ever
> > > > >> > be defined when building NaCl, right?
> > > > >> > Is GC_LINUX_THREADS defined when building NaCl with
> multi-threaded
> > > > >> support?
> > > > >> >
> > > > >> > I have very little knowledge of NaCl - could you briefly explain
> > > what
> > > > >> does
> > > > >> > stand for NaCl portability - is it possible to call Win32 API
> if I'm
> > > > >> compiling
> > > > >> > on Cygwin or should I use the NaCl API (and, thus, the compiled
> > > binary
> > > > >> code
> > > > >> > will run on any x86 target)?
> > > > >> >
> > > > >> > If NaCl is some kind of OS then LINUX, DARWIN, WIN32, etc
> shouldn't
> > > be
> > > > >> defined
> > > > >> > (even if __linux__ defined) if NACL.
> > > > >> > Same for GC_xxx_THREADS - I think GC_NACL_THREADS could be
> defined
> > > > >> instead of
> > > > >> > GC_LINUX_THREADS, etc.
> > > > >> >
> > > > >> > I also think that I386 and X86_64 should stay defined for
> > > respectively
> > > > >> the
> > > > >> > corresponding CPU type (I guess it is already for NaCl but i
> haven't
> > > > >> checked
> > > > >> > yet)
> > > > >> > I think there should be 2 ifdef NACL define OS_TYPE "NACL" ...
> > > sections
> > > > >> (one
> > > > >> > for every supported CPU).
> > > > >> >
> > > > >> > > > - for non-static GC-internal symbols use GC_ prefix (eg. for
> > > > >> > > > nacl_thread_parked);
> > > > >> > > > - define SIG_SUSPEND to -1 (instead of 0) as it is returned
> by
> > > > >> > > > GC_get_suspend_signal;
> > > > >> > > > - GC functions called from NaCl it self (eg,
> > > nacl_pre_syscall_hook)
> > > > >> shoud
> > > > >> > > > be tagged with some attribute (like public GC functions are)
> > > both
> > > > >> for code
> > > > >> > > > readability and to prevent that symbols stripping when
> compiled
> > > as a
> > > > >> > shared
> > > > >> > > > lib with -DGC_DLL);
> > > > >> > > >
> > > > >> > >
> > > > >> > > I'll address these issues.  (note that NaCl currently doesn't
> > > support
> > > > >> shared
> > > > >> > > libs yet so your dll example won't happen, but I agree that
> these
> > > > >> should be
> > > > >> > > treated like other public GC functions)
> > > > >> >
> > > > >> > Ok. But what is eg. nacl_pre_syscall_hook() - a callback from
> the
> > > NaCl
> > > > >> > subsystem? (I guess this should be treated as GC public API)
> > > > >> >
> > > > >> > Of course, use STATIC or static where possible (all STATIC
> symbols
> > > start
> > > > >> with
> > > > >> > GC_, while static typically not).
> > > > >> > More tips: use GC_INNER and GC_EXTERN for internal global
> > > variables; use
> > > > >> > GC_INNER for internal functions.
> > > > >> >
> > > > >> > > > - libatomic_ops does not use signals API (except for CAS
> > > emulation
> > > > >> which
> > > > >> > is
> > > > >> > > > not used for x86/x64).
> > > > >> > >
> > > > >> > > I think I saw sigprocmask and related functions and assumed
> the
> > > worst,
> > > > >> but I
> > > > >> > > see now that's windows code.  Looking at the x86 variants it
> looks
> > > > >> like a
> > > > >> > > NaCl port of libatomic_ops is probably not going to be too
> bad.
> > >  I'll
> > > > >> look
> > > > >> > > into this eventually.
> > > > >> >
> > > > >> > Most probably, it work w/o any porting afforts but it would be
> good
> > > to
> > > > >> port
> > > > >> > atomic_ops.c (similar to what I did for Win32-pthreads targets
> - see
> > > > >> > AO_USE_WIN32_PTHREADS, I guess you should add AO_USE_NACL macro
> > > testing
> > > > >> in
> > > > >> > that file (looks easy to add). I think it's worth doing first
> (and
> > > > >> submit me a
> > > > >> > separate patch for libatomics_op when done).
> > > > >> >
> > > > >> > What's about GC_HAVE_BUILTIN_BACKTRACE and
> GC_CAN_SAVE_CALL_STACKS?
> > > At
> > > > >> least,
> > > > >> > gc.h should be consistent with the GC implementation (I mean
> eg. if
> > > > >> > GC_HAVE_BUILTIN_BACKTRACE not supported then it shouldn't be
> > > defined in
> > > > >> gc.h
> > > > >> > regardless of __linux__, _MSC_VER, etc. provided
> > >  __native_client__).
> > > > >> Same for
> > > > >> > GC_ADD_CALLER, GC_RETURN_ADDR.
> > > > >> >
> > > > >> > Regards.
> > > > >> >
> > > > >> > > > PS. Let me not do the benefits analysis (probably someone
> else
> > > can
> > > > >> do
> > > > >> > > > this).
> > > > >> > > >
> > > > >> > > Well, if the gc7.2 port is as easy as it's looking now, I
> think
> > > it's
> > > > >> > > probably worth doing it.  I would still love to hear anyone
> chime
> > > in
> > > > >> on the
> > > > >> > > benefits of gc7.2 vs 6.8 though
> > > > >> > >
> > > > >> > > > Regards.
> > > > >> > > >
> > > > >> > > > Thu, 13 Jan 2011 10:21:03 -0800 Elijah Taylor <
> > > > >> [email protected]<https://e.mail.ru/sentmsg?mailto=mailto%[email protected]>
> <
> > > http://sentmsg?compose&[email protected]>
> > > > >> > (sentmsg?compose&[email protected]<https://e.mail.ru/sentmsg?mailto=mailto%[email protected]>
> <
> > > http://sentmsg?compose&[email protected]>)
> > > > >> >:
> > > > >> > > >
> > > > >> > > > > Hi GC folks,
> > > > >> > > >
> > > > >> > > > > I saw a little chatter in the archives related to porting
> > > libgc to
> > > > >> > Native
> > > > >> > > > Client, so I joined this list to share some details. I'm the
> > > > >> engineer at
> > > > >> > > > Google who ported of libgc to Native Client for Mono. I've
> also
> > > > >> included a
> > > > >> > > > patch for vanilla gc6.8 in our naclports repository:
> > > > >> > > >  http://code.google.com/p/naclports/ (
> > > > >> http://code.google.com/p/naclports/)
> > > > >> > . This version will be available to
> > > > >> > > > users that want to use libgc as part of their Native Client
> > > > >> projects.
> > > > >> > > >
> > > > >> > > > > Before porting gc6.8 I had attempted to port one of the
> newer
> > > > >> versions,
> > > > >> > > > gc7.2alpha4, but ran into snags. The largest snag right now
> I
> > > think
> > > > >> is
> > > > >> > that
> > > > >> > > > gc 7+ includes libatomic_ops which will require some
> non-trivial
> > > > >> effort in
> > > > >> > > > order to work under Native Client. Most notably we don't
> support
> > > > >> signals;
> > > > >> > > > that was the biggest effort in porting libgc in the first
> place
> > > for
> > > > >> NaCl,
> > > > >> > > > and I assume that will require the most work in porting
> > > > >> libatomic_ops too.
> > > > >> > > >
> > > > >> > > > > Can someone give me the high level details of what kind of
> > > things
> > > > >> we
> > > > >> > > > might be missing if we only support gc6.8 instead of the
> latest
> > > > >> version?
> > > > >> > > > Because of our thread stopping implementation, we may not
> even
> > > > >> benefit
> > > > >> > from
> > > > >> > > > some of the newer features. I just wanted to get a sense of
> > > what the
> > > > >> > > > benefits are of getting a newer version available for users.
> > > > >> > > >
> > > > >> > > > > -Elijah
> > > > >>
> > > > >>
> > > > >
> > > > >
> > > >
> > > >
> > >
> >
> >
>
>
>
> --
> Иван Майданский
>

_______________________________________________
Gc mailing list
Gc-V9/[email protected]
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/