Re: Thread Local Storage?

Your misconception appears to be that the GC allocates memory in the area
you told it about with GC_add_roots().

It does not. The GC only allocates memory in the GC heap. Roots areas are
read-only for the GC. When you use GC_add_roots(), you are telling the GC
that *you* will be putting pointers to memory allocated by the GC (in the
GC heap) in that roots area.


On Mon, Oct 21, 2013 at 8:19 AM, The Devils Jester <
[email protected]> wrote:

> That is exactly how I thought add_roots worked, so I must be missing
> something.  Let me explain my understanding of the process, and we can see
> where I am going astray.
>
> The GC will collect from areas it "knows" about, (i.e. roots).  In my
> situation it is likely that it does not know to collect from the location
> that the TLS is placed.  I would use add_roots to tell the GC to consider
> this area too.  I would then, before the thread ends, remove_roots to no
> longer look at this area?
>
> If my above summary is correct, I come back to my previous question.  If I
> tell the GC to look at the TLS area for collection (add_roots) when I
> create a new thread, but then I tell it to no longer consider that area
> (remove_roots) when I end the thread, but I do this before the data that
> was created in that area was collected (so the GC is no longer looking
> there, but I still have unfree'ed data there), then would this data not
> persist until the completed thread free's its resources?
>
> If my summary is off, which is likely considering the complex nature of
> this process, what do I have wrong?
>
>
> On Sat, Oct 19, 2013 at 5:00 PM, Bruce Hoult <[email protected]> wrote:
>
>> Roots are areas of memory that the GC searches for pointers at the start
>> of GCing, even though it doesn't have pointers to THEM and they may not be
>> in the GC heap. You always should have as roots the CPU registers, the
>> global variables for the main program and hopefully any libraries, and the
>> stacks of every thread. The GC hopefully finds all of those automatically
>> (though DLLs and thread stacks can be hard to find on some platforms)
>>
>>
>> On Sun, Oct 20, 2013 at 10:52 AM, The Devils Jester <
>> [email protected]> wrote:
>>
>>> I know what the GC does, I think its confusion about what add_roots does.
>>> On Oct 19, 2013 4:48 PM, "Bruce Hoult" <[email protected]> wrote:
>>>
>>>> There seems to be some confusion about what a GC is.
>>>>
>>>> The GC will collect and reuse *everything* unless it can see that it is
>>>> still wanted because it found a pointer to it.
>>>>
>>>> The reason to use GC_add_roots() is so that the GC can see your
>>>> pointers and won't reuse memory you allocated using the GC. If you don't
>>>> call GC_add_roots(), or after you call GC_remove_roots(), the GC is
>>>> free to reuse GC memory pointed to from your TLS. That would be bad if you
>>>> were still using it.
>>>>
>>>>
>>>> On Sat, Oct 19, 2013 at 7:42 AM, The Devils Jester <
>>>> [email protected]> wrote:
>>>>
>>>>> Since a GC can happen at any arbitrary time, what happens if I use
>>>>> GC_add_roots on my TLS at the start of the thread, and then GC_remove_roots
>>>>> right before the thread exits, but the GC has not collected the data yet?
>>>>>  Will the resources continue to pile up until the thread has had its
>>>>> resources released?
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Oct 18, 2013 at 1:35 PM, Bruce Hoult <[email protected]> wrote:
>>>>>
>>>>>> Yes, of course, that's the reason GC_remove_roots() exists.
>>>>>>
>>>>>> What do you mean "even if GC has not occurred"? That's irrelevant.
>>>>>>
>>>>>>
>>>>>> On Sat, Oct 19, 2013 at 1:11 AM, The Devils Jester <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> The number of threads created is not a known value, but I suspect it
>>>>>>> will often be very low with longer life spans.  My question is more about
>>>>>>> accumulation.  If I create thousands (or hundreds of thousands) of threads
>>>>>>> over the lifetime of the program, even if this is days, weeks, or months,
>>>>>>> will the performance be impacted by all of the registered areas?  Would it
>>>>>>> be safe to 'unregister' an area when the thread is ended, even if GC has
>>>>>>> not occurred?
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Oct 18, 2013 at 12:42 AM, Bruce Hoult <[email protected]>wrote:
>>>>>>>
>>>>>>>> That all depends on how often you are creating and deleting threads.
>>>>>>>>
>>>>>>>> If you are creating and deleting thousands of threads between each
>>>>>>>> GC then 2) will be lower impact. If threads live for many seconds or
>>>>>>>> minutes then 1) will be lower impact.
>>>>>>>>
>>>>>>>> If you only have a few dozen threads then performance in this area
>>>>>>>> is probably irrelevant.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Oct 18, 2013 at 6:29 PM, The Devils Jester <
>>>>>>>> [email protected]> wrote:
>>>>>>>>
>>>>>>>>> The platform that I am currently experiencing the issue on is
>>>>>>>>> Ubuntu Linux 64bit 13.04, using GCC 4.8.
>>>>>>>>>
>>>>>>>>> If I just have a few pointers to some containers, would it be safe
>>>>>>>>> to simply add those pointers to the 'roots'?
>>>>>>>>>
>>>>>>>>> You say that there might be performance problems with approach 1?
>>>>>>>>> I only have a few TLS pointers, do you think this would have any kind of
>>>>>>>>> performance impact?  Since I (re)instantiate these pointers for every
>>>>>>>>> thread, I would assume that I would have to "mark" every instance of these
>>>>>>>>> pointers?  Would the performance hit start to grow, then, as new threads
>>>>>>>>> are created and points are marked?
>>>>>>>>>
>>>>>>>>> On a slightly related note, your approach 2 has me wondering if
>>>>>>>>> there is some other generic call back (not specific to roots) that I can
>>>>>>>>> set for before (and maybe after) a GC occurs.  (It would help debug things
>>>>>>>>> that happen specifically at a GC).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Oct 18, 2013 at 12:09 AM, Bruce Hoult <[email protected]>wrote:
>>>>>>>>>
>>>>>>>>>> Fundamentally, someone needs to make sure that the thread local
>>>>>>>>>> storage is added to the root areas the GC scans pointers from.
>>>>>>>>>>
>>>>>>>>>> Ideally, the standard GC would do that. I don't know what your
>>>>>>>>>> system is and therefore whether it is practical for the GC to locate the
>>>>>>>>>> TLS global areas.
>>>>>>>>>>
>>>>>>>>>> But you can do it yourself.
>>>>>>>>>>
>>>>>>>>>> There are two approaches:
>>>>>>>>>>
>>>>>>>>>> 1) for fixed size areas (or even individual pointer variables)
>>>>>>>>>> you can call GC_add_roots(low_address, high_address) on as many areas as
>>>>>>>>>> necessary. You can also remove these areas if necessary. Or remove and
>>>>>>>>>> re-add if they change size. See gc.h
>>>>>>>>>>
>>>>>>>>>> 2) for things that change often during execution, it can be
>>>>>>>>>> better to point GC_push_other_roots at your own function, which will be
>>>>>>>>>> called near the start of every GC. Your function should call e.g.
>>>>>>>>>> GC_push_all(bottom, top) on each appropriate area you want the GC to scan
>>>>>>>>>> from. Make sure you save the old value of GC_push_other_roots before
>>>>>>>>>> installing your function, and call it too. This is all defined in
>>>>>>>>>> include/private/gc_priv.h
>>>>>>>>>>
>>>>>>>>>> Unless you see performance problems, it's safer to stick to
>>>>>>>>>> approach 1).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Oct 18, 2013 at 5:20 PM, The Devils Jester <
>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>
>>>>>>>>>>> I have a couple of stl containers that hold pointers to classes
>>>>>>>>>>> which inherit from gc.  The containers are created via a template that
>>>>>>>>>>> includes the gc_allocator.  This has worked great for quite awhile, however
>>>>>>>>>>> I have recently made a couple of the containers thread local using
>>>>>>>>>>> __thread.  When I do this, however, (while still in my main thread and
>>>>>>>>>>> never having created a new thread), at some arbitrary point (maybe when GC
>>>>>>>>>>> kicks in?) my container size goes from whatever it was, to zero.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I can remove the __thread from the containers and everything
>>>>>>>>>>> works as desired, for an indefinite period of time, but adding __thread
>>>>>>>>>>> back in, always causes the same issue.  Is there some special
>>>>>>>>>>> initialization I need to do to allow for __thread (and similar) to work
>>>>>>>>>>> correctly in the GC?
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> This message has been scanned for viruses and
>>>>>>>>>>> dangerous content by *MailScanner*<http://www.mailscanner.info/>,
>>>>>>>>>>> and is
>>>>>>>>>>> believed to be clean.
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Gc mailing list
>>>>>>>>>>> Gc-V9/[email protected]
>>>>>>>>>>> http://www.hpl.hp.com/hosted/linux/mail-archives/gc/
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> This message has been scanned for viruses and
>>>>>>>>> dangerous content by *MailScanner* <http://www.mailscanner.info/>,
>>>>>>>>> and is
>>>>>>>>> believed to be clean.
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Gc mailing list
>>>>>>>>> Gc-V9/[email protected]
>>>>>>>>> http://www.hpl.hp.com/hosted/linux/mail-archives/gc/
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> This message has been scanned for viruses and
>>>>>>> dangerous content by *MailScanner* <http://www.mailscanner.info/>,
>>>>>>> and is
>>>>>>> believed to be clean.
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> This message has been scanned for viruses and
>>>>> dangerous content by *MailScanner* <http://www.mailscanner.info/>,
>>>>> and is
>>>>> believed to be clean.
>>>>>
>>>>> _______________________________________________
>>>>> Gc mailing list
>>>>> Gc-V9/[email protected]
>>>>> http://www.hpl.hp.com/hosted/linux/mail-archives/gc/
>>>>>
>>>>
>>>>
>>> --
>>> This message has been scanned for viruses and
>>> dangerous content by *MailScanner* <http://www.mailscanner.info/>, and
>>> is
>>> believed to be clean.
>>>
>>
>>
>
> --
> This message has been scanned for viruses and
> dangerous content by *MailScanner* <http://www.mailscanner.info/>, and is
> believed to be clean.
>

_______________________________________________
Gc mailing list
Gc-V9/[email protected]
http://www.hpl.hp.com/hosted/linux/mail-archives/gc/