Re: [Sbcl-bugs] Memory leaks in weak hash tables

Weak tables don't generally suffer from leaks, but I would not expect this
particular test of yours to pass for a few reasons-

- The only allocations performed in the test are table insertions, so the
table itself, plus the conses, will consume the entire nursery space before
the garbage collector decides to run. This is not an anticipated use-case
for weak tables. As the table has unrestricted use of the nursery, its
backing vector will grow to about 85 megabytes in the default heap size of
1Gb.
- No SBCL hash-table will ever shrink its backing vector. So after this
table reaches a stable size (at which the GC can keep up with removing
items), it's just going to stay there and never shrink. This is a fairly
common implementation detail of hash-table algorithms.
- You are cycling objects too quickly for the GC's default promotion rate
to deal effectively with the test.  It's not the table that eats up all
heap, it's the cons cells themselves that are sometimes getting promoted to
older generations and then making an GC  entire page unusable.  1 cons is
all it takes for an old space page not to allow any new objects onto it.

So this essentially comes down to a GC tuning issue. Increasing
number-of-generations-before-promotion to 10 will behave much more nicely
with this test.
With the change below, I watched it run for about 19 minutes.  It will
still exhaust the heap.  (It is a curious that the memory report shows
little waste, so I think that statistic is actually wrong.). A workaround
may be to occasionally request a full GC,  if your application makes such
intensive use of weak tables.


(setf (generation-number-of-gcs-before-promotion 0) 10)

(defglobal my-hash-table (make-hash-table :weakness :key))

(defun test-weak-hash-table ()

  (loop (setf (gethash (cons nil nil) my-hash-table) t)))

(defglobal start (get-internal-real-time))

(defun show-htsize ()

  (format t " after ~D sec hash-table has ~d entries, size=~D, mem=~D~%"

          (floor (- (get-internal-real-time) start)
internal-time-units-per-second)

          (hash-table-count my-hash-table)

          (hash-table-size my-hash-table)

          (primitive-object-size (sb-impl::hash-table-pairs
my-hash-table))))

(compile 'show-htsize)

(push 'show-htsize *AFTER-GC-HOOKS*)

(test-weak-hash-table)
..

 after 1172 sec hash-table has 25 entries, size=5347737, mem=85563840

Heap exhausted during garbage collection: 16 bytes available, 32 requested.

     | Immobile Objects |


 As for shrinking the table: the table thinks it is doing what you want by
staying large in its physical representation.

_______________________________________________
Sbcl-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sbcl-devel