Re: Function to force table to shrink?

Glad you liked it!

Sorry if I made it sound like this hack can "force" a rehash. It can 
not. It can only "allow" it, meaning a space-conscious Lua 
implementation would be allowed to rehash without worrying about 
breaking iteration.

I was under the impression that PUC Lua does actually shrink tables 
sometimes, at least the hash part, but I will check the sources and 
test again (unless the authors would like to answer off the top of 
their heads ;)) so I can make some reasonable statements about what is 
likely to happen and when.

By the way, as for the discussions for how Lua could be extended: I 
think a "table.rehash" function (name TBD) which shrinks the table 
would not be unreasonable, and not dissimilar to existing features in 
many other languages, like C++'s std::vector::shrink_to_fit (though 
that shrinks a bit more aggressively than we would like in Lua). LuaJIT 
already has a "table.clear", but this is explicitly supposed to keep 
capacities intact.

An extension of collectgarbage would feel a bit dirty.

- Lars

On Thu, Jan 29 2026 at 19:48:43 +02:00:00, 'Martin Eden' via lua-l 
<[email protected]> wrote:
> On 2026-01-29 16:38, 'Lars Müller' via lua-l wrote:
>> This is an unfortunate consequence of the requirement that items can 
>>  be deleted from a table while it is iterated. This means that you 
>> more  or less need to avoid a rehash, because that would change the 
>>  iteration order while you are iterating.
>> 
>> A trick to permit a (shrinking) rehash is to rawset a *new* entry, 
>>  e.g. using a fresh table as key (by reference), then deleting that 
>>  entry afterwards. But this need not work, it depends on whether the 
>>  implementation decides to shrink.
>> 
>> I'm also not a fan of this behavior because it means that the 
>> "garbage  collection" is unintuitive: There is no guarantee that 
>> after a full  collection, the memory usage is in any way 
>> proportional to the actual  observable size of the data being stored.
>> 
>> The solution Lua currently wants you to use is to replace the table 
>> if  you drop most of its contents and need the memory back. 
>> Alternatively,  Lua could fix this, even while allowing the deletion 
>> of elements  during iteration, but a more sophisticated, likely less 
>> efficient  table representation would be required for this.
>> 
>> I wrote at a bit more length about this in a blog post: 
>>  <<https://luatic.dev/posts/2025-04-12-lua-misconceptions/#setting-elements-to-nil-frees-up-space>> 
>>  
>> 
>> - Lars
> Thanks for link to your essay Lars, I enjoyed reading it much!
> 
> However I don't understand how and why your proposed hack can force
> table rehash:
> 
>   local allow_rehash
>   do
>     local key = {} -- can't be in the table, because table equality 
> is by reference
>     function allow_rehash(t)
>       rawset(t, key, true) -- following this, lua is allowed to 
> rehash t
>       rawset(t, key, nil) -- restore t to original state
>     end
>   end
> 
> Tried it and saw no effect.
> 
> And I share opinion that collectgarbage() is not intuitive. It does
> nothing for tables with nils occupying all memory.
> 
> I would join to Rett's proposal for table 
> garbage-collect/shrink/rehash
> function.
> 
> Maybe add one more ad-hoc string argument to collectgarbage()?
> 
>   collectgarbage('table', t)
> 
>   Forces rehash of table <t>. Frees memory by not storing empty slots.
>   Side effect is keys sequence reordering so do not mix with 
> next()/pairs().
> 
> -- Martin
> 
> --
> You received this message because you are subscribed to the Google 
> Groups "lua-l" group.
> To unsubscribe from this group and stop receiving emails from it, 
> send an email to [email protected] 
> <mailto:[email protected]>.
> To view this discussion visit 
> <https://groups.google.com/d/msgid/lua-l/cb1ab78e-3b0e-49c6-8332-541f4ea424aa%40disroot.org>.

-- 
You received this message because you are subscribed to the Google Groups "lua-l" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion visit https://groups.google.com/d/msgid/lua-l/UYJN9T.RDPY21GF7CUR3%40gmx.de.