[ruby-core:95875] [Ruby master Bug#16352] Marshal limit of >= 2 GiB
| Newsgroups | gmane.comp.lang.ruby.core |
|---|---|
| Message-ID | <redmine.issue-16352.20191118113902.e0c4f6d5e0e9f21e@ruby-lang.org> |
Issue #16352 has been reported by seoanezonjic (Pedro Seoane).
----------------------------------------
Bug #16352: Marshal limit of >= 2 GiB
https://bugs.ruby-lang.org/issues/16352
* Author: seoanezonjic (Pedro Seoane)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
* ruby -v: ruby 2.7.0dev (2019-11-12T12:03:22Z master 3816622fbe) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
Hi
Using a gem to handle matrix operations called Numo-array I found the following error when save large matrix:
in `dump': long too big to dump (TypeError)
Github thread: https://github.com/ruby-numo/numo-narray/issues/144
Digging with the authors, we found the following code that reproduces the error:
ruby -e 'Marshal.dump(" "*2**31)'
Executed in :
ruby 2.7.0dev (2019-11-12T12:03:22Z master 3816622fbe) [x86_64-linux]
The marshal library has a limit that is checked with the SIZEOF_LONG constant. This check is performed in this line https://github.com/ruby/ruby/blob/e7ea6e078fecb70fbc91b04878b69f696749afac/marshal.c#L301 to 321 of the Marshal.c file. I don't understand the motivation of this limit and has a great impact in libraries that need to serialize large objects as numeric matrix. In this case, the limit of >= 2 GiB it's reached easily and it blocks the ruby development in scientifical projects as cited. I found other bug related: #1560, but the Marshal problem itself was not addressed in this case.
Thank you in advance
PEdro Seoane
--
https://bugs.ruby-lang.org/
Unsubscribe: <mailto:[email protected]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>