Re: Prolog Memory Problem

Dear Joe,

On 02/16/2014 01:30 PM, atom meta wrote:
> Hi,
> 
> I am using Prolog to reason Semantic Web data. I have a problem with
> loading a large volume of data into Prolog. My machine has 64GB RAM and the
> data size is about 10GB but when I load this data to Prolog, it becomes
> about 58GB in Prolog memory. I don't understand why it happens?
> Can anyone help me with this?

The size comparison between data on file and in memory depends a lot on
the file format and various properties about the data.  If you run

	?- rdf_statistics(S).

and backtrack over the results, you get some insight on the data.  A
triple requires 96 bytes on a 64-bit machine.  This excludes a lot of
stuff though: each resource and each literal also take space and there
is also space required for the arrays of hash buckets, which depends
on the indexes that you use as well as the distribution of the
data.

Hope this helps understanding where the space goes.

With 64Gb, you should be able to handle 200-300 million triples,
depending in the characteristics of the data.

	Cheers --- Jan