Re: Problem with storage of large data

Dear E.D.,

You should
1/ use readvec for large data
2/ not perform operations in data files (in particular, do not store 
polynomials as sums of products coef * x^i)

Example:

\\fake data
nbpol = 700;
dgpol = 71;
lgpol = dgpol+1;
ncoefs = nbpol*lgpol;
vN = vector(ncoefs,i,random(10^10000));
vD = vector(ncoefs,i,random(10^10000));
for(i=1,ncoefs,write("fileN",vN[i]));
for(i=1,ncoefs,write("fileD",vD[i]));

\\read data
vN = readvec("fileN");
vD = readvec("fileD");
vrat = vector(ncoefs,i,vN[i]/vD[i]);
vpol = vector(nbpol,i,Pol(vrat[lgpol*(i-1)+1 .. lgpol*i]));

\\basic sanity check
Set(apply(poldegree,vpol))

Maybe a file in readvec format with each line being Pol([ list of 
coefficients as rational numbers ]) would also be ok, I have not tried.

This could also be made more efficient in library mode (avoid copies, do 
not compute gcd between numerator and denominator, etc).

Cheers,
Aurel

On 25/08/2025 11:12, Ewan Delanoy wrote:
> I've got a large (1.2 GB, around 100000 lines) gp file that the GP 
> interpreter cannot read - when I call `read` on that file the 
> interpreter, after a few minutes, prints "Killed" and exits suddenly.
>
> The issue is only one of file size, because the operations in the file 
> are very elementary - it only uses addition, multiplication and 
> accessing/setting elements of an array.
>
> My intent was to store the results of somewhat long computations. The 
> values involved are all univariate polynomials, and the heavy core of 
> the data is a set of about 700 polynomials of degree 71, whose 
> coefficients are rationals whose numerators and denominators have up 
> to 10000 digits.
>
> When I simply and naively use the `write` function to put all this 
> data into a file, I get this
> large GP-unreadable file.
>
> Any ideas on how to make this storage work ?
>
> E. D.
>