Re: Over 1.5GB file limit?

--00032555988a82fa6a049f918453
Content-Type: text/plain; charset=ISO-8859-1

I am planning on running metakit on some amazon ec2 servers, so I can choose
32bit or 64bit linux.

So for a 32bit linux server, plan on a max filesize of around 2GB, and on a
64bit linux server would be ____GB ?

--
Thadeus




On Mon, Mar 28, 2011 at 3:30 PM, Pat Thoyts <[email protected]> wrote:

> On 28 March 2011 21:13, Thadeus Burgess <[email protected]> wrote:
> > What happens when the file goes over 1.5GB ?
> >
> > In my testing, generating random data to provide a metakit file of about
> 2GB
> > (about 11milliion records). Metakit seems to be able to open and parse
> the
> > file as normal.
> >
> > Is there any specific reason the GB file limit has been placed?
> >
> Metakit memory maps the whole database file in one single view. So on
> 32 bit systems the total addressable memory tends to be 2GB (this is
> certainly the case for Windows) so taking into account your process
> plus any dll's loaded you are likely to only be able to map in a file
> of about 1.5 GB. Metakit itself permits larger files. There are more
> than 32 bits available to hold the size of the data file (its not a
> 64bit number - its a 32bit + some bits from the next slot - I forget
> how many).
> On a 64 bit system with a 64 bit process you should be able to address
> more space.
> Its not clear to me what limit you are discussing here - but its
> likely just the Windows 32bit process addressing limit.
>

-- 
You received this message because you are subscribed to the "metakit" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [email protected]
For more options, visit this group at http://groups.google.com/group/metakit?hl=en
--00032555988a82fa6a049f918453
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I am planning on running metakit on some amazon ec2 servers, so I can choos=
e<br>
32bit or 64bit linux.<br>
<br>
So for a 32bit linux server, plan on a max filesize of around 2GB, and on a=
<br>
64bit linux server would be ____GB ?<br><br clear=3D"all">--<br>Thadeus<br>=
<br><br>
<br><br><div class=3D"gmail_quote">On Mon, Mar 28, 2011 at 3:30 PM, Pat Tho=
yts <span dir=3D"ltr">&lt;<a href=3D"mailto:[email protected]">patthoyts@=
gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=
=3D"margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); p=
adding-left: 1ex;">

<div><div></div><div class=3D"h5">On 28 March 2011 21:13, Thadeus Burgess &=
lt;<a href=3D"mailto:[email protected]">[email protected]</a>&gt; w=
rote:<br>
&gt; What happens when the file goes over 1.5GB ?<br>
&gt;<br>
&gt; In my testing, generating random data to provide a metakit file of abo=
ut 2GB<br>
&gt; (about 11milliion records). Metakit seems to be able to open and parse=
 the<br>
&gt; file as normal.<br>
&gt;<br>
&gt; Is there any specific reason the GB file limit has been placed?<br>
&gt;<br>
</div></div>Metakit memory maps the whole database file in one single view.=
 So on<br>
32 bit systems the total addressable memory tends to be 2GB (this is<br>
certainly the case for Windows) so taking into account your process<br>
plus any dll&#39;s loaded you are likely to only be able to map in a file<b=
r>
of about 1.5 GB. Metakit itself permits larger files. There are more<br>
than 32 bits available to hold the size of the data file (its not a<br>
64bit number - its a 32bit + some bits from the next slot - I forget<br>
how many).<br>
On a 64 bit system with a 64 bit process you should be able to address<br>
more space.<br>
Its not clear to me what limit you are discussing here - but its<br>
likely just the Windows 32bit process addressing limit.<br>
</blockquote></div><br>

<p></p>

-- <br />
You received this message because you are subscribed to the &quot;metakit&q=
uot; group.<br />
To post to this group, send email to [email protected]<br />
To unsubscribe from this group, send email to metakit-unsubscribe@googlegro=
ups.com<br />
For more options, visit this group at <a href=3D"http://groups.google.com/g=
roup/metakit?hl=3Den">http://groups.google.com/group/metakit?hl=3Den</a>
--00032555988a82fa6a049f918453--