Re: On the speed of jarred BootCP

On Wed, 2004-10-06 at 19:01, Rene Wagner wrote:
> Hi Grzegorz,

Hi Rene,

sorry for late reply.  Busy week and weekend might be worse :)

> On Sun, 2004-09-26 at 18:07, Grzegorz B. Prokopski wrote:
> > I have patches (not in SVN) that do 1) but have not played with 2) yet
> > and most probably won't have time ATM.  If anyone is interested in
> > getting it finished (by splitting bootcp in two parts in svm-cp) the
> > LGPLed patches against staging are at:
> > 	http://gadek.homelinux.org/bootcp
> 
> I've applied your patches against staging and have the splitting part
> mostly working here.
> 
> A few gotchas:
> a) zip on my system creates v2.0 zip archives which don't work with
>    the zip loader (took me some time and a lot of fprintf's to figure
>    that out).

Yes, I've noticed that, forgot to mention.  I didn't get as far as
gdb-ing the problem though so could you please tell me at what 
statment/line it fails actually?  If it's not about the compression
itself (which we handle by zlib, so we're bound to what it supports)
but i.e. about where some data are stored I might be able to fix it.
The pain is that ZIP usually has 2-3 places where some info might be
stored, from which I only implemented one, i.e. and no zip64 support.

I also want to improve SableVM's error reporting.  This should be
straightforward, as *I* see it from perspective of an idea I have,
but I want to consult it with Etienne first, who, unfortunatelly,
I haven't catched today.

> b) the sablevm-classpath .jar's end up in
>    $prefix/share/sablevm/sablevm-classpath instead of
>    $prefix/share/sablevm/ (as we agreed on on IRC)
> 
> The solution to a) is using fastjar, but I haven't got to adjusting
> the configury to use that instead of zip yet.

Yes, we should be using fastjar IMO, unless we fix the zip problem.
Or maybe there's some switch to zip that tells it to use less
sophisticated storage methods?  But we should support fastjar anyway.

> b) is caused by this part of configure.ac:
> ---snip---
> datadir='${prefix}/share/sablevm'
> PACKAGE=sablevm-classpath
> ---snip---
> 
> One solution would be to change that to
> ---
> datadir='${prefix}/share/

Isn't this exactly the default value?

> PACKAGE=sablevm
> ---

Yes, data are installed in pkgdatadir = $(datadir)/$PACKAGE
PACKAGE could be "sablevm-classpath" for now, so it'd end up in
/usr/share/sablevm-classpath which also sounds natural.  I can't
remember why we wanted to have everything in /usr/share/sablevm?
I don't think there's any really good reason.  All that would need
to be changed is just some paths in src/libsablevm/Makefile IIRC.

> and to adjust the parts of the build system that "manually" install
> the (uncompressed) classpath/docs to
> ${prefix}/share/sablevm/sablevm-classpath.

Actually the real fix would be to finally merge sablevm and
sablevm-classpath.  But ATM I'd rather see 1.1.7 out ASAP.
We'll come back to the merging issue as soon as release is done.

> Also, I'm not sure whether switching to "libclasspath.jar" (as requested
> by Grzegorz) and "resources.jar" instead of "glibj.zip" requires
> adjusting the configure options accordingly, --with-glibj=zip might
> be a bit confusing...

Forget about that option.  It's commented out and we won't be using it.
We want something like:
	--enable-compressed-classes

By default it should try to find fastjar, and if it's not available
find zip (and bomb out if none of them is found).  Note that we
*may* provide =something suffix to --enable-* parameters, so this
would be possible:

	--enable-compressed-classes=zip

And we *might* support that *if* we wanted to allow somebody to force
usage of zip instead of fastjar.  BUT I do NOT think it's necessary.
Autodetection with preference of fastjar over zip should do just fine.


There's one thing that would be nice to support though, sth. like:

	--enable-compressed-classes=storeonly

which would tell fastjar or zip not to compress the files in .jar.
This IMO should actually be the default :-)  This way we could save
on a) disk space b) startup time.  And some software (ANT comes to mind)
really expects classpath to consist of zip/jar files so it's much
better to have bootclasspath in jarred form even if we care for
startup time.

> I'd be happy to sort the remaining bits out but obviously need some
> input first.

So to summarize my suggestions:

* Remove "datadir='${prefix}/share/sablevm'" and let sablevm-classpath
  install its data in default location.  So we'd have libclasspath.jar
  and properties.jar in i.e. /usr/share/sablevm-classpath which sounds
  just about right.
* Add --enable-compressed-classes handling using fastjar and zip
  (and if you could implement =storeonly that'd be just perfect!)
* Let me know where our zip handling fails when you compress with zip.
* After all that is done - check startup time of a HelloWorld that does
  String.split() 1) when  the compressed classpath IS split into
  libclasspath and resources, 2) when everything is in
  libclasspath.jar, 3) when the classpath IS split into libclasspath
  and resources but in =storeonly mode 4) bootclasspath consists of
  uncompressed .class files tree (like we have it right now).

I cut the ideas/steps into small pieces, but I hope you're not afraid
of the number of items :-)  I would love to make release on monday
evening that has "the right" kind of compressed bootclasspath
handling included.  Do you think you can make it work, like, by the end
of the weekend?

If you have any more questions please ask.  I'll do my best to deliver
you answers in a more timely manner.

Thanks a lot for all your effort,

				Grzegorz B. Prokopski

-- 
Grzegorz B. Prokopski      <[email protected]>
Debian GNU/Linux           http://www.debian.org
SableVM - LGPL'ed Java VM  http://www.sablevm.org
Why SableVM ?!?            http://sablevm.org/wiki/Features