Re: Question about memory size and performance

Peter Booth wrote:
> Let me be the dissenter here. I think it sounds like a smart solution
> thats looking for a problem. Often it seems that any smart sounding
> optimization turns out to be sub-optimal in a different circumstance.
> Yes all modern OSes have virtual memory. But I don't think I have seen a
> production application deployed where the process RSS/maximum memory
> footprint + baseline system memory consumption  > physical memory.
> Certainly not in the past seven years. Perhaps I have just been
> fortunate in working for clients with too much cash? But if page
> scanning is never occurring then this problem is moot.

It is.

But people seems to always think that Java is a server side system ONLY.

It is not. In fact, one currently tries to position java as a viable
Desktop alternative these days ("Desktop Java", anyone?). In this case,
it does matter hugely, as I believe I mentioned in those earlier posts.

>
> Jeff mentioned that Hotspot's GC effectiveness depends on memory access
> patterns. The other point is that different applications need to
> optimize for different performance variables. For example, in an
> automatic trading system it might be crucial that a full GC never cause
> execution to pause for seconds. Rather than building connections between
> VMs and kernels, there are crude, ugly ways to achieve this by reducing
> heap size (multiple smaller JVMs on a single host) or even ensuring that
> processes get restarted before they reach a high water mark.

I find such crude methods to be somewhat lame - OBVIOUSLY the JVM and GC
should be able to outperform a damn restart of the JVM. Or else there is
room for improvement, wouldn't you think?
   Your statements seem to indicate that you don't think there is use in
any more improvement, since you might as well a) throw more money at the
problem, b) run clusters, even on a single box, and c) RESTART the JVMs.

Why, one could in that case just still have used an interpreter, using
the full naive stop/mark/sweep GC??

However, back to your topic, have you actually tried the newer "adaptive
sizing"? Indeed, the new bunch of GC'ers are way better.

 From my earlier links:
http://java.sun.com/j2se/1.5.0/docs/guide/vm/gc-ergonomics.html

" The implementation of -XX:+UseAdaptiveSizePolicy used by default with
the -XX:+UseParallelGC garbage collector ...

          1. If the GC pause time is greater than the pause time goal
then reduce the generations sizes to better attain the goal.
          2. If the pause time goal is being met then consider the
application's throughput goal. If the application's throughput goal is
not being met, then increase the sizes of the generations to better
attain the goal.
          3. If both the pause time goal and the throughput goal are
being met, then the size of the generations are decreased to reduce
footprint.
"

So, basically, it will _reduce_ the heap size to try to get smaller
collection times.

It's all very interesting and stuff - but these technologies already exist.

Endre.

===================================
This list is hosted by DevelopMentor®  http://www.develop.com

View archives and manage your subscription(s) at http://discuss.develop.com