Re: preallocating memory used by javaspace

Dan,



Let's review again the roots of our discussion:

xaea alvein - Mon, 28 Aug 2006 11:36:58: "is it possible to preallocate
memory used by javaspace? can anyone give me some code on how to do that?"

The basic answer here is that Java -Xmx setting should be used to define to
max memory the JavaSpace can consume and have a simple code that calculates
the Entry size. See:

http://martin.nobilitas.com/java/sizeof.html -
http://martin.nobilitas.com/java/SizeOf.jar

http://www.javaworld.com/javaworld/javaqa/2003-12/02-qa-1226-sizeof.html



Still , this is just the tip of the iceberg when coming down to a complete
solution. In reality very small amount of users will settle with the -Xmx
setting and calculating the Entry size.



They would like the space to evict data based on some policy into some
durable media , in some cases users would expect to get some sort of
notification for this event. Some of them would like to utilize multiple
JVMs to run their JavaSpace and have the # of JVMs X -Xmx value to act as
the max memory to be used. In other cases, users would like to pull the
evicted data (expired entries) from a database based on existing database
model or let the space to write the evicted data into RDBMS for better
survivability and recoverability.



As you have just correctly wrote we would like to provide users "_best_
possible solution to _their_ problem." - More than this we "don't want to be
prescribing a solution which turns out to be a poor option".



Based on our experience at GigaSpaces there is no one right solution , but
from what we see in most cases the preferred architecture is the SBA.

With this approach users moving from this centric architecture:

http://www.gigaspaces.com/wiki/download/attachments/8599/IMG104.png



To:

http://www.gigaspaces.com/wiki/download/attachments/8599/IMG105.png



The details for this architecture is described in WP at
http://www.gigaspaces.com/whitepaper.htm#a1 - see the Space-Based
Architecture and the End of Tier-based Computing article.



The partitioned space with this approach can store large amount of
tasks/results objects , have the worker business logic running with it in
the same memory address to save remote calls and have also the relevant data
cached - all within the same process. The space acting in the classic way as
the tasks/results collaboration area , running the worker and also a a
cache. Clients viewing this partitioned space as one entity where tasks are
routed to the relevant partition based on some load-balancing policy. If
each partition cannot store its tasks/results or cached data it is moved
into database or indexed file in sync or async manner. In large deployments
each business processing unit (pres + business + data tier) will be
allocated on-the-fly based on SLA (overall system current throughput
threshold , CPU utilization ,  memory utilization , backlog .) where the
physical machine to use to run the newly created business processing unit
will be selected from an existing pool of machines.



As you can see the above is not one feature or an enhancement request to an
exiting JavaSpace implementation - it is a fully processed architecture that
includes several components that need to be fully integrated. All the above
can be found as part of GigaSpaces. This is essentially what GigaSpaces is
all about.



Shay





-----Original Message-----
From: Dan Creswell [mailto:[email protected]]
Sent: Saturday, September 02, 2006 11:09 AM
To: [email protected]
Subject: Re: preallocating memory used by javaspace



Hmmm, so I still don't feel like I was entirely clear - let's keep it
simple:



My interest is in giving any user that wants to play with Jini/JavaSpaces
the _best_ possible solution to _their_ problem.



Thus I have a strong preference for completely understanding a user's needs
before I try and provide an answer.  For me this is part of being
professional.  I don't want to be prescribing a solution which turns out to
be a poor option.



And ethically speaking I would be very uncomfortable knowing I'd recommended
a poor option as the result of not listening to a user more carefully.



Hope that's clear,



Dan.









Hi Shay,



I'm very aware of the need for vendors to do the things they do, it's all
part of doing business.  And I have no issue with that.



My only concern in all of this is to make sure we understand what the user
wants and what their solution boundaries might in terms of cost, lock-in
etc.  Then we can provide a good answer to their question.



Thus, when you talk about value add features as below, it's important that
we understand whether they are relevant or not and the tradeoffs involved.
And if we're not clear what the user wants we can't make such judgements.



I take your point that people do move between database implementations but I
think you'd agree it's a pretty rare thing because, as we all know, it's
hard.  And it's made harder by features that exist in one product but not
another.



I'd have to say that I'm surprised you haven't faced the vendor lock in
question as myself and a number of architects I know working in banks, telco
etc all have that question high on their list.



Best,



Dan.



Shay Hassidim wrote:

> Dan,

>

>

>

> Your question is in fact very good one.

>

> You should know that in the software industry you build a product on top

> of some specification and add your own flavor into it.

>

> Take for example IBM webshpere , BEA Weblogic and JBoss Application

> Servers. All these following the J2EE Spec. Are you sure all of them got

> the exact set of features?

>

> Same for Oracle , DB2 and MSSQL - all of them support SQL. Are you sure

> all of them got the exact features?

>

> Still, users do move between them. It is up to the architecture to

> decide which features he wants to use - but the basic core features are

> exactly the same.

>

>

>

> Jini/JavaSpace is a great technology and one of the most simplistic and

> powerful specs I have seen.

>

> Still , in order to build enterprise scale applications architects and

> develops need a product that cover their needs.

>

>

>

> This is what GigaSpaces all about - The core that is powerful JavaSpace

> implementation (that you can get for free as part of the community

> edition that includes in fact most of the options I listed below) and a

> sophisticated Service Grid using Jini and Rio that allows you to deploy

> your application into the Grid. The space will be started for you and

> injected into the Service itself. Your space entries and Application

> Services can be pure POJOs - no need to implement any interface to write

> entries into the space or to deploy your classes as Services.

>

>

>

> You may access the space using JavaSpaces , SQL , Map , JMS or JDBC with

> Java , C++ and C# utilizing GigaSpaces as Grid enabler using

> master/worker framework , Service Bus or a distributed Cache. All the

> above are wrapped with Spring templates to provide one common ground for

> developers to start with. This means you can use Standard Spring

> templates for all the above. To allow the In Memory Data Grid we

> interact implicitly with external data sources you can plug-in your

> database DAOs using Hibernate and allow the space to read data or write

> data into the database - in sync or async manner.

>

> See:

>

> http://www.gigaspaces.com/wiki/display/GS/5.1

>

>

>

> The above includes the latest info about our 5.1 release. See POJO ,

> Spring , CacheLoader/Store , Mirror , PONO , POCO and many more.

>

>

>

> In terms of performance - in general all APIs provide more or less the

> same performance - simply because they are built on top of the JavaSpace

> API that is the super set of all other API's. Quick performance numbers:

>

> In embedded mode Space API - 100K operations per second , strict remote

> mode 5K operations per second , send and forget remote mode 20K

> operations per second - All these for one space. For Map API users we

> got special optimization that allows us to provide local cache that

> provides the same performance as Hash table. The local cache is sync

> with the master space. You can scale and use the clustered partitioned

> space and start new services on the fly based on SLA to process the

> incoming requests to accommodate your needs - so there is no limit to

> the throughput you can get from the system.

>

>

>

> Why users would need to debate with the vendor locking question when

> they get exactly what they need , especially when all what GigaSpaces

> provides is build on top of industry standards and exposing only

> standard APIs? See the effort we put into supporting POJO both at the

> Data Grid layer and the Service Grid layer - No need to implement any

> special interfaces! Even not to implement the Entry interface to write

> entries to the space! The architect and developer fantasy becomes the

> reality!

>

>

>

> As part of my job I'm visiting customers and prospects around the world

> - when presenting the above I never got the vendor locking question.

>

>

>

> Hope the above clarifies our approach and answer your questions.

>

>

>

> Regards,

>

> Shay

>

>


===========================================================================
To unsubscribe, send email to [email protected] and include in the body
of the message "signoff JAVASPACES-USERS".  For general help, send email to
[email protected] and include in the body of the message "help".

To view past JAVASPACES-USERS postings, please see:
http://archives.java.sun.com/archives/javaspaces-users.html