RE: Advertisement presence component

Hi Fabrice,
 
Thanks for the response, I really appreciate you taking the time to help
me out.  I have added (inlined) some additional details below...
 
Thanks,
 
Paul


  _____  

	From: Fabrice Di Meglio [mailto:[email protected]] 
	Sent: Thursday, February 16, 2006 4:24 PM
	To: [email protected]
	Cc: Di Meglio Fabrice
	Subject: Re: [JXTA user] Advertisement presence component
	
	
	Hi Paul, 

	I think we need more info about what you try to do to be able to
give you a good answer.
	[Poole, Paul] I am developing a framework around jxta that eases
development of jxta-based applications, increases overall quality of
service, and offers additional services that are not included in jxta
out of the box.  One of the services I would like to offer is something
I call "presence" awareness.  I would like to be able to offer to the
end-developer a component that allows them to monitor groups, nodes,
files, etc that are added to or removed (time out) from the network.
The use of this component will be completely optional so that developers
who are developing an application for very large networks or extremely
bandwidth constrained networks can opt not to use this service.  I am
looking to develop something that is as scalable as possible.  For
example, if the developer is building a chatting application, they would
be able to include the presence awareness component to keep their
network view updated.  So, let's say that a user has joined a group,
they would be able to view all the current members of the group and
their list will update in (mostly) realtime as members join and resign
from the group.
	
	
	As stated into the wiki, such ADV repositories does not exist
for scalability. They will not able to scale to millions of
devices/resources. You can look at the JXTA architecture papers that
talk about RDV, DHT, search and other items.
	[Poole, Paul] Yeah... I am concerned about scalability as well.
I will try to make the component as scalable as possible.  That said, it
will be optional, so if it fails the scalability test for a particular
network, the end-developer can opt-out. 
	
	
	The central idea is to have only caching of ADVs on edge Peers
and RDVs just cache indexes to ADVs (which is a position into the DHT).
When a Peer is searching for an ADV (whatever its type), he just submit
a query which may be forwarded by its RDV or another RDV in the DHT to
the real Peer(s) that can send a respnse. As the response is ultimately
sent by the edge peers, the query initiator will get the latest version
of the ADV and thus of its associated resource.
	[Poole, Paul] It is good to know that the system ensures that
the latest advertisement is always sent. 

	I have also noted that you talked about ADV deletion. This
feature is not part of JXTA platform currently. You can just publish an
ADV with some expiration / aging info. You may republish an ADV is you
need to change some of its parameters.
	[Poole, Paul] Yes, I have built the deletion capability by
flushing the old advertisement and re-publishing it with a 1 millisecond
timeout.  This is currently untested and I am not sure yet how well this
will work across the network. 

	As far as I understand your need, if your service will just have
to cope with a couple of hundred/thousands of "subscribers" your
proposal "2" may work depending on your OS or hardware.
	[Poole, Paul] In the end, my framework will be used to develop
projects that will range in size and useage considerably.  This is not
easily predictable, so I will have to offer flexible alternatives - part
of being in the research arena I guess :-) 

	BTW, it is easier to put the bandwidth constraints on the edge
peers than on the RDVs/Relays peers. So your solution where Peers submit
a query/search at regular interval should be ok for scalability.
	[Poole, Paul] This surprises me.  I would think that if all edge
peers are constantly sending out queries (maybe once a minute), this
would result in excessive bandwidth. 

	Last comment, using PeerGroups and Peers partionning is
fundamental for global scalability. 
	[Poole, Paul] What do you mean by "partitioning"?  How is this
fundamental for global scalability?  I am sorry... I just don't
understand this comment.

	Hth,

	Fabrice

	On Feb 16, 2006, at 12:33 PM, Poole, Paul P1 wrote:


		Hi all: 

		I would like to solicit you all for some advice on
creating an Advertisement aware system.  According to
http://wiki.java.net/bin/view/Jxta/PublishingDiscoveringPipeAndOtherAdve
rtiesements
<http://wiki.java.net/bin/view/Jxta/PublishingDiscoveringPipeAndOtherAdv
ertiesements>  in the "How do peers know an advertisement has been
published?" section, there is no system in place for listening for new
advertisements or the expiration/deletion of others.  I was already
aware of this, but was hoping that someone might have some ideas about
how best to build this around jxta.  I would like to build an optional
component that would allow developers to "opt-in" for notifications of
changes in the network (additions and losses of peers, peergroups, etc).

		Are there any projects that are working on this?  Any
thoughts on some more efficient ways to do this? 

		2 ideas I am kicking around right now are: 

		1. Each peer group uses the DiscoveryService to search
every x minutes and compare the results with the current state.  I don't
like this idea as I think it will be very resource intensive.
Especially if there are many peergroups all utilizing the
DiscoveryService to search... this will be a bandwidth bottleneck.

		2. Peers that are rendezvous can monitor for changes in
the advertisements (additions, deletions, etc).  Then changes can be
messaged to a PeerGroup.  This way, only peers that want/need updates
can subscribe to a particular peer group.  The messages can be throttled
back to be more like digests every X seconds/minutes/hours, etc or be
allowed to be sent per change.  

		I appreciate any thoughts/advice that anyone has to
offer. 

		Thanks, 


		Paul Poole
		Software Engineer
		Lockheed Martin