Re: Increase outdegree
Greg Bildson <[email protected]>
| Newsgroups | gmane.network.gnutella.devel |
|---|---|
| Message-ID | <[email protected]> |
Arne Babenhauserheide wrote: > > Hi Greg, > > My replies are inline... > > Am Montag, 10 de September de 2007 15:29:57 schrieben Sie: > > Our goal a long time ago was to add nonblocking IO so that we could > > increase the outdegree by another order of magnitude and decrease > the TTL > > by one. Basically dynamic querying and other network factors work much > > better the more you up the outdegree. We'll have to look into whether > > other state that we keep per ultrapeer isn't too memory intensive > the way > > we've done things. > > > Another thing to keep in mind when making an outdegree magnitude > increase > > (like to 128 or 256) is that you then need to differentiate between old > > connections and new. Old connections need to get routed to less new > > connections to slow down their query flow in the new architecture. You > > can't really test these changes without applying them to a large > segment of > > the network due to network effects. > > Af far as I understand it this doesn't apply to small scale increase of > outdegree without TTL reduction, or does it? > I'm not sure what "this" your referring to. Any increase in outdegree increases the ability of an ultrapeer to effectively apply selective querying. More connections should imply lower average TTL and thus more routing at the last hop, etc ... All good things. Increasing outdegree will magnify the power of any incoming query so if you don't want a disproportionate amount of resources to be devoted to incoming queries from old network hosts, you need to route those queries to less hosts to at least balance this out. You need to do this to keep the old hosts from using up the resource gains that the new structure affords you. (In practice, you should probably cheat the old network hosts a little so that they are encouraged to upgrade to new software.) > > > The goal is never to necessarily reach the entire network. Yes we > want to > > be able to do as well as we can for rare searches. We can probably > > increase the coverage of queries without overloading ultrapeers. > However, > > you do need to be sensitive to the people injecting spam into the > network. > > Upping the outdegree without considering spam could be > counterproductive. > > How would it affect the possibililty of spamming? > > If I reach more hosts which spam I also reach more hosts which don't > spam. And > DQ stops the searches before they get too many (spam) results. > > If you get to higher outdegree with TTL reduction, Spam travels one > step less > and affects the network less that way. > The effect on spam is somewhat unknown. One thing that would happen with LimeWire now is that we would receive more spam on rare queries but ignore it. Thus, there could be a huge increase of spam that is ignored with respect to dynamic querying while we are still trying to get our target number of results. If you increase the horizon, you could dramatically increase the spam results and load just to get a few more results from a rare query. Spam would be bad in multiple ways - this doesn't necessarily defeat the idea but has to be thought through in detail. > But these problems also don't apply to small scale outdegree > improvement to > make the whole network reacheable again (there is a very noticeable > effect, > because as leaf about half the network isn't reacheable I assume due to > overlap of UP reach). > > And DQ and QRP grow stronger with this small outdegree increase, too. > > So these don't affect the basic suggestion: > > Increase the default outdegree to 50, so all 5 Million sources are > reacheable > > again for an Ultrapeer. > You really need to start with what you want as the resulting maximum traffic limit on ultrapeers and work backwards. There is never any guarantee of being able to query the entire network so I don't like to hear that as a goal. If you can achieve it without burdening ultrapeers too much then fine. Otherwise, it just isn't doable. Ultrapeer traffic flow is sensitive to various factors with outdegree being one major factor. Another thing to keep in mind as well is that without making other changes, you would increase the average query time by increasing the outdegree. Ultrapeers would have to be queried at a faster rate leaving less time to see the feedback from those queries. If the TTL of the query drops however, then feedback rate is less an issue. At higher outdegree, dynamic querying might need better support for doing fractional TTLs to satisfy intermediate cases (i.e. TTL=2.5 implies alternating between a TTL of 2 and 3 for sequential connections). > > Best wishes, > Arne > > PS: About traffic I dare to paste something I wrote to someone else > today: > ------ > Optimization efficiency of Gnutella: At the moment one search reaches > about a > million people and a UP has only about 7kB/s traffic (out and in), so a > Gnutella UP can operate off a standard ISDN line. > > A leaf has even less: 1kB/s network traffic at most, and I'm currently > down to > 100 bytes per second mean traffic which is close to nothing, and it > "flares > up" to a blazing 800 bytes per second when I fire out a search, which is > still low enough to be used on a 24kbit/s modem line (Statistics by > Phex with > 5 Leaf to UP connections). And this with 609 aka 13.43 GB shared files. > ----- > > PPS: And I also dare to reference a text on scaleability of Gnutella I > wrote a > while ago: > - http://draketo.de/english/p2p/light/why-gnutella-scales-quite-well > <http://draketo.de/english/p2p/light/why-gnutella-scales-quite-well> > > -- > Unpolitisch sein > Heißt politisch sein > Ohne es zu merken. > - Arne Babenhauserheide ( http://draketo.de <http://draketo.de> ) > -- Weblog: http://blog.draketo.de <http://blog.draketo.de> > > [Non-text portions of this message have been removed] > > > Messages in this topic > <http://groups.yahoo.com/group/the_gdf/message/23204;_ylc=X3oDMTM2MTQzYjIxBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBG1zZ0lkAzIzMjA2BHNlYwNmdHIEc2xrA3Z0cGMEc3RpbWUDMTE4OTQzMzEyOAR0cGNJZAMyMzIwNA--> > (0) Reply (via web post) > <http://groups.yahoo.com/group/the_gdf/post;_ylc=X3oDMTJxbms2c2p2BF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBG1zZ0lkAzIzMjA2BHNlYwNmdHIEc2xrA3JwbHkEc3RpbWUDMTE4OTQzMzEyOA--?act=reply&messageNum=23206> > | Start a new topic > <http://groups.yahoo.com/group/the_gdf/post;_ylc=X3oDMTJlYmIxajI0BF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA250cGMEc3RpbWUDMTE4OTQzMzEyOA--> > > Messages > <http://groups.yahoo.com/group/the_gdf/messages;_ylc=X3oDMTJlcXRyNmllBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA21zZ3MEc3RpbWUDMTE4OTQzMzEyOA--> > | Files > <http://groups.yahoo.com/group/the_gdf/files;_ylc=X3oDMTJmaXFqbW9qBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA2ZpbGVzBHN0aW1lAzExODk0MzMxMjg-> > | Photos > <http://groups.yahoo.com/group/the_gdf/photos;_ylc=X3oDMTJlOGJxMzlrBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA3Bob3QEc3RpbWUDMTE4OTQzMzEyOA--> > | Links > <http://groups.yahoo.com/group/the_gdf/links;_ylc=X3oDMTJmbGlkMnA4BF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA2xpbmtzBHN0aW1lAzExODk0MzMxMjg-> > | Database > <http://groups.yahoo.com/group/the_gdf/database;_ylc=X3oDMTJjc2VvZnNzBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA2RiBHN0aW1lAzExODk0MzMxMjg-> > | Polls > <http://groups.yahoo.com/group/the_gdf/polls;_ylc=X3oDMTJmMGllZThiBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA3BvbGxzBHN0aW1lAzExODk0MzMxMjg-> > | Members > <http://groups.yahoo.com/group/the_gdf/members;_ylc=X3oDMTJlMGxzMjEzBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA21icnMEc3RpbWUDMTE4OTQzMzEyOA--> > | Calendar > <http://groups.yahoo.com/group/the_gdf/calendar;_ylc=X3oDMTJkdGpxYW9hBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA2NhbARzdGltZQMxMTg5NDMzMTI4> > > Yahoo! Groups > <http://groups.yahoo.com/;_ylc=X3oDMTJkbm83Z2E4BF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA2dmcARzdGltZQMxMTg5NDMzMTI4> > > Change settings via the Web > <http://groups.yahoo.com/group/the_gdf/join;_ylc=X3oDMTJmaHV2NHFhBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA3N0bmdzBHN0aW1lAzExODk0MzMxMjg-> > (Yahoo! ID required) > Change settings via email: Switch delivery to Daily Digest > <mailto:[email protected]?subject=Email%20Delivery:%20Digest> > | Switch format to Traditional > <mailto:[email protected]?subject=Change%20Delivery%20Format:%20Traditional> > > Visit Your Group > <http://groups.yahoo.com/group/the_gdf;_ylc=X3oDMTJkam9pNTdvBF9TAzk3MzU5NzE0BGdycElkAzI2ODQyNTMEZ3Jwc3BJZAMxNzA1MDE2MDYxBHNlYwNmdHIEc2xrA2hwZgRzdGltZQMxMTg5NDMzMTI4> > | Yahoo! Groups Terms of Use <http://docs.yahoo.com/info/terms/> | > Unsubscribe <mailto:[email protected]?subject=> > > > __,_._ [Non-text portions of this message have been removed]