Re: [core-dev] Bandwidth drop-off in Limewire 4.2.6 (test results)

Hi Greg, pardon the duplicate if this comes through more than once, as my 
original reply to you didn't appear so I will try again now. To 
your question of firewalls I replied that on all of my systems the 
firewalls (ZoneAlarm) are set to give Limewire and Java full access to 
the Intenet, unrestricted -- both client and server capabilities are 
enabled for Limewire and Java (full access), and I've tested in the 
past with the firewalls turned completely off also.

Your point about overhead being possible is a good one, although any 
such overhead shouldn't drop-off my bandwidth so heavily (but who 
knows, maybe there's a problem in that code that could be making it do 
so?). The best I can figure is that the upload/download code is in 
need of an overhaul as Sam mentioned when I first reported my testing 
results, and I wonder if anyone else has the same problems.

Can you run a similar test? It's pretty simple and fast, as long as 
you have two computers (one as the cliet, the other to be the source). 
Try manually downloading (direct connect) one of your own private 
files that's not on Gnutella, and that gives you your average download 
speed without any other sources. Then, from your server, download a 
popular movie (or other large file) that will have several alternate 
sources. After it has downloaded, manually connect to that server 
(direct connection again) from your client and download it. You should 
start off at full speed, but as sources are added (the alternate 
sources are found and added to the download) you'll see your server 
bandwidth that is dedicated to the upload drop-off. This more sources 
the client finds, the more you'll see your server's upload bandwidth 
drop off.

I ran this test a few times, ensuring that my client was the only one 
uploading from my server (to be certain that the upload drop-off in 
speed wasn't the result of others downloading from my server). It 
happened every time, and at times I went from full speed (about 
80K/second) down to about 1/8th of what was actualy available (my 
server would drop to around 10K/sec when a few sources were addded). 
Very strange, and quite surprising since that means the swarming is 
actually degrading my overall performance -- until a large number of 
sources are found (about 5 or more) the bandwidth to my client is 
actually lower than when I use just my server as a source; it's 
basically in need of 5x or more the necessary clients to perform at 
the same level as my single server, which is a huge amount of wasted 
network traffic and processing power.

Brian
----- Original Message -----
From: [email protected]
To: [email protected], "Brian Fall" <[email protected]>
Subject: Re: [core-dev] Bandwidth drop-off in Limewire 4.2.6 (test results)
Date: Wed, 22 Dec 2004 14:27:43 -0500

> 
> Brian,
> 
> I didn't notice in this email but were any of those firewalled transfers?  I
> guess not but there is a wastage there of uplink capacity due to 
> ACKing that we
> are going to take care of shortly.
> 
> Otherwise, it sounds very fishy.  Not having followed the earlier conversation
> totally, I would suggest neutralizing any bandwidth throttles that 
> are used and
> turning off the 1/2 second delay that we add in to allow other TCP traffic to
> get through for browsers, etc and see what happen.  I think Sam and Zlatty can
> point those areas out.
> 
> Thanks
> -greg
> Quoting Brian Fall <[email protected]>:
> > Hello everyone, I've finished testing Limewire 4.2.6 to
> > determine if the bandwidth drop-off that I reported
> > earlier is fundamental to the magnet links (my original
> > testing method) or is independent of magnet links. In my
> > earlier tests with earlier versions of Limewire 4.2.x I used
> > magnet links with multiple sources (using the as and xt
> > attributes), each of which are my own servers. That is,
> > the magnet link sources pointed to my own servers so that
> > I could monitor bandwidth during the upload from those
> > servers to the client that clicked on the magnet links.
> >
> > What I observed was very surprising: with each new source
> > the bandwidth for ongoing uploads dropped. For example,
> > if I used just one of my servers I got the download at
> > full speed, but when two were used both of those sources
> > only uploaded at about half-speed -- it was no faster to
> > upload from two sources than one because of the upload
> > speed drop-off. Things got worse with each new source;
> > with a handful of sources or more my own two servers
> > dropped off  to just a fraction of the bandwidth they
> > had available. As a result, most of the available
> > bandwidth on my servers was not used despite the fact
> > that the client actually started with them (that is,
> > my two servers are the first sources that the client
> > starts with and they have ample bandwidth, but as the client
> > starts to connect with other clients on the Gnutella
> > network the upload/download speed between the client and
> > my servers drops off to the point where only a fraction
> > of the available bandwidth is used).
> >
> > With Limewire 4.2.6 I decided to eliminate the magnet links
> > from the equation entirely, and provide a testing method
> > that is easy enough for anyone to try. If you do the same
> > and get the same results we can then assume that there is
> > a fundamental upload/download issue with Limewire and that
> > magnet links have nothing to do with it. I ran these new
> > tests and had the same result as previously; with each new
> > source the upload/download speed drops off until eventually
> > only a fraction of the available bandwidth on my own servers
> > is used. Following are the steps I took for these new tests:
> >
> >  1) Using a personal file that I know isn't available on the Gnutella
> >     network I tested the upload/download speed on my own servers.
> >     Because the file isn't available on the larger network I didn't
> >     have to worry about the client finding new sources; I got full
> >     speed downloads from each of my servers with this test when only
> >     one of them was used by the client. But, when both were used the
> >     drop-off occured as previously (each server used only about half
> >     of their bandwidth when both servers were used, yet transferred at
> >     full speed when only one for the transfer).
> >
> >
> >  2) Using popular files on the Gnutalla networks (movies, music files, etc.)
> >     that have a large number of sources) I was able to determine that the
> >     download speed drops-off as more sources are found. To observe this I
> >     would download the file onto my servers using Limewire, so that my
> > servers
> >     had the files *and* retained any alternate sources associated with the
> > files.
> >     Then, using my client, I established a direction connection to my server
> > and
> >     manually downloaded the media file. This would start the download at full
> >     speed (one source - my server), and within a minute or so the client
> > would
> >     also start downloading from the alternate sources associated with that
> > file
> >     (multiple sources on the Gnutella network). That's when the drop-off
> > started...
> >
> > As I observed previously when conducting tests with magnet links the upload/
> > download bandwidth for my own server began to drop-off as the client adds
> > more sources; this drop-off continues to the point where only a fraction of
> > the available bandwidth my server has is actually used. In other words,
> > the exact same behavior occurred when magnet links weren't involved,
> > indicating that the problem is isolated withing the core upload/download
> > code in Limewire.
> >
> > Can others replicate the drop-off in bandwidth? The test is rather simple,
> > and you can do it as long as you have two computer (one to act as the source,
> > the other the client). Merely share a private file that isn't on the larger
> > Gnutella network and record how fast the upload/download happens, and then
> > do the same for a popular file available on the Gnutella network -- download
> > it first to your server, then direct connect to your server from the client
> > and manually start the download. The download starts at full speed, and
> > should
> > drop-off in speed as the alternate sources are found and added.
> >
> > Unless my particular test results are unique to me it seems like a big
> > problem that limits Limewire's overall effectiveness since it
> > loses speed with each new source added to the download (in order to get
> > decent speed a lot of sources are needed to compensate for the drop-off).
> > In my case it's faster to have the download from my own server than to
> > swarm it from a few (2-3) sources (in which case a standard Web server
> > would be better to transfer the files since users only need a Web
> > browser to get the files at the same speed as Limewire offers). If I have
> > enough sources however, with ample bandwidth available from each, the
> > drop-off in bandwidth is overcome. This means that more sources are tapped
> > just to overcome the problem, which in turn means more traffic on the
> > network and more resources consumed than needed.
> >
> > But if the swarming didn't drop-off in speed it would offer a
> > tremendous advantage for such file transfers. We're looking at
> > Limewire's potential as a file transfer tool for companies, but
> > so far it seems like the bandwidth drop-off makes it on par with
> > traditional file sharing. But it could just be my setup, so I wonder:
> >
> >   Can anyone confirm that this happens to them as well?
> >
> > Brian
> >
> > --
> > ___________________________________________________________
> > Sign-up for Ads Free at Mail.com
> > http://promo.mail.com/adsfreejump.htm
> >
> >
> > _______________________________________________
> > core-dev mailing list
> > [email protected]
> > http://www.limewire.org/mailman/listinfo/core-dev
> >

-- 
___________________________________________________________
Sign-up for Ads Free at Mail.com
http://promo.mail.com/adsfreejump.htm


_______________________________________________
core-dev mailing list
[email protected]
http://www.limewire.org/mailman/listinfo/core-dev