Re: Re: Upload queue idea: Candidate selection based on on the response ratio

Am Donnerstag 04 September 2008 13:42:33 schrieb [email protected]:
> This is true, although a little bit less so with PARQ and multiple queues.
> But still, within a PARQ queue you can have a wide range of file sizes,
> and what you propose is interesting.

The quite advanced implementation in PARQ was one of my inspirations, and I 
was thrilled when I found the "highest response rate next" scheduling policy, 
because it gives a very simple solution to a challenge we've been facing for 
years. 

Besides: Recommended Reading: William Stallings: Operating Systems, chapter 
scheduling. 

Gnutella is in some ways _very_ similar to a highly distributed operating 
system, though it serves file data instead of processing time. 

-> Another way to talk to computer scientists about Gnutella to get more 
researchers into real-life Gnutella development. 

"The largest existing fully distributed ad-hoc operating system with 5 million 
simultaneously active nodes." 

Does that sound buzz-word heavy enough to become the next big hype? ;-) 

But damnit, it fits. 

> :The proposed selection strategy has the advantage of making downloads of
> : small files more efficient and removing the current unfairness without
> : needing multiple queues.
>
> Another thing which can probably be improved is the trade-off between
> queuing and serving immediately.  For intance, if a client requests 20 KiB
> only (e.g. to complete a small hole), it is silly to go through an active
> queueing proces where the amount of headers exchanged will quickly reach
> 20 KiB anyway.  Short enough requests should be allowed to bypass any
> queuing and be served immediately.
>
> Note this has nothing to do with the file size.  You can request 20 KiB of
> a 2 GiB file, it's still a small request.

This is dangerous, because it provides an incentive for abusive behaviour: 
Request files in little pieces when you see that a source has a full queue, so 
I wouldn't open that hole. 

If there is a weakness, someone will exploit it, and the users will applaud 
because of the great speeds - till the whole network goes down, and they with 
it. 

But if it wasn't for that hole, the feature would be very useful. 

Maybe an adaption can be implemented as optional optimization (though not for 
small segments, but only for small files to avoid to provide an opening to 
abuse): 

If a new request gets to the top of the queue when taking the average 
wait_time in your queue, then serve it instantly. 

This opens a small hole for small files, but the tradeoff of avoiding a 
double-request could be worth it - it would also open the option of doing some 
small-file database-like operations without having to resort to "rich" (aka 
huge) query replies or some additional queues. 

For example query replies could include a magnet-uri to preview images (which 
will need very careful type checking before opening), which could be shown 
alongside the search result. 

Also other kinds of metadata could this way be served as standard uploads. 

> Finally, clients should make sure download mesh exchanges take place during
> the queuing exchanges.  It is useless to wait for the actual transfer to
> take place to start exchanging sources.

Isn't that standard behaviour? That's why queued sources are still useful if 
they never make it through the queue, because they finish before reaching the 
end of the queue :) 

Best wishes, 
Arne
-- 
-- My stuff: http://draketo.de - stories, songs, poems, programs and stuff :)
-- Infinite Hands: http://infinite-hands.draketo.de - singing a part of the 
history of free software.
-- Ein Würfel System: http://1w6.org - einfach saubere (Rollenspiel-) Regeln.

-- PGP/GnuPG: http://draketo.de/inhalt/ich/pubkey.txt


[Non-text portions of this message have been removed]