[quagga-users 14910] Re: IXP Design with Quagga Server

On Wed, 17 Jan 2018, Pardeep wrote:

> Queries;
>
> 1. Will Quagga server be able to handle BGP millions of routes in its
> routing table?

With few peers, yes. In an IXP with many peers, likely not - well, most 
of the time it will, as long as the peers are stable, but peer's going 
down will cause havoc (if they advertise non-trivial numbers of routes 
esp.). This is why the other respondent doesn't know of many IXes using 
Quagga.

Attempts have been made to address this, by adding even more buffering 
of works (on top of the buffering I added years ago with the workqueue), 
by adding threading (either trying to fully multi-thread bgpd, or trying 
to just thread off non-UPDATE peer messages like KEEPALIVE).

The former perhaps buys some latency here and there, but ultimately it 
adds more overheads, more CPU, more memory usage, and makes bgpd 
throughput slower again - which actually can make the problem worse, by 
making bgpd scale even worse. It also adds complexity, and bugs, and 
makes the code harder to maintain. Which the threading approach also 
does, but even more so.

Killing one of the good properties of Quagga bgpd (it scales up well 
memory and CPU wise, route-wise - better than BIRD, ignoring the poor 
"significant peer down" problem), to add dubious fixes for the peer-down 
performance problem, is probably not the right long-term approach.

BIRD proves the buffering approaches in Quagga were, ultimately, largely 
dumb. BIRD (and OpenBGPd) also proves you don't need threads everywhere. 
If you need layers and layers of buffering, and threads per peer, to 
handle BGP, you're just doing something wrong and have screwed up.

The lessons from BIRD should be heeded. ;)

I was thinking about that a while ago, and there may actually some very 
simple fixes to the peer-down problem. Least some significant 
improvements.

Hoping to try those out sometime. Just need to find some elusive spare 
time.

----

At the protocol level, there is also the issue that standard BGP simply 
isn't very good at the job an IXP wants it to do: Act as a light-weight 
central 'hub'.

Stock BGP intrinsically is a distributed shortest-path algorithm. The 
very semantics of its UPDATE messages inherently implement an iterative, 
distributed computation of a matrix of costs over a graph.

Moving a lump of data quickly from one neighbour to another simply is 
_not_ what a BGP-4 UPDATE message is designed to be good at.

There have been attempts to try bend BGP UPDATEs to be good at moving 
data, but they (the 2 I'm thinking of anyway) retain significant 
elements of the distributed path computation aspects of BGP UPDATE, for 
compatibility / minimum change reasons. Hence they will continue to suck 
for efficiently moving data from one place to another (cause just that's 
intrinsically at odds with doing an iterative, distributed path 
computation at the same time with the same messages).

We could actually add a Quagga specific extension though..

Again, just needs someone with time. ;)

regards,
-- 
Paul Jakma | [email protected] | @pjakma | Key ID: 0xD86BF79464A2FF6A
Fortune:
Freedom is what you do with what's been done to you.
 		-- Jean-Paul Sartre