Re: Benchmarking Methodology for SDN Controller Performance

Hi Bhuvan, Anton, Vishwas, and Mark,

I zoomed in, on one of your ‘Performance Benchmarking Tests’, (7.1.4 Path Provisioning Time).

It appears to me, that the procedure for 'proactive path provisioning’ may have to be clarified a bit further. To begin with, when the controller is benchmarked, the dependency on the SDN Nodes/Routing Elements ('Network Devices' as described in the draft Al points us to -http://tools.ietf.org/html/draft-irtf-sdnrg-layer-terminology-04#page-11) could distort the actual measurements due to processing, propagation or buffering delays experienced by the routing elements. Agree ?

“Install the flow with the learnt source and destination address through controller's northbound or management interface.” Not following this. The northbound I/f of the controller is the application. Is this a case where the application layer is simulating/creating a ‘demand’ from a source to a destination ?

“Record the time when a successful response for the flow installation is received (Tp) from the controller.” Does ‘ low installation' denote, programming the forwarding plane or the arrival of the first packet/frame corresponding to this test stream ? How will the controller determine this and is the controller notifying the SDN nodes ?

How is ‘the expiry of test interval (To)' used in the equation ?

Regards,
Jay


From: Bhuvaneswaran Vengainathan <[email protected]<mailto:[email protected]>>
Date: Tuesday, September 30, 2014 at 3:04 AM
To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>
Cc: vishwas manral <[email protected]<mailto:[email protected]>>, 'Anton Basil' <[email protected]<mailto:[email protected]>>, "Tassinari, Mark A" <[email protected]<mailto:[email protected]>>
Subject: [bmwg] Benchmarking Methodology for SDN Controller Performance

Hi folks,


We have submitted next version (https://tools.ietf.org/html/draft-bhuvan-bmwg-of-controller-benchmarking-01) of the draft addressing the feedbacks and comments received on the previous version. The draft is based on the points that were discussed during the IETF 90 meeting (http://www.ietf.org/proceedings/90/slides/slides-90-bmwg-2.pdf).



Highlights of changes:

1.       Redefined metrics and methodologies to benchmark wide range of controller implementations independent of southbound and northbound protocols.

2.       Defined additional metrics including Topology Discovery Time, Path Provisioning Time and Rate.

3.       Mapped the defined benchmarks in 3x3 matrix against Performance, Scalability and Reliability.


We would love to hear any comments and queries on the same.

Thanks,
Authors

_______________________________________________
bmwg mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/bmwg