Re: Benchmarking Methodology for SDN Controller Performance

Hello Bhuvan, Anton, Vishwas, and Mark,



I think the draft is very interesting and well written.

I have to note that my experience with SDN technology is limited.
Nevertheless, the following comments might be of some use:



-          Section 4. Test setup:

o   A definition for “SDN Applications” , “Traffic endpoints” in the
terminology part might clarify their role in the test setup

-          Section 7. Benchmarking tests

o   The recommended number of iterations is “3” . I there a reason for
recommending 3 ? The number might not be large enough to ensure the
stability of the empirical data.

-          Section 7.4.1 Controller Failover Time

o    The Packet Loss is calculated with the formula “Nsnt - Nrvd”. I was
thinking a percentage might be more intuitive, hence how about “(Nsnt -
Nrvd)*100/Nsnt ” ?



In closing, I reiterate the comment from the IETF91 meeting: some testing
examples can help the reader better understand how to use the methodology.



Best regards,

Marius Georgescu 　

Internet Engineering Laboratory

Nara Institute of Science and Technology

IPv6NET Project: http://www.ipv6net.ro/





From: bmwg [mailto:[email protected]] On Behalf Of Bhuvaneswaran
Vengainathan
Sent: Tuesday, September 30, 2014 5:05 PM
To: [email protected]
Cc: vishwas manral; 'Anton Basil'; Tassinari, Mark A
Subject: [bmwg] Benchmarking Methodology for SDN Controller Performance



Hi folks,



We have submitted next version
(https://tools.ietf.org/html/draft-bhuvan-bmwg-of-controller-benchmarking-01
) of the draft addressing the feedbacks and comments received on the
previous version. The draft is based on the points that were discussed
during the IETF 90 meeting
(http://www.ietf.org/proceedings/90/slides/slides-90-bmwg-2.pdf).

Highlights of changes:
1.       Redefined metrics and methodologies to benchmark wide range of
controller implementations independent of southbound and northbound
protocols.
2.       Defined additional metrics including Topology Discovery Time, Path
Provisioning Time and Rate.
3.       Mapped the defined benchmarks in 3x3 matrix against Performance,
Scalability and Reliability.


We would love to hear any comments and queries on the same.



Thanks,

Authors

_______________________________________________
bmwg mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/bmwg