FW: Benchmarking Methodology for OpenFlow SDN Controller Performance

Hello BMWG,

 

I am forwarding our response that we have provided to Sarah. 

 

Thanks,

Bhuvan

 

From: Bhuvan (Veryx Technologies) [mailto:[email protected]] 
Sent: Thursday, April 03, 2014 12:13 PM
To: Banks, Sarah ([email protected])
Cc: 'Anton Basil'; vishwas manral ([email protected])
Subject: RE: [bmwg] Benchmarking Methodology for OpenFlow SDN Controller Performance

 

Hello Sarah,

 

We have gone through your comments and they are good.

Please find below our responses to your comments with tag [Authors].

 

Also I would like to check with you if we can forward your comments to the mailing list to generate more discussion around this.

 

Best regards,

Bhuvan

 

-----Original Message-----
From: Bhuvan (Veryx Technologies) [mailto:[email protected]] 
Sent: Wednesday, April 02, 2014 3:10 PM
To: 'Banks, Sarah'
Cc: '[email protected]'; '[email protected]'
Subject: RE: [bmwg] Benchmarking Methodology for OpenFlow SDN Controller Performance

 

Hello Sarah,

 

Thanks for taking time to review this draft and provide your comments.

We will go over all your comments and provide our response shortly.

 

Best regards,

Bhuvan

 

-----Original Message-----

From: Banks, Sarah [mailto:[email protected]]

Sent: Wednesday, April 02, 2014 5:58 AM

To: Bhuvan (Veryx Technologies); [email protected]; [email protected]

Subject: Re: [bmwg] Benchmarking Methodology for OpenFlow SDN Controller Performance

 

Authors,

 

                Thanks for your draft! SDN is interesting, and has come up in some hallway conversations - it's nice to see a draft on it. I've reviewed the doc, and I'll try not to take a stance, but rather, address issues that I see that are either technically wrong or lead to the potential to be wrong, or make the draft more readable. I'll point out that your draft could use an Editor's eye - there are grammatical issues throughout. I'm happy to help if you need, please let me know.

 

[Authors] Sure. We would love to take your help in addressing them in the draft.

 

                I like to review section by section. I hope that approach works for you.

Here goes. :)

 

                Overall, I like that your approach is very straight forward, but I find that the document suffers from a true introduction. Indeed, your Appendices read very nicely, and I found myself thinking that this information would have been nice at the top of the hour - at the beginning of the document. Please consider relocating this information. Not everyone is an SDN guru, and not everyone takes an identical approach - level setting where you're coming from and your scope will make consuming the rest of the document (hopefully) easier. Does this make sense?

 

[Authors] Yes. We do agree your point. We will move the appendix B (OpenFlow Protocol Overview) information after the scope section.

 

                Let me go section by section now.

 

In Section 5.1 - You mention that the switches need not be physical switches, but emulated/simulated by software or test tools instead - the last sentence of this paragraph says that you "MUST indicate the number of switches and hosts..." - consider referencing the emulated/simulated number too. A quick read might leave open the idea that only physical switches used be noted/listed.

 

[Authors] We will reword this sentence to reflect the emulated/simulated entities too. 

 

Section 5.3 refers to the ability to have differing ways to setup channels and connection type, yet only leverages some (as yet undefined) channel setup modes. Why? Why are all of the methods/modes not applicable or being used?

 

[Authors] Though we have described all the methods/modes that are applicable, our intent is to recommend the methods/modes that are supported by controllers available in the market. Performance measurement over other methods/modes is mentioned as optional, depending on the support available on the controller

 

Section 5.3.1.2: In active-passive mode, you state, "Any asynchronous communications from the switch have to be sent on the active channel."

While I suspect you don't mean this in a normative way, it begs the question - why?

 

[Authors]  We think we can remove this sentence from the draft, as in one of the test we have mentioned that to initiate communication on the passive channel as well.

 

Section 5.3.1.3 - why call this out if you aren't going to require that their use be noted in the test results?

 

[Authors] We did mention their use in Section 5.3.2. So we think of having this section in the draft. Please let us know your thoughts.

 

Section 5.3.2 - What is controller teaming? (OK, I know what it is, but you're using a term not yet defined. Please define.)

 

[Authors] Sure. We will define this in the terminology section.

 

Section 5.3.2 - Why are you recommending the use of TCP here, over the alternatives? Be specific.

 

[Authors] Since TCP is being supported by all OpenFlow implementations, we have recommended TCP over others.

 

In general, you don't take a stance on the number of times a test should be run, nor the amount of time to wait/pause between tests. Would you consider adding values to your recommendations?

 

[Authors] We could certainly look at recommending values for parameters like ‘number of times a test should be run’. But for others, we may not be able to define/recommend values, as these parameters may vary on implementation to implementation.

 

Your reporting format is repeated over and over - I'm a fan of state it once, and modify it when needed. :) Just a thought.

 

[Authors] This is a good suggestion. We can remove this portion wherever repeated in this draft.

 

Section 9 - Security Considerations - to my eye, it's unacceptable to state that there might be security considerations in the security considerations section, and then state that they're out of scope. I disagree. They are in scope. What are the issues? It's a north bound interface in the lab. What's up?

 

[Authors] Fine. We will review this section and add specific issues.

 

Is Appendix A (Section 10) a sample test report? There's no information about the setup/topology of the setup. If this isn't what App A is intended to be, consider adding a sample test report. I'd argue it might not live as an appendix, either. :)

 

In that vein, Appendix A has no quantification about WHAT it is - what is it? Why are there Nas? :) Please clarify.

 

[Authors] We have missed to provide the required descriptions to this section. We shall add a few lines of description that would clarify the section’s purpose.

 

Thanks

Sarah

BMWG Co-Chair

 

 

On 4/1/14, 8:07 AM, "Bhuvan (Veryx Technologies)"

< <mailto:[email protected]> [email protected]> wrote:

 

>Hi folks,

> 

>SDN is gaining lot of traction in the industry among vendors and 

>service providers.

>We clearly see a need for test methodologies to measure the performance 

>of SDN based network. Keeping this viewpoint, we have drafted a 

>benchmarking methodology for OpenFlow SDN controller performance.

> <http://tools.ietf.org/html/draft-bhuvan-bmwg-of-controller-benchmarking> http://tools.ietf.org/html/draft-bhuvan-bmwg-of-controller-benchmarking

>-00  We would love to hear any comments and queries on the same.

> 

>Thanks,

>Authors

>

_______________________________________________
bmwg mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/bmwg