Fwd: Review: draft-ietf-bmwg-sip-bench-term-10 and draft-ietf-bmwg-sip-bench-meth-10

Oops - meant to copy the list. Forwarding...


-------- Original Message --------
Subject: 	Review: draft-ietf-bmwg-sip-bench-term-10 and 
draft-ietf-bmwg-sip-bench-meth-10
Date: 	Wed, 18 Jun 2014 16:19:06 -0500
From: 	Robert Sparks <[email protected]>
To: 	MORTON, ALFRED C (AL) <[email protected]>, Vijay Gurbani 
<[email protected]>, joel jaeggli <[email protected]>, Carol Davids 
([email protected]) <[email protected]>, [email protected] 
<[email protected]>
CC: 	Banks, Sarah ([email protected]) <[email protected]>



Reviews of draft-ietf-bmwg-sip-bench-term-10 and
draft-ietf-bmwg-sip-bench-meth-10

Thank you for the restructure and cleanup work on these drafts.

Most of the comments I made on version -08 of each of them have been
addressed.
Since there were so many changes, rather than revisit the thread from
the -08 review,
I'll start a new thread here, bringing up unaddressed points from the
earlier review
as necessary.

Again, thank you for the adjustment to the title and the text to address
the scope
of the documents. The result is much more straightforward and clear than
what I
had earlier reviewed.

I have a few remaining points and questions and a few nits to call out.

Major technical points:

1) I've read through this enough times now that maybe I've become blind
to where
it's discussed, but for the INVITE tests, I think you have an unstated
assumption
that the responding EA is configured to send 200s as quickly as it can,
and that
whatever delay it has in responding is fairly constant. Otherwise, your
tests
will have widely varying results due to retransmissions of the INVITE.
Is it your
intent that the EA would ever retransmit? If not, the assumptions above
should
be spelled out in the methodology document.

2) The documents still don't make it clear that each register request needs
to be to a distinct AOR (otherwise, there is no sense in having a separate
re-registration test).

3) The documents (particularly the report forms) assume you will use the
same transport on both sides of the DUT. Please state that explicitly, or
if allowing for (for instance) UDP on one side of a proxy and TCP on the
other was intended, please adjust the document to talk about it.

4) I think the documents are assuming that an EA will make one connection
if it's using a connection oriented transport (like TLS). The results of
the test will be very different if it opens a new connection for each
sent message. Similarly, I think you're assuming that a connection
gets set up between the DUT and the EA that will respond to an INVITE
once. Those should be called out explicitly. If you're not assuming
that, there needs to be more discussion about how connections get
established, and whether that's a parameter that needs to be captured
as part of the test.

5) Is it the intent to only test with media that acts like G.711 over
RTP? The configuration parameters you have for EA imply that. If its
not the case, are you missing configuration parameters for, say, video
codecs (an EA won't be able to use what you have in terminology's 3.3.4
for that stream), or for an msrp media session?

Major editorial points:

1) The terms IS and NS are holdovers from when the document was trying to
do more than it is now. NS is not accurately defined and is at cross
purposes with the tests you _do_ define in the document (The discussion
of MESSAGE and SUBSCRIBE that's still in 3.1.1 of the terminology document
does not help this document at all.) I know these words took effort to
write,
but they really should just be removed. A very short introduction to the two
types of session you are actually testing would leave much less room for
confusion.

2) I challenged the usefulness of the definition of session
(sig,medc,med) and
the diagram trying to show these as points in some three-dimensional
space in
my previous review.  I did see Carol's explanation of it in the summary
to that
review, but I disagree that it is helping this document. I think it's
hurting
by adding confusion.  If you deleted it, the rest of the document means
what
it meant before, and the reader is no less informed.  Please remove it.
If the primary point is to make sure that the tester considers covering
every
permutation of the test parameters, just say that. The graphic implies that
some things have more sess.sig than others, and that you might talk about
the distance between points in this vector space. Save all this aside and
bring it back in a document that actually uses it if you must, but please
take it out of this one.

3) The code in Appendix A of the methodology doc should be identified as
a "Code Component" as described in the TLP
(http://trustee.ietf.org/license-info/IETF-TLP-4.htm)
and a license block should be added.

More minor points.

* The methodology document says the DUT can be any conforming 3261 device.
The terminology document says the device can't be user equipment (see
bullet 1
in section 1.1) . Neither are correct. (A presence server, for instance, is
not a reasonable thing for a DUT for the tests in this document). I
suggest in
both documents you just explicitly list what the DUT can be. That list
should
not include "User Agent Server" as it currently does in 3.2.2 of the
terminology
document. A UAS is just a role that any of these devices, including
end-user
terminals, can hold.

* If a Registrar is the DUT, then neither of the topology figures are
correct.
(Unless you're intending for an EA to be the registrar, and the DUT is just
a proxy). Consider adding a figure that more clearly shows what the topology
for your registration test is intended to be, and making it clear that you
are only testing INVITE through devices that forward messages.

* There are several places that call out RTSP as a media protocol.
RTSP is a control protocol, not a media protocol - it doesn't make sense
being listed with RTP and SRTP. MSRP would make more sense.

* The Expected Results in Section 6.8 of the methodology claim that
the rate should not be more than what was in 6.7. How do you come to
that conclusion? There are several valid implementation choices that
could lead to reregistration taking slightly longer than an initial
registration.

NITS:

Methodology document:

Introduction paragraph 1 s/in Terminology document/in the Terminology
document/

Introduction paragraph 5. This points to section 4 (the IANA
consideration section)
and section 2 (the introduction) of the terminology document for an
explanation
of configuration options. Neither of those sections explain
configuration options.
Where did you mean to point?

Terminology document:

Security Considerations: Please remove "and various other drafts". If
you know of
other important documents to point to, ase add them as references.

The definition of Stateful Proxy and Stateless Proxy copied the words
"defined
by this specification" from RFC3261. This literal copy introduces ambiguity.
Please replace "by this specification" with "by [RFC3261]".

Introduction paragraph 4, last sentence: By calling out devices that include
the UAS and UAC functions, you have eliminated stateless proxies, which
contain
neither.

In section 3, when you use the templates, you are careful to say None under
issues when there are no issues. Please use the same care with See Also.
Right now, you have empty See Also: sections that could be misread to
take up whatever content follows (particularly by a text-to-speech engine).

3.1.6: s/tie interval/time interval/

_______________________________________________
bmwg mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/bmwg