Re: Comments on BUNDLE -37 - RTP handling
Flemming Andreasen <[email protected]> Wed, 30 Aug 2017 16:45:26 -0400
| Newsgroups | gmane.ietf.mmusic |
|---|---|
| Message-ID | <[email protected]> |
As you said Christer, nobody has objected or provided an alternative suggestion, so please proceed with the suggested text. Thanks -- Flemming (as MMUSIC co-chair) On 8/28/17 6:17 AM, Christer Holmberg wrote: > Hi, > > Nobody has objected to Colin¹s text, so my suggested is to now merge the > PR. > > Regards, > > Christer > > > > On 18/07/17 16:42, "Colin Perkins" <[email protected]> wrote: > >> Hi Christer, >> >> As discussed - pull request submitted. >> Colin >> >> >> >>> On 9 Apr 2017, at 08:54, Christer Holmberg >>> <[email protected]> wrote: >>> >>> Hi Colin, >>> >>> Thanks for your input! >>> >>> Would it be possible for you to create a pull request with your >>> suggested changes? >>> >>> BUNDLE can be found at: https://github.com/cdh4u/draft-sdp-bundle >>> >>> Regards, >>> >>> Christer >>> >>> -----Original Message----- >>> From: mmusic [mailto:[email protected]] On Behalf Of Colin Perkins >>> Sent: 07 April 2017 20:06 >>> To: mmusic (E-mail) <[email protected]> >>> Subject: [MMUSIC] Comments on BUNDLE -37 - RTP handling >>> >>> I have some comments on BUNDLE -37 - apologies that I wasn¹t able to >>> participate fully in the mailing list discussion before the meeting. >>> >>> The current text is generally reasonably clear, and is certainly >>> precisely written, but I do think it has some problems. Most of these >>> relate to the issue of whether RTP packets or RTP streams are being >>> processed. I think it¹s important that this draft is written in terms of >>> how to associate RTP streams with m= lines, rather than how to route and >>> demultiplex RTP packets, since RTP packet routing and demultiplexing is >>> already specified by the various RTP specifications. In particular, >>> there are some places where the draft as written conflates the two >>> layers in ways that conflict with the RTP and RTCP specifications, that >>> I¹ve tried to correct by more cleanly separating the layers. >>> >>> I¹ve tried to write my suggestions in a prescriptive manner, keeping to >>> the style of the existing text where possible. Hopefully the result is >>> clear and understandable. >>> >>> Comments line below: >>> >>>> 10.2. Associating RTP/RTCP Streams With Correct SDP Media Description >>>> >>>> NOTE: The text in this section is copied from Appendix B of JSEP. >>>> The community has not yet agreed on the text. >>>> >>>> As described in [RFC3550], RTP packets are associated with RTP >>>> streams [RFC7656]. Each RTP stream is identified by an SSRC value, >>>> and each RTP packet includes an SSRC field that is used to associate >>>> the packet with the correct RTP stream. RTCP packets also use SSRCs >>>> to identify which RTP streams the packet relates to. However, a RTCP >>>> packet can contain multiple SSRC fields, in the course of providing >>>> feedback or reports on different RTP streams, and therefore can be >>>> associated with multiple such streams. >>>> >>>> In order to be able to process received RTP/RTCP packets correctly, >>>> it must be possible to associate an RTP stream with the correct "m=" >>>> >>>> >>>> >>>> Holmberg, et al. Expires October 2, 2017 [Page >>>> 20] >>>> Internet-Draft Bundled media March >>>> 2017 >>>> >>>> >>>> line, as the "m=" line and SDP attributes associated with the "m=" >>>> line contain information needed to process the packets. >>>> >>>> As all RTP streams associated with a BUNDLE group use the same >>>> address:port combination for sending and receiving RTP/RTCP packets, >>>> the local address:port combination cannot be used to associate an RTP >>>> stream with the correct "m=" line. In addition, multiple RTP streams >>>> might be associated with the same "m=" line. >>>> >>>> An offerer and answerer can inform each other which SSRC values they >>>> will use for an RTP stream by using the SDP 'ssrc' attribute >>>> [RFC5576]. However, an offerer will not know which SSRC values the >>>> answerer will use until the offerer has received the answer providing >>>> that information. Due to this, before the offerer has received the >>>> answer, the offerer will not be able to associate an RTP stream with >>>> the correct "m=" line using the SSRC value associated with the RTP >>>> stream. In addition, the offerer and answerer may start using new >>>> SSRC values mid-session, without informing each other using the SDP >>>> 'ssrc' attribute. >>>> >>>> In order for an offerer and answerer to always be able to associate >>>> an RTP stream with the correct "m=" line, the offerer and answerer >>>> using the BUNDLE extension MUST support the mechanism defined in >>>> section 14, where the offerer and answerer insert the identification- >>>> tag associated with an "m=" line (provided by the remote peer) into >>>> RTP and RTCP packets associated with a BUNDLE group. >>>> >>>> When using this mechanism, the mapping from an SSRC to an >>>> identification-tag is carried in RTP header extensions or RTCP SDES >>>> packets, as specified in section 14. Since a compound RTCP packet >>>> can contain multiple RTCP SDES packets, and each RTCP SDES packet can >>>> contain multiple chunks, a single RTCP packet can contain several >>>> SSRC to identification-tag mappings. The offerer and answerer >>>> maintain tables used for routing that are updated each time an RTP/ >>>> RTCP packet contains new information that affects how packets should >>>> be routed. >>> The text above looks good. In particular, it correctly focusses on >>> associating RTP streams with m= lines, which is the correct level of >>> abstraction. >>> >>>> However, some implementations of may not include this identification- >>>> tag in their RTP and RTCP traffic when using the BUNDLE mechanism, >>>> and instead use a payload type based mechanism for demuxing. In this >>> The term ³demuxing² is unclear. For precision, I suggest changing ³use >>> a payload type based mechanisms for demuxing² to ³use a payload type >>> based mechanism to associate RTP streams with SDP m= lines². >>> >>>> situation, each "m=" line MUST use unique payload type values, in >>>> order for the payload type to be a reliable indicator of the relevant >>>> "m=" line for the RTP stream. Note that when using payload type >>>> based demuxing, >>> Similarly, for precision, I suggest changing ³when using payload type >>> based demuxing² to ³when using the payload type to associate RTP streams >>> with m= lines². >>> >>>> an SSRC will be mapped to an ³m=³ line >>> Suggest changing to ³an RTP stream, identified by SSRC, will be >>> mappedв to be clear what¹s being done. >>> >>>> by the first >>>> packet with that SSRC, and the mapping will not be changed even if >>> Suggest changing to ³when the first RTP packet of that RTP stream is >>> received, andв to be clear, since there could be RTCP packets with the >>> same SSRC. >>> >>>> the same SSRC is received with a different payload type. In other >>> Suggest changing to ³the payload type used by that RTP stream changes. >>> In other² to be precise. >>> >>>> words, the SSRC cannot to "move" to a different "m=" line simply by >>>> changing the payload type. >>>> >>>> Holmberg, et al. Expires October 2, 2017 [Page >>>> 21] >>>> Internet-Draft Bundled media March >>>> 2017 >>>> >>>> >>>> Applications can implement RTP stacks in many different ways. The >>>> algorithm below details one way that demultiplexing can be >>>> accomplished, but is not meant to be prescriptive about exactly how >>> To be clear about what is being demultiplexed, I suggest changing ³one >>> way that demultiplexing can be accomplished² to ³one way that RTP >>> streams can be associated with m= lines². >>> >>>> an RTP stack needs to be implemented. Applications MAY use any >>>> algorithm that achieves equivalent results to those described in the >>>> algorithm below. >>>> >>>> To prepare for demultiplexing RTP/RTCP packets to the correct "m=" >>>> line, the following steps MUST be followed for each BUNDLE group. >>> I suggest changing ³Šprepare for demultiplexing RTP/RTCP packets to the >>> correctв to ³Šprepare to associate RTP streams with the correctв. RTP >>> and RTCP packets are not demultiplexed to m= lines, they¹re >>> demultiplexed into RTP streams, and then those RTP streams are >>> associated with m= lines. The distinction is important, in order to >>> implement RTCP correctly. >>> >>>> Construct a table mapping MID to "m=" line for each "m=" line in >>>> this BUNDLE group. Note that an "m=" line may only have one MID. >>>> >>>> Construct a table mapping incoming SSRC to "m=" line for each "m=" >>>> line in this BUNDLE group and for each SSRC configured for >>>> receiving in that ³m=" line. >>> The SSRC is a property of an RTP stream, so I suggest changing ³mapping >>> incoming SSRC to² to ³mapping SSRCs of incoming RTP streams to². >>> >>>> Construct a table mapping outgoing SSRC to "m=line" for each "m=" >>>> line in this BUNDLE group and for each SSRC configured for sending >>>> in that ³m=" line. >>> Similarly, change ³mapping outgoing SSRC² to ³mapping the SSRC of each >>> outgoing RTP stream² >>> >>>> Construct a table mapping payload type to "m=" line for each "m=" >>>> line in the BUNDLE group and for each payload type configured for >>>> receiving in that "m=" line. If any payload type is configured >>>> for receiving in more than one "m=" line in the BUNDLE group, do >>>> not it include it in the table, as it cannot be used to uniquely >>>> identify a "m=" line. >>>> >>>> Note that for each of these tables, there can only be one mapping >>>> for any given key (MID, SSRC, or PT). In other words, the tables >>>> are not multimaps. >>>> >>>> As "m=" lines are added or removed from the BUNDLE groups, or their >>>> configurations are changed, the tables above MUST also be updated. >>>> >>>> For each RTP packet received, the following steps MUST be followed to >>>> route the packet to the correct "m=" section within a BUNDLE group. >>> The goal is not to route RTP packets to m= lines, but rather to >>> associate the corresponding RTP streams with m= lines. This keeps the >>> distinction between RTP and RTCP processing that has to happen >>> irrespective of the m= line, and the application level processing that >>> depends on the choice of m= line. I suggest changing this sentence to: >>> ³When an RTP packet is received, it MUST be delivered to the RTP stream >>> corresponding to its SSRC. That RTP stream MUST then be associated with >>> the correct m= line within a BUNDLE group, according to the following >>> steps.² >>> >>>> Note that the phrase 'deliver a packet to the "m=" line' means to >>>> further process the packet as would normally happen with RTP/RTCP, if >>>> it were received on a transport associated with that "m=" line >>>> outside of a BUNDLE group (i.e., if the "m=" line were not BUNDLEd), >>>> including dropping an RTP packet if the packet's PT does not match >>>> any PT in the ³m=" line. >>> Dropping RTP packets with unknown payload type breaks RTCP. The RTP >>> packet must be processed by the RTP layer as normal, updating the >>> statistics that are maintained by RTCP. The payload is then discarded, >>> because there¹s no decoder associated with the payload type. I suggest >>> removing this part of the paragraph entirely. >>> >>>> If the packet has a MID, and that MID is not in the table mapping >>>> MID to ³m=" line, drop the packet and stop. >>> Dropping the RTP packet will break the RTCP reports. The RTP packet has >>> to be processed as normal, but the corresponding RTP stream is not >>> decoded if it¹s not associated with an ³m=³ line (i.e., you drop the >>> payload, not the RTP packet). I suggest replacing the above with >>> something like: >>> >>> If the MID associated with the RTP stream is not in the table mapping >>> MID to ³m=³ line, then the RTP stream is not decoded and the payload >>> data is discarded. >>> >>>> If the packet has a MID, and the packet's extended sequence number >>>> is greater than that of the last MID update, as discussed in >>>> [RFC7941], Section 4.2.6, update the incoming SSRC mapping table >>>> to include an entry that maps the packet's SSRC to the "m=" line >>>> for that MID. >>> There are two things that need to be done here: update the MID >>> associated with the RTP stream to match that in the newly received RTP >>> packet, and update the mapping table. I suggest changing ³update the >>> incoming SSRC mapping table to include an entry that maps the packet¹s >>> SSRC to the ³m=³ line for that MID² to ³update the MID associated with >>> the RTP stream to match the MID carried in the RTP packet, then update >>> the mapping tables to include an entry that maps the SSRC of that RTP >>> stream to the ³m=³ line for that MID². >>> >>>> If the packet's SSRC is in the incoming SSRC mapping table, check >>>> that the packet's PT matches a PT included on the associated "m=" >>>> line. If so, route the packet to that associated "m=" line and >>>> stop; otherwise drop the packet and stop. >>> Dropping the packet breaks RTCP. The RTP packet is processed as normal, >>> but the corresponding RTP stream is not decoded if it¹s not associated >>> with an ³m=³ line. I suggest replacing the above with: >>> >>> If the SSRC of the RTP stream is in the incoming SSRC mapping table, >>> check that the payload type used by the RTP stream matches a payload >>> type included on the matching ³m=³ line. If so, associate the RTP >>> stream with that ³m=³ line. Otherwise, the RTP stream is not decoded >>> and the payload data is discarded. >>> >>>> If the packet's payload type is in the payload type table, update >>>> the the incoming SSRC mapping table to include an entry that maps >>>> the packet's SSRC to the "m=" line for that payload type. In >>>> addition, route the packet to the associated ³m=" line and stop. >>> RTP packets correspond to RTP streams, and those RTP streams are >>> associated with m= lines. Suggest changing to: >>> >>> If the payload type used by the RTP stream is in the payload type >>> table, update the incoming SSRC mapping table to include an entry >>> that maps the RTP stream¹s SSRC to the ³m=³ line for that payload >>> type. Associate the RTP stream with the corresponding ³m=³ line. >>> >>>> Otherwise, drop the packet. >>> The packet cannot be discarded without breaking RTCP. I suggest >>> replacing the above with: >>> >>> Otherwise, mark the RTP stream as not for decoding and discard the >>> payload. >>> >>>> For each RTCP packet received (including each RTCP packet that is >>>> part of a compound RTCP packet), the packet MUST be routed to the >>>> ³m=³ line for the RTP streams it contains information about. This >>>> routing is type-dependent, as each kind of RTCP packet has its own >>>> mechanism for associating it with the relevant RTP streams. >>> The first sentence might be clearer written ³For each RTCP packet >>> received (including each RTCP packet that is part of a compound RTCP >>> packet), the packet is processed as usual by the RTP layer, then is >>> passed to the ³m=³ lines corresponding to the RTP streams it contains >>> information about for further processing.² >>> >>>> Packets for which no appropriate "m=" line can be identified (i.e., >>>> for unknown RTP streams) are not relevant in the context of this >>>> algorithm and MAY be dropped. This situation may occur with certain >>>> multiparty RTP topologies. >>> These packets can¹t be dropped. They setup state at the RTP layer that >>> might become relevant when further RTP/RTCP packets are received (RTCP >>> packets can round-robin SDES items, such as the MID, in some cases, so >>> these can potentially be important). I suggest changing this to: >>> >>> RTCP packets for which no appropriate ³m=³ line can be identified >>> MUST be processed as usual by the RTP layer, updating the metadata >>> associated with the corresponding RTP streams, but are not passed >>> to any ³m=³ line. This situation can occur with certain multiparty >>> RTP topologies, or when RTCP packets are sent containing a subset >>> of the SDES information. >>> >>>> Rules for handling the various types of RTCP packets are explained >>>> below. >>> Perhaps change to ³Rules for additional processing of the variousв, to >>> make it clear that this doesn¹t replace the usual RTCP processing. >>> >>>> If the packet is of type SDES, for each chunk in the packet whose >>> "If the RTCP packet isв >>> >>>> SSRC is found in the incoming SSRC table, deliver a copy of the >>>> packet to the "m=" line associated with that SSRC. In addition, >>> ³a copy of the SDES packet² presumably, since otherwise it¹s ambiguous >>> if the entire compound RTCP packet is delivered or just the SDES packet. >>> >>>> for any SDES MID items contained in these chunks, if the MID is >>>> found in the table mapping MID to "m=" line, update the incoming >>>> SSRC table to include an entry that maps the chunk¹s SSRC to the >>> ³Šmaps the RTP stream associated with the chunk¹s SSRC toв >>> >>>> "m=" line associated with that MID, unless the packet is older >>>> than the packet that most recently updated the mapping for this >>>> SSRC, as discussed in [RFC7941], Section 4.2.6. >>>> >>>> Note that if an SDES packet is received as part of a compound RTCP >>>> packet, the SSRC to "m=" line mapping may not exist until the SDES >>>> packet is handled (e.g., in the case where RTCP for a source is >>>> received before any RTP packets). Therefore, when processing a >>>> compound packet, any contained SDES packet MUST be handled first. >>> It might be worth referencing RFC 3550 section 6.1, which says: >>> >>> Each individual RTCP packet in the compound packet may be processed >>> independently with no requirements upon the order or combination of >>> packets. >>> >>> as justification for this. >>> >>>> Holmberg, et al. Expires October 2, 2017 [Page >>>> 23] >>>> Internet-Draft Bundled media March >>>> 2017 >>>> >>>> >>>> If the packet is of type BYE, it indicates that the RTP streams >>> ³If the RTCP packet isв >>> >>>> referenced in the packet are ending. Therefore, for each SSRC >>>> indicated in the packet that is found in the incoming SSRC table, >>> ³Šin the BYE packetв >>> >>>> first deliver a copy of the packet to the ³m=" line associated >>> ³Ša copy of the BYE packetв >>> >>>> with that SSRC, but then remove the entry for that SSRC from the >>>> incoming SSRC table after an appropriate delay to account for >>>> "straggler packets", as specified in [RFC3550], Section 6.2.1. >>>> >>>> If the packet is of type SR or RR, for each report block in the >>> ³If the RTCP packet isв >>> >>>> report whose "SSRC of source" is found in the outgoing SSRC table, >>>> deliver a copy of the RTCP packet to the ³m=" line associated with >>> ³the RTCP packet² or ³the SR or RR packet²? To avoid confusion when >>> compound RTCP packets are used, I suggest the latter phrasing here, and >>> in the later sections. >>> >>>> that SSRC. In addition, if the packet is of type SR, and the >>>> sender SSRC for the packet is found in the incoming SSRC table, >>>> deliver a copy of the packet to the ³m=" line associated with that >>> ³a copy of the SR packet²? >>> >>>> SSRC. >>>> >>>> If the implementation supports RTCP XR and the packet is of type >>>> XR, as defined in [RFC3611], for each report block in the report >>>> whose "SSRC of source" is is found in the outgoing SSRC table, >>>> deliver a copy of the RTCP packet to the ³m=" line associated with >>> ³the RTCP packet² or ³the XR packet²? >>> >>>> that SSRC. In addition, if the sender SSRC for the packet is >>>> found in the incoming SSRC table, deliver a copy of the packet to >>> ³the packet² -> ³the RTCP packet² or ³the XR packet²? >>> >>>> the "m=" line associated with that SSRC. >>>> >>>> If the packet is a feedback message of type RTPFB or PSFB, as >>> ³If the RTCP packet isв >>> >>>> defined in [RFC4585], it will contain a media source SSRC, and >>>> this SSRC is used for routing certain subtypes of feedback >>>> messages. However, several subtypes of PSFB messages include >>>> target SSRC(s) in a section called Feedback Control Information >>>> (FCI). For these messages, the target SSRC(s) are used for >>>> routing. >>>> >>>> If the packet is a feedback message that does not include target >>> ³If the RTCP packet isв >>> >>>> SSRCs in its FCI section, and the media source SSRC is found in >>>> the outgoing SSRC table, deliver the packet to the ³m=" line >>> Which packet? >>> >>>> associated with that SSRC. RTPFB and PSFB types that are handled >>>> in this way include: >>>> >>>> Generic NACK: [RFC4585] (PT=RTPFB, FMT=1). >>>> >>>> Picture Loss Indication (PLI): [RFC4585] (PT=PSFB, FMT=1). >>>> >>>> Slice Loss Indication (SLI): [RFC4585] (PT=PSFB, FMT=2). >>>> >>>> Reference Picture Selection Indication (RPSI): [RFC4585] >>>> (PT=PSFB, FMT=3). >>>> >>>> >>>> >>>> >>>> >>>> Holmberg, et al. Expires October 2, 2017 [Page >>>> 24] >>>> Internet-Draft Bundled media March >>>> 2017 >>>> >>>> >>>> If the packet is a feedback message that does include target >>> ³If the RTCP packet isв >>> >>>> SSRC(s) in its FCI section, it can either be a request or a >>>> notification. Requests reference a RTP stream that is being sent >>>> by the message recipient, whereas notifications are responses to >>>> an earlier request, and therefore reference a RTP stream that is >>>> being received by the message recipient. >>>> >>>> If the packet is a feedback request that includes target SSRC(s), >>> ³If the RTCP packet isв >>> >>>> for each target SSRC that is found in the outgoing SSRC table, >>>> deliver a copy of the RTCP packet to the "m=" line associated with >>>> that SSRC. PSFB types that are handled in this way include: >>>> >>>> Full Intra Request (FIR): [RFC5104] (PT=PSFB, FMT=4). >>>> >>>> Temporal-Spatial Trade-off Request (TSTR): [RFC5104] (PT=PSFB, >>>> FMT=5). >>>> >>>> H.271 Video Back Channel Message (VBCM): [RFC5104] (PT=PSFB, >>>> FMT=7). >>>> >>>> Layer Refresh Request (LRR): [I-D.ietf-avtext-lrr] (PT=PSFB, >>>> FMT=TBD). >>>> >>>> If the packet is a feedback notification that include target >>>> SSRC(s), for each target SSRC that is found in the incoming SSRC >>>> table, deliver a copy of the RTCP packet to the "m=" line >>>> associated with that SSRC. PSFB types that are handled in this >>> ³deliver a copy of the RTCP packet to the ³m=³ line associated with the >>> RTP stream with matching SSRC² >>> >>>> way include: >>>> >>>> Temporal-Spatial Trade-off Notification (TSTN): [RFC5104] >>>> (PT=PSFB, FMT=6). This message is a notification in response >>>> to a prior TSTR. >>>> >>>> If the packet is of type APP, the only routing information >>>> included is the source of the packet, and therefore the packet >>>> could be related to any existing "m=" line. Accordingly, deliver >>>> a copy of the packet to each ³m=" line. >>> Are APP packets exposed in the WebRTC APIs? Given that we have the data >>> channel, it¹s not clear that we want to support arbitrary application >>> information passing via RTCP. It might be better to say that APP packets >>> are processed in an application specific manner, and if the application >>> doesn¹t understand them, they¹re not passed to any m= line? >>> >>> Finally, I note that this section of the draft doesn¹t mention the CSRC >>> list anywhere, but should probably do so. As written, RTP packets with a >>> CSRC list will be delivered to the RTP stream matching their SSRC in the >>> usual way, then passed up to the m= line associated with that RTP >>> stream. That may well be sufficient, but if so it¹s likely useful to say >>> that, to make the intent clear. >>> >>> Colin >>> >>> >>> >>> >>> -- >>> Colin Perkins >>> https://csperkins.org/ >>> >>> >>> >>> >>> _______________________________________________ >>> mmusic mailing list >>> [email protected] >>> https://www.ietf.org/mailman/listinfo/mmusic >>> _______________________________________________ >>> mmusic mailing list >>> [email protected] >>> https://www.ietf.org/mailman/listinfo/mmusic >> >> >> -- >> Colin Perkins >> https://csperkins.org/ >> >> >> >> > _______________________________________________ > mmusic mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/mmusic > . >