Considerations for Selecting RTCP Extended Report (XR) Metrics for the RTCWEB Statistics API
draft-huang-xrblock-rtcweb-rtcp-xr-metrics-02

Network Working Group                                           R. Huang
INTERNET-DRAFT                                                   R. Even
Intended Status: Informational                                    Huawei
Expires: April 20, 2014                                         V. Singh
                                                        Aalto University
                                                            D. Romascanu
                                                                   Avaya
                                                        October 17, 2013

        Considerations for Selecting RTCP Extended Report (XR) 
                 Metrics for the RTCWEB Statistics API
             draft-huang-xrblock-rtcweb-rtcp-xr-metrics-02

Abstract

   This document describes monitoring features related to RTCWEB. It
   provides a list of RTCP XR metrics that are useful and may need to be
   supported in some RTCWEB implementations.

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

Copyright and License Notice

   Copyright (c) 2013 IETF Trust and the persons identified as the
   document authors. All rights reserved.

 

<Huang, et al.>          Expires April 20, 2014                 [Page 1]
INTERNET DRAFT        <RTCP XR Metrics for RTCWEB>      October 17, 2013

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document. Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1  Introduction  . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2  Terminology . . . . . . . . . . . . . . . . . . . . . . . . . .  3
   3 Considerations for Selecting Monitoring Metrics  . . . . . . . .  3
     3.1 Considerations for Impact of Measurement Interval  . . . . .  4
     3.2 Candidate Metrics  . . . . . . . . . . . . . . . . . . . . .  4
       3.2.1 Loss and Discard Packet Count Metric . . . . . . . . . .  4
       3.2.2 Discard Octets Metric  . . . . . . . . . . . . . . . . .  5
       3.2.3 Retransmitted and Post-repair Packet Count Metric  . . .  6
       3.2.4 Frame Impairment Summary Metrics . . . . . . . . . . . .  6
       3.2.5 Burst/Gap Pattern Metrics for Loss and Discard . . . . .  7
       3.2.6 Run Length Encoded Metrics for Loss, Discard and
             Post-repair  . . . . . . . . . . . . . . . . . . . . . .  8
       3.2.7 Jitter Buffer Metrics  . . . . . . . . . . . . . . . . .  8
   4  Security Considerations . . . . . . . . . . . . . . . . . . . .  9
   5  IANA Considerations . . . . . . . . . . . . . . . . . . . . . .  9
   6. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . .  9
   7  References  . . . . . . . . . . . . . . . . . . . . . . . . . . 10
     7.1  Normative References  . . . . . . . . . . . . . . . . . . . 10
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 12

 

<Huang, et al.>          Expires April 20, 2014                 [Page 2]
INTERNET DRAFT        <RTCP XR Metrics for RTCWEB>      October 17, 2013

1  Introduction

   Web based real-time communication (WebRTC) is becoming prevalent. To
   help measure the quality of the WebRTC services better, applications
   need the ability to estimate the service quality and to inform about
   network problems. If sufficient information (metrics or statistics)
   are provided to the applications, it can function better in providing
   better media quality. [RTCWEB-REQ] specifies a requirement for
   statistics, which is listed below for convenient reading.

   "F38    The browser MUST be able to collect statistics, related to
   the transport of audio and video between peers, needed to estimate
   quality of service." 

   [RTCWEB-STAT] describes a registration procedure for choosing metrics
   reported by the Javascript API. It also identifies basic metrics
   reported in the RTCP Sender and Receiver Report (SR/RR) to fulfill
   this requirement. They are SentPacketCount, SentOctetCount,
   packetsLost, Jitter, ReceivedPacketCount, ReceivedOctetCount.
   However, these basic metrics from RTCP SR/RR may not be sufficient
   for precise quality monitoring or troubleshooting. For example,
   packetsLost is controversial and could be misleading (see section
   3.2.3 for discussion). They're better to be complemented with
   correspondent metrics defined in RTCP XR. Thus, indicating a minimum
   set of additional statistic metrics would be helpful. In this
   document, we provide some guidelines on how to choose these
   additional metrics and on what kind of metrics should be chosen to
   complement the metrics from basic RTCP SR/RR specified in [RTCWEB-
   STAT].

2  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

3 Considerations for Selecting Monitoring Metrics

   There are two ways to convey these metrics in WebRTC. One way is the
   Javascript application extracts the local statistics from the
   browser's internals using an API. Since the media path goes directly
   between browsers, the application is able to query the statistics
   information directly from the local RTP stack. The remote-side
   information could be fetched by some other means outside the scope of
   WebRTC.

   Another possible method is to use RTCP XRs, implemented in the
 

<Huang, et al.>          Expires April 20, 2014                 [Page 3]
INTERNET DRAFT        <RTCP XR Metrics for RTCWEB>      October 17, 2013

   browser to allow sending the statistics information directly between
   endpoints. Since RTP is used as the media transport protocol for
   RTCWEB and RTCP XR can provide more useful statistics information
   than RTCP SR/RR concerning media quality. However, no RTCP report
   extensions are currently mandated in RTCWEB for monitoring because
   the chosen metrics usually depend on the application. It is agreed
   that these RTP monitoring extensions could be supported later by SDP
   negotiation between browsers. So at the current stage, we should only
   consider those metrics that can be measured at a local endpoint and
   useful for WebRTC. 

   Since RTP is the media transport protocol in RTCWEB, we mainly
   discuss the RTP based monitoring metrics.  RTCP SR/RR collects
   information and reports it periodically.  However, it only provides
   partial information, which means that one can use it in a limited way
   to identify network problems. This may not be sufficient for
   diagnosing problems or performance monitoring. RTP Control Protocol
   Extended Reports [RFC3611] and other extensions discussed in XRBLOCK
   working group have defined many monitoring metrics that complement
   the RTCP SR/RR. These metrics are useful for a range of RTP
   applications. In this chapter some recommendations are provided to
   help RTCWEB applications choose the metrics specified in RTCP XR and
   other extensions defined in XRBLOCK working. 

3.1 Considerations for Impact of Measurement Interval

   RTCP extensions like RTCP XR usually share the same timing interval
   with RTCP SR/RR, i.e., they are sent as compound packets, together
   with the RTCP SR/RR. Alternatively, the RTCP XR can use a different
   measurement interval and all XRs using the same measurement interval
   are compounded together and the measurement interval is indicated in
   a specific measurement information block [RFC6776].

   When using WebRTC Statistics APIs (see section 7 of [WebRTCAPI]),
   Javascript applications can query this information at arbitrary
   intervals. Some applications may choose 1 second or another interval.
   Currently, RTCP XRs are not required to be implemented by the
   browsers; the endpoint can query only local statistics. In the
   following sections, we only discuss about the metrics, which are
   mainly local reception statistics.  

3.2 Candidate Metrics

   Following candidate types of metrics are sorted by their importance. 

3.2.1 Loss and Discard Packet Count Metric

   In multimedia transport, packets that are received abnormally are
 

<Huang, et al.>          Expires April 20, 2014                 [Page 4]
INTERNET DRAFT        <RTCP XR Metrics for RTCWEB>      October 17, 2013

   classified into 3 types: lost, discarded and duplicate packets.
   Packet loss may be caused by network devices breakdown, bit-error
   corruption or serious congestions (packets dropped by an intermediate
   router queue). Duplicated data packets may be due to a slight long
   network delay which causes the sender to retransmit the original
   packets. Discarded packets are packets that have been delayed long
   enough and are considered useless by the receiver. Lost and discarded
   packets cause problems for multimedia services, as missing data and
   long delay can cause degradation in service quality, e.g., large
   blocks of packets that are missing (lost or discarded) at once may
   cause choppy audio, and long network transmission time may cause
   audio or video buffering. RTCP SR/RR defines a metric for counting
   the total number of RTP data packets that have been lost since the
   beginning of reception. But this statistic doesn't distinguish lost
   packets from discarded and duplicate packets. Packets that arrive
   late and are discarded are not treated as lost, and duplicate packets
   will be regarded as a normally received packet. This metric is
   misleading if many duplicate packets are received or packets
   discarded, which causes the quality of media transport to look okay
   from the statistic while actually users are experiencing bad service
   quality, because packets are still missing. So in such cases, it's
   better to use more accurate metrics in addition to those defined in
   RTCP SR/RR.

   The lost packets and duplicated packets metrics defined in Statistics
   Summary Report Block of [RFC3611] extend the information of loss
   carried in standard RTCP SR/RR. They explicitly give an account of
   lost and duplicated packets. Lost packets counts are useful for
   network problem diagnosis. It's better to use the loss packets
   metrics of [RFC3611] to indicated the packet lost counting instead of
   the cumulative number of packets lost metric of [RFC3550]. Duplicated
   packets are usually rare and have little effect on QoS evaluation. So
   it's not suitable to be recommended in RTCWEB scenarios. 

   Using loss metrics without considering discard metrics may result in
   inaccurate quality evaluation, as packet discard due to jitter is
   often more prevalent than packet loss in modern IP networks. The
   discarded metric specified in [RFC7002] counts the number of packets
   discarded due to the jitter. It augments the loss statistics metrics
   specified in standard RTCP SR/RR. For those RTCWEB services with
   jitter buffer requiring precise quality evaluation and accurate
   troubleshooting, this metric is useful as a complement to the metrics
   of RTCP SR/RR.

3.2.2 Discard Octets Metric

   The metric reports the cumulative size of the packets discarded in
   the interval, it is complementary to number of discarded packets. An
 

<Huang, et al.>          Expires April 20, 2014                 [Page 5]
INTERNET DRAFT        <RTCP XR Metrics for RTCWEB>      October 17, 2013

   application measures sent octets and received octets to calculate
   sending rate and receiving rate, respectively. The application can
   calculate goodput in a particular interval by subtracting the
   discarded octets from the received octets. 

   For WebRTC, discarded octets supplements the sent and received octets
   and provides an accurate method for calculating goodput. The
   discarded bytes metric is defined in [XRBLOCK-DISCARDBYTES].

3.2.3 Retransmitted and Post-repair Packet Count Metric

   RTP retransmission is not required to be implemented in RTCWEB. As
   depicted in [RTCWEB-RTPUSAGE], NACKs may be sent by receivers to
   indicate missing RTP packets and senders may send retransmission
   packets in response to these NACKs. In low delay networks with low
   loss rates, retransmission has great value without incurring
   additional complexity. Providing some retransmission statistic
   information in such applications could help to provide a more
   accurate QoS evaluation since retransmission could greatly reduce the
   impact of packet loss. 

   Number of retransmission packets metric counts the retransmitted
   packets that are successfully received by receivers. It could be used
   for quality evaluation in RTCWEB systems that has negotiated to
   support the transmission mechanism. The number of retransmitted
   packets is subtracted from the number of lost packets, which
   indicates the residual lost packets. 

   When RTCWEB applications uses error-resilience mechanisms like
   Forward Error Correction (FEC) or retransmission, the post-repair
   packets count defined in [XRBLOCK-PRCOUNT] provides the success
   information of the error-resilience mechanisms to the monitoring
   application or the sending endpoint. The endpoint can correlate the
   loss and post-repair loss to ascertain the ratio of repaired packets
   to lost packets. Including this kind of metrics helps the application
   evaluate the effectiveness of the applied repair mechanisms.      

3.2.4 Frame Impairment Summary Metrics

   RTP has different framing mechanisms for different payload types. For
   audio streams, a single RTP packet may contain one or multiple audio
   frames, each of which has a fixed length. On the other hand, in video
   streams, a single video frame may be transmitted in multiple RTP
   packets. The size of each packet is limited by the Maximum
   Transmission Unit (MTU) of the underlying network. However,
   statistics from standard SR/RR only collect information from
   transport layer, which may not fully reflect the quality observed by
   the application. Video is typically encoded using two frame types
 

<Huang, et al.>          Expires April 20, 2014                 [Page 6]
INTERNET DRAFT        <RTCP XR Metrics for RTCWEB>      October 17, 2013

   i.e., key frames and derived frames. Key frames are normally just
   spatially compressed, i.e., without prediction from other pictures.
   The derived frames are temporally compressed, i.e., depend on the key
   frame for decoding. Hence, Key frames are much larger in size than
   derived frames. The loss of these key frames results in a substantial
   reduction in video quality. Thus it is meaningful to consider this
   application layer information in WebRTC implementations, which
   influence sender strategies to mitigate the problem or require the
   accurate assessment of users' quality of experience.

   [RFC7003] defines metrics conveyed in RTCP XR by receivers reporting
   to senders or other monitor devices. The following metrics can also
   be considered for WebRTC's Statistics API: number of discarded key
   frames, number of duplicated key frames, number of fully lost key
   frames, number of partial lost key frames, number of discarded
   derived frames, number of duplicated derived frames, number of full
   lost derived frames, number of partial lost derived frames. Details
   of the definition of these metrics are in [RFC7003].

3.2.5 Burst/Gap Pattern Metrics for Loss and Discard

   RTCP SR/RR defines coarse metrics regarding loss statistics, the
   metrics are all about per call statistics and not detailed enough to
   capture some transitory nature of the impairments like bursty packet
   loss. Even if the average packet loss rate is low, the lost packets
   are likely to occur during short dense periods, resulting in short
   periods of degraded quality. Thus, bursty packet loss has a severe
   impact on media quality. Distributed burst provides a higher
   subjective quality than a non burst distribution for low packet loss
   rates whereas for high packet loss rates the converse is true. So
   capturing burst gap information is very helpful for quality
   evaluation and locating impairments. If RTCWEB services have the
   requirement to evaluate the services quality, burst gap metrics
   provides more accurate information than RTCP SR/RR.

   [RFC3611] introduces burst gap metrics in VoIP report block. These
   metrics record the density and duration of burst and gap periods,
   which are helpful in isolating network problems since bursts
   correspond to periods of time during which the packet loss/discard
   rate is high enough to produce noticeable degradation in audio or
   video quality. Burst gap related metrics are also introduced in
   [RFC7003] and [RFC6958] which define two new report blocks for usage
   in a range of RTP applications beyond those described in RFC3611.
   These metrics distinguish discarded packets from loss packets that
   occur in the bursts period and provides more information for
   diagnosing network problems. Besides that, the metric number of
   bursts counts the burst events which could provide useful information
   to evaluate the frequency of burst occurrences. So if WebRTC services
 

<Huang, et al.>          Expires April 20, 2014                 [Page 7]
INTERNET DRAFT        <RTCP XR Metrics for RTCWEB>      October 17, 2013

   have the requirement to do quality evaluation and observe when and
   why quality degrades, these metrics should be considered.

3.2.6 Run Length Encoded Metrics for Loss, Discard and Post-repair

   Run-length encoding uses a bit vector to encode information about the
   packet. Each bit in the vector represents a packet and depending on
   the signaled metric it defines if the packet was lost, duplicated,
   discarded, or repaired. An endpoint typically uses the run length
   encoding to accurately communicate the status of each packet in the
   interval to the other endpoint. [RFC3611], [XRBLOCK-DISCARDRLE] and
   [RFC5725] define run-length encoding for lost and duplicate packets,
   discarded packets and post-repair packets. 

   For WebRTC, the application may benefit from the additional
   information. If losses occur after discards, an endpoint may be able
   to correlate the two run length vectors to identify congestion-
   related losses, i.e., a router queue  became overloaded causing
   delays and then overflowed. If the losses are independent, it may
   indicate bit-error corruption. Lastly, when using error-resilience
   mechanisms, the endpoint can correlate the loss and post-repair run
   lengths to ascertain where the losses and repairs occurred in the
   interval. For example, consecutive losses are likely not to be
   repaired by a simple FEC scheme.

3.2.7 Jitter Buffer Metrics

   The size of the jitter buffer affects the end-to-end delay on the
   network and also the packet discard rate. When the buffer size is too
   small, slower packets are not played out and dropped, while when the
   buffer size is too large, packets are held longer than necessary and
   consequently reduce conversational quality. Measurement of jitter
   buffer should not be ignored in the evaluation of end user perception
   of conversational quality. Jitter buffer related metrics, such as
   maximum and nominal jitter buffer, could be used to show how the
   jitter buffer behaves at the receiving end of RTP stream. They are
   useful for providing better end-user quality of experience (QoE) when
   jitter buffer factors are used as inputs to calculate QoE metrics.
   RTCWEB services are point-to-point connections. Usually, senders
   don't care what the perception quality of the remote end is. But in
   some cases, it may be meaningful for receivers to send this kind of
   information to senders telling what the media quality the receiver is
   being through. For example, senders who have the ability to adjust
   the media codecs may require to know the quality of the receivers so
   that they can switch to a lower bandwidth usage codec when service
   degradation happens. Thus for those cases, jitter buffer metrics
   could be considered. The definition of these metrics could be found
   in [RFC7005].
 

<Huang, et al.>          Expires April 20, 2014                 [Page 8]
INTERNET DRAFT        <RTCP XR Metrics for RTCWEB>      October 17, 2013

4  Security Considerations

   The monitoring activities are implemented between two browsers or
   browser-to-server.  Also encryption procedures, such as those being
   suggested for a Secure RTCP (SRTCP), can be used.  It is believed
   that monitoring in RTCWEB introduces no new security considerations
   beyond those described in [RTCWEB-RTPUSAGE] and [RTCWEB-SECURITY].

5  IANA Considerations

   There is no IANA action in this document.

6. Acknowledgement

   The authors would like to thank Colin Perkins for their valuable
   comments and suggestions on this document.

 

<Huang, et al.>          Expires April 20, 2014                 [Page 9]
INTERNET DRAFT        <RTCP XR Metrics for RTCWEB>      October 17, 2013

7  References

7.1  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC3550]  Schulzrinne, H., Casner, S., Frederick, R., and V.
              Jacobson, "RTP: A Transport Protocol for Real-Time
              Applications", STD 64, RFC 3550, July 2003.

   [RFC3611]  Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed.,
              "RTP Control Protocol Extended Reports (RTCP XR)",
              RFC 3611, November 2003.

   [RTCWEB-REQ] Holmberg, C., Hakansson, S., and G. Eriksson, "Web Real-
              Time Communication Use-cases and Requirements", I-D.ietf-
              rtcweb-use-cases-and-requirements, December 2012.

   [RTCWEB-STAT] Alvestrand, H., "A Registry for WebRTC statistics
              identifiers", I-D.alvestrand-rtcweb-stats-registry,
              September 24, 2012.

   [RTCWEB-RTPUSAGE] Perkins, C., Westerlund, M., and J. Ott, "Web Real-
              Time Communication (WebRTC): Media Transport and Use of
              RTP", I-D.ietf-rtcweb-rtp-usage, February 2013.

   [RTCWEB-SECURITY] Rescorla, E., "Security Considerations for RTC-
              Web", I-D.ietf-rtcweb-security, January 2013.

   [RFC6792] Wu, Q., Ed., Hunt, G., and P. Arden, "Monitoring
              Architecture for RTP", RFC 6792, November 2012.

   [RFC7002] Hunt, G., Clark, A., Zorn, G., and Q. Wu, "RTCP XR Report
              Block for Discard Metric Reporting", RFC 7002, September
              2013.

   [XRBLOCK-DISCARDBYTES] Singh, V., Ott, J., Curcio, I.D.D., "RTP
              Control Protocol (RTCP) Extended Reports (XR) for Bytes
              Discarded Metric", I-D.ietf-xrblock-rtcp-xr-bytes-
              discarded-metric, October 2013

   [XRBLOCK-PRCOUNT] Huang, R., Singh, V., "RTP Control Protocol (RTCP)
              Extended Report (XR) for Post-Repair Non-Run Length
              Encoding (RLE) Loss Count Metrics", I-D.huang-xrblock-
              post-repair-loss-count, September 2013.

   [RFC7003] Hunt, G., Clark, A., and Q. Wu, Ed., "RTCP XR Report Block
 

<Huang, et al.>          Expires April 20, 2014                [Page 10]
INTERNET DRAFT        <RTCP XR Metrics for RTCWEB>      October 17, 2013

              for Burst/Gap Discard Metric Reporting", RFC 7003,
              September 2013.

   [RFC6958] Hunt, G., Clark, A., Zhang, S., Ed., "RTCP XR Report Block
              for Burst/Gap Loss Metric Reporting", RFC 6958, April
              2013.

   [XRBLOCK-DISCARDRLE] Singh, V., Ott, J., Curcio, I.D.D., "RTP Control
              Protocol (RTCP) Extended Reports (XR) for  Run Length
              Encoding (RLE) of Discarded Packets", I-D.ietf-xrblock-
              discard-rle-metrics, October 2013.

   [RFC5725] Begen, A., Hsu, D., Lague, M., "Post-Repair Loss RLE Report
              Block Type for RTP Control Protocol (RTCP) Extended
              Reports (XRs)", February 2010

   [RFC7005] Clark, A., Singh, V., and Q. Wu, "RTCP XR Report Block for
              Jitter Buffer Metric Reporting", RFC 7005, September 2013 

   [RFC6798] Clark, A., Wu, Q., Ed., "RTCP Control Protocol (RTCP)
              Extended Report (XR) Block for Packet Delay Variation
              Metric Reporting", November 2012.

   [RFC7003] Zorn, G., Schott, R., Wu, Q., Huang, R., "RTP Control
              Protocol (RTCP) Extended Report (XR) Blocks for Summary
              Statistics Metrics Reporting", September 2013

   [WebRTCAPI] Bergkvist, A., Burnett, D., Jennings, C., Ed.,
              http://dev.w3.org/2011/webrtc/editor/webrtc.html, June
              2013

 

<Huang, et al.>          Expires April 20, 2014                [Page 11]
INTERNET DRAFT        <RTCP XR Metrics for RTCWEB>      October 17, 2013

Authors' Addresses

   Rachel Huang
   Huawei
   101 Software Avenue, Yuhua District
   Nanjing 210012
   China

   EMail: rachel.huang@huawei.com

   Roni Even
   Huawei
   14 David Hamelech
   Tel Aviv 64953
   IsraelOctober 11, 2013October 11, 2013

   EMail: roni.even@mail01.huawei.com

   Varun Singh
   Aalto University
   School of Electrical Engineering
   Otakaari 5 A
   Espoo, FIN  02150
   Finland

   Email: varun@comnet.tkk.fi
   URI:   http://www.netlab.tkk.fi/~varun/

   Dan Romascanu
   Avaya

   Email: Email: dromasca@avaya.com

<Huang, et al.>          Expires April 20, 2014                [Page 12]