Liaison statement
LS/r to ITU-T SG11 on “QoS-related work in ITU-T SG11” (SG11-LS 105) [from ITU-T SG12]

State Posted
Posted Date 2016-02-02
From Group ITU-T-SG-12
From Contact Judit Kiss
To Group ippm
To Contacts Brian Trammell
Bill Cerveny
Scott Mansfield
Bill Cerveny
Brian Trammell
Spencer Dawkins
IP Performance Metrics Discussion List
Martin Stiemerling
Response Contact
Purpose For information
Attachments oLS 095-td863R1
oLS 095att-td862R1
Study Group 12 thanks you for your liaison reply concerning the development of
Recommendations closely related to our mandate, as lead Study Group on Quality
of Service and Quality of Experience. 

We recognize the revisions to the Scope of Q.3960 (the former
Q.FW_Int_sp_test) to change the emphasis to end users of an IP packet transfer
service and supporting infrastructure, and to “estimate the access speed to
the Internet and to the Internet resources” using measurements that “can be
established at the national or international level” (from the current scope).
However, the terms “Internet Access”, Internet Connection”, and “Internet
Service” are not currently defined in the ITU terms and definitions, so it is
necessary to de-scope aspects describing international applicability. National
definitions of these terms may exist, but with considerable variability, again
putting the applicability of this draft text into question.

In our prior liaison reply, we asked that you take advantage of in-force
Recommendation Y.1540, which covers many critical aspects of the performance
of IP-based Networks (beyond highlighting the requirements of  a single
section which we considered particularly relevant to the draft text you shared
with us). This Recommendation provides a wide range of fundamental and
secondary performance parameters – all defined at the IP layer. The IP-layer
is the layer with end-to-end significance to Internet service providers.
Higher layers are implemented in hosts beyond the control of service
providers, and the headers of higher layers are part of the payload octets
conveyed in an IP packet transfer service. Thus, the availability of IP packet
transfer and the performance of IP packet transfer attempts determine the
quality of the service, according to the metrics defined in Y.1540. Other
metrics are under study and should be removed from the scope from any ITU-T
Recommendation (Q.FW_Int_sp_test) until on-going study is complete.

Your Liaison also mentions a new Recommendation planned for development in
collaboration with ETSI INT, and which overlaps with the existing
Recommendations of Q13/12. The true nature of this Recommendation was not
clear from the description in the liaison, and was only clarified by your
counsellor in response to questions.  Again, this planned effort is apparently
un-aware of in-force Recommendations of the lead SG on the topics where you
intend to design tests. The literature of SG 12 Question 13 must be studied
and appreciated before you proceed:
	G.1010    End-user multimedia QoS categories
	G.1031    QoE factors in web-browsing
	G.1080    Quality of experience requirements for IPTV services
	G.1091    Quality of Experience requirements for telepresence services

In the brief interval allowed for us to prepare a liaison response, we list
the following additional issues with your proposed text under Consent in

•	SG 11 believes that Y.1540 and IPPM RFCs are primarily relevant to the
second phase of their Internet Speed development on test methodology. The SG
12 LR did not ask for references to be added to the list, it asked for the
development of text to take advantage of the in-force specifications. Although
Recommendation Y.1541 is an existing reference, there is no citation in the
text. The same is true for BBF Technical Report 304 (2015), which references
many IETF RFCs, and the TSB Editors could rightly remove these references from
the list (since they are not cited in any way).
•	All Figures in the Consented Internet Speed Test Framework (Q. 3960) are now
designated as examples. Examples or not, the Figures lack the detail to
describe service scope, and so all discussion of Service Level Agreements
(SLAs) is inappropriate.
•	In a brief review of the text of Q. 3960, there was strong opposition
expressed to use of a “single Internationally recognized entity” to provide
Internet Speed test facilities. AT&T and DT representatives expressed that the
idea of a single entity having control of this testing was completely
unacceptable (and probably not feasible, due to the multiplicity of peering
arrangements required). Every Recommendation must allow for multiple
independent implementations.
•	The measurement test definitions seek to measure the “absolute value” of
transmission speed between different measurement points. However, there is no
known approach which provides such a value. In fact, the surveys of Internet
performance on which this framework is partly based have been informative as
relative measures comparing different technologies of several service
providers, but do not pretend to quantify anything in absolute terms. It must
be made clear that any tests used to judge the performance of IP network
service providers necessarily exclude factors beyond the service scope (e.g.,
the User-Network Interface, UNI, and Network-Network Interface, NNI).

Other comments are embedded in the text of Q. 3960, attached.

We reiterate that descriptions of popular measurement studies do not
constitute a basis for an international standard, as they do not possess the
necessary specificity to guarantee equivalent results from multiple
independent implementations, or necessarily use performance parameters that
exhibit important attributes such as repeatability.

We urge you to continue study of the relevant Recommendations, IETF RFCs, and
other work-in-progress, and to suspend plans to seek approval for all related
activities in SG 11. We invite you to join SG12 at our meeting in June 2016,
where we will be able to share additional background on IP-based network
performance parameters and methods of measurement. 

We are unable to send a representative to the Joint meeting you have arranged
with ETSI INT, and we apologize for our absence. If we are to coordinate and
collaborate on this topic, then the first step is to coordinate among the
calendars of key organizations and their interested participants.

Attachment: TD862 Rev.1 (Comments on the Last Call Text of Draft new
Recommendation ITU-T Q. 3960)