Video Frame Marking RTP Header Extension
draft-ietf-avtext-framemarking-16

[Ballot comment]
Thanks for working on this specification.

I hope to see the experimental results, but missing any description about what we are experimenting and any success criteria. I think it would be nice to have a section on suggested experiments. It could describe the intended usefulness of this extension and if successful becomes a standard track specification; also can document any security or privacy concern about the usage of meta information.

[Ballot comment]
Thanks for working on this specification.

I hope to see the experimental results, but missing any description about what we are experimenting and any success criteria. I think it would be nice to have a section on suggested experiments. It could describe the intended usefulness of this extension and if successful becomes a standard track specification also can document any security or privacy concern about the usage of meta information.

[Ballot comment]
Thanks to Carl Wallace for the SECDIR review.

I support Paul’s DISCUSS for needing to discuss the privacy considerations.

My refinement on his position is whether there would be any circumstance where an attacker knowing that an encrypted payload has a I-frame (as revealed by this extension) would provide additional information in traffic analysis style attack (i.e., observing the changes in the rate of the I-frames).  A hypothetical scenario might be a variable rate codecs that reduces the I-frame rate in the absence of  motion in the video (to save bandwidth), and increases the I-frame rate when there is motion (to improve the fidelity of the video stream).  In such a circumstance, traffic analysis would reveal at least something about the encrypted payload (i.e., the presence or absence of motion)

[Ballot discuss]
Thanks to Carl Wallace for their secdir review: https://datatracker.ietf.org/doc/review-ietf-avtext-framemarking-13-secdir-lc-wallace-2022-04-11
which raises a good point about referencing RFC 8285 and authenticated headers.

Additionally, I wonder why this document has no Privacy Considerations Section. By exposing previously encrypted information on the outer header,
some private information is exposed. For example, when watching a TV or live event stream, just by exposing the Independent Frame bit one might
be able to deduce the content being watched. And perhaps advise could be added to not send these bits all the time, but only when an RTP Switch
requests it specifically (and then deny excessively repeated attempts to foil fingerprinting the stream). Please consider adding a Privacy Considerations
section, or let me know why this would not be useful

[Ballot comment]
Hi,

No issues with this document, that I generally found pretty easy to read.

In general, I'm pleased to see IETF working on solutions in this space (specifically, experimenting with what information can/should be exposed to middle boxes to improve the end-user experience whilst limiting the leakage of private information).  I'm not an expert in this area, but if exposing some limited information to middle boxes allows the video streams to remain encrypted, then this arguably ends up increasing overall end user privacy (e.g., compared with the middle box decrypting everything).

Regards,
Rob

[Ballot comment]
# GEN AD review of draft-ietf-avtext-framemarking-14

CC @larseggert

Thanks to Gyan S. Mishra for the General Area Review Team (Gen-ART) review
(https://mailarchive.ietf.org/arch/msg/gen-art/I-fIfxfS98c6BVePYcpjP3Hf7ng).

## Comments

### Boilerplate

This document uses the RFC2119 keywords "MUST NOT", "MUST", "RECOMMENDED",
"MAY", "SHOULD", and "NOT RECOMMENDED", but does not contain the recommended
RFC8174 boilerplate. (It contains a variant of the RFC2119 boilerplate.)

## Nits

All comments below are about very minor potential issues that you may choose to
address in some way - or ignore - as you see fit. Some were flagged by
automated tools (via https://github.com/larseggert/ietf-reviewtool), so there
will likely be some false positives. There is no need to let me know what you
did with these suggestions.

### Grammar/style

#### Section 1, paragraph 3
```
by recipients without prior frames, e.g switch on an intra-frame. * In many
                                    ^^^
```
The abbreviation "e.g." (= for example) requires two periods.

#### Section 3.5, paragraph 2
```
orm to common, fixed patterns of inter-layer dependencies and referencing str
                                ^^^^^^^^^^^
```
This word is normally spelled as one.

#### Section 3.5, paragraph 2
```
cies and referencing structures. Therefore it is RECOMMENDED to use LID and
                                ^^^^^^^^^
```
A comma may be missing after the conjunctive/linking adverb "Therefore".

## Notes

This review is in the ["IETF Comments" Markdown format][ICMF], You can use the
[`ietf-comments` tool][ICT] to automatically convert this review into
individual GitHub issues. Review generated by the [`ietf-reviewtool`][IRT].

[ICMF]: https://github.com/mnot/ietf-comments/blob/main/format.md
[ICT]: https://github.com/mnot/ietf-comments
[IRT]: https://github.com/larseggert/ietf-reviewtool

[Ballot comment]

Thank you for the work put into this document.

Please find below some non-blocking COMMENT points.

Special thanks to Jonathan Lennox for the shepherd's detailed write-up including the WG "consensus" and the justification of the intended status.

I hope that this review helps to improve the document,

Regards,

-éric


## Section 1

It would be nice to expand "MCU".

## Section 2

Why having a section about normative language (moreover referring only to RFC 2119) in an experimental doc ?

[Ballot comment]
Thanks to Jonathan Lennox for the careful shepherd write up. When I read

“The document has been reviewed by WG members, and appears to be
complete insofar as it goes; however, there are concerns that it is
not sufficient for the uses cases for which it was intended.
Notably, the significantly more complex AV1 Dependency Descriptor
[https://aomediacodec.github.io/av1-rtp-spec/#appendix] addresses a
similar use-case much more thoroughly.”

It led me to wonder whether it would be a good idea to make some mention of this in the present document, for the edification of people referring to it who might not know about the more comprehensive solution?

(Via drafts-lastcall@iana.org): IESG/Authors/WG Chairs:

The IANA Functions Operator has completed its review of draft-ietf-avtext-framemarking-13. If any part of this review is inaccurate, please let us know.

The IANA Functions Operator understands that, upon approval of this document, there is a single action which we must complete.

In the RTP Compact Header Extensions registry on the Real-Time Transport Protocol (RTP) Parameters registry page located at:

https://www.iana.org/assignments/rtp-parameters/

a new registration will be made as follows:

Extension URI: urn:ietf:params:rtp-hdrext:framemarkinginfo
Description: Frame marking information for video streams
Contact: mzanaty@cisco.com
Reference: [ RFC-to-be ]

As this document requests registrations in an Expert Review (see RFC 8126) registry, we will initiate the required Expert Review via a separate request. This review must be completed before the document's IANA state can be changed to "IANA OK."

The IANA Functions Operator understands that this is the only action required to be completed upon approval of this document.

Note:  The actions requested in this document will not be completed until the document has been approved for publication as an RFC. This message is meant only to confirm the list of actions that will be performed.

Thank you,

Sabrina Tanamal
Lead IANA Services Specialist

The following Last Call announcement was sent out (ends 2022-04-08):

From: The IESG
To: IETF-Announce
CC: Jonathan Lennox , avt@ietf.org, avtcore-chairs@ietf.org, draft-ietf-avtext-framemarking@ietf.org, jonathan.lennox@8x8.com, superuser@gmail.com
Reply-To: last-call@ietf.org
Sender:
Subject: Last Call:  (Frame Marking RTP Header Extension) to Experimental RFC


The IESG has received a request from the Audio/Video Transport Core
Maintenance WG (avtcore) to consider the following document: - 'Frame Marking
RTP Header Extension'
  as Experimental RFC

The IESG plans to make a decision in the next few weeks, and solicits final
comments on this action. Please send substantive comments to the
last-call@ietf.org mailing lists by 2022-04-08. Exceptionally, comments may
be sent to iesg@ietf.org instead. In either case, please retain the beginning
of the Subject line to allow automated sorting.

Abstract


  This document describes a Frame Marking RTP header extension used to
  convey information about video frames that is critical for error
  recovery and packet forwarding in RTP middleboxes or network nodes.
  It is most useful when media is encrypted, and essential when the
  middlebox or node has no access to the media decryption keys.  It is
  also useful for codec-agnostic processing of encrypted or unencrypted
  media, while it also supports extensions for codec-specific
  information.




The file can be obtained via
https://datatracker.ietf.org/doc/draft-ietf-avtext-framemarking/


The following IPR Declarations may be related to this I-D:

  https://datatracker.ietf.org/ipr/5248/
  https://datatracker.ietf.org/ipr/5249/
  https://datatracker.ietf.org/ipr/3154/
  https://datatracker.ietf.org/ipr/2876/
  https://datatracker.ietf.org/ipr/5245/
  https://datatracker.ietf.org/ipr/5246/
  https://datatracker.ietf.org/ipr/5247/

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up. Changes are expected over time.

This version is dated 1 November 2019.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Experimental.

The draft was originally developed with the intention of bringing it
to Proposed Standard, but interest waned and the WG was uncertain that
it met all the use-cases for which it was developed.  However, the WG
thought that it was better to publish it as Experimental, to document
the work that had been done and to support any implementations that
had been made, rather than simply abandon the document (especially as
there are documents in the IETF editor's queue with reference
dependencies on it.)


(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:

  This document describes a Frame Marking RTP header extension used to
  convey information about video frames that is critical for error
  recovery and packet forwarding in RTP middleboxes or network nodes.
  It is most useful when media is encrypted, and essential when the
  middlebox or node has no access to the media decryption keys.  It is
  also useful for codec-agnostic processing of encrypted or unencrypted
  media, while it also supports extensions for codec-specific
  information.

Working Group Summary:

This document has been under development for quite some time, the
first version having been published in 2015; this is why its draft
name refers to the AVTExt working group, which was merged back into
AVTCore in 2017.

There have been multiple WGLCs carried out in the AVTCore working
group, which gave thorough review to the document.

Document Quality:

The specification was implemented for some time in libwebrtc and
deployed in Google Chrome's WebRTC stack; however, Chrome later
removed support for it, as its applicability seemed to be limited.

The document has been reviewed by WG members, and appears to be
complete insofar as it goes; however, there are concerns that it is
not sufficient for the uses cases for which it was intended.
Notably, the significantly more complex AV1 Dependency Descriptor
[https://aomediacodec.github.io/av1-rtp-spec/#appendix] addresses a
similar use-case much more thoroughly.

Who is the Document Shepherd? Who is the Responsible Area Director?

The document shepherd is Jonathan Lennox; the responsible area
director is Murray Kucherawy.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

The document shepherd reviewed the document, and feels it is ready for publication.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

None.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Yes.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

Two IPR declarations have been filed on the document; they were
mentioned to the working group, and no concerns were raised.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

Consensus involved most of the group's active participants.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

No nits.

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

N/A.

(13) Have all references within this document been identified as either normative or informative?

Yes.

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No.

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

No.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 8126).

The IANA registration of a header extension is correct.  No new registries are created.

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

N/A.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, YANG modules, etc.

N/A.

(20) If the document contains a YANG module, has the module been checked with any of the recommended validation tools (https://trac.ietf.org/trac/ops/wiki/yang-review-tools) for syntax and formatting validation? If there are any resulting errors or warnings, what is the justification for not fixing them at this time? Does the YANG module comply with the Network Management Datastore Architecture (NMDA) as specified in RFC8342?

N/A.

As required by RFC 4858, this is the current template for the Document
Shepherd Write-Up. Changes are expected over time.

This version is dated 1 November 2019.

(1) What type of RFC is being requested (BCP, Proposed Standard, Internet Standard, Informational, Experimental, or Historic)? Why is this the proper type of RFC? Is this type of RFC indicated in the title page header?

Experimental.

The draft was originally developed with the intention of bringing it
to Proposed Standard, but interest waned and the WG was uncertain that
it met all the use-cases for which it was developed.  However, the WG
thought that it was better to publish it as Experimental, to document
the work that had been done and to support any implementations that
had been made, rather than simply abandon the document (especially as
there are documents in the IETF editor's queue with reference
dependencies on it.)


(2) The IESG approval announcement includes a Document Announcement Write-Up. Please provide such a Document Announcement Write-Up. Recent examples can be found in the "Action" announcements for approved documents. The approval announcement contains the following sections:

Technical Summary:

  This document describes a Frame Marking RTP header extension used to
  convey information about video frames that is critical for error
  recovery and packet forwarding in RTP middleboxes or network nodes.
  It is most useful when media is encrypted, and essential when the
  middlebox or node has no access to the media decryption keys.  It is
  also useful for codec-agnostic processing of encrypted or unencrypted
  media, while it also supports extensions for codec-specific
  information.

Working Group Summary:

This document has been under development for quite some time, the
first version having been published in 2015; this is why its draft
name refers to the AVTExt working group, which was merged back into
AVTCore in 2017.

There have been multiple WGLCs carried out in the AVTCore working
group, which gave thorough review to the document.

Document Quality:

The specification was implemented for some time in libwebrtc and
deployed in Google Chrome's WebRTC stack; however, Chrome later
removed support for it, as its applicability seemed to be limited.

The document has been reviewed by WG members, and appears to be
complete insofar as it goes; however, there are concerns that it is
not sufficient for the uses cases for which it was intended.
Notably, the significantly more complex AV1 Dependency Descriptor
[https://aomediacodec.github.io/av1-rtp-spec/#appendix] addresses a
similar use-case much more thoroughly.

Who is the Document Shepherd? Who is the Responsible Area Director?

The document shepherd is Jonathan Lennox; the responsible area
director is Murray Kucherawy.

(3) Briefly describe the review of this document that was performed by the Document Shepherd. If this version of the document is not ready for publication, please explain why the document is being forwarded to the IESG.

The document shepherd reviewed the document, and feels it is ready for publication.

(4) Does the document Shepherd have any concerns about the depth or breadth of the reviews that have been performed?

No.

(5) Do portions of the document need review from a particular or from broader perspective, e.g., security, operational complexity, AAA, DNS, DHCP, XML, or internationalization? If so, describe the review that took place.

No.

(6) Describe any specific concerns or issues that the Document Shepherd has with this document that the Responsible Area Director and/or the IESG should be aware of? For example, perhaps he or she is uncomfortable with certain parts of the document, or has concerns whether there really is a need for it. In any event, if the WG has discussed those issues and has indicated that it still wishes to advance the document, detail those concerns here.

None.

(7) Has each author confirmed that any and all appropriate IPR disclosures required for full conformance with the provisions of BCP 78 and BCP 79 have already been filed. If not, explain why?

Yes.

(8) Has an IPR disclosure been filed that references this document? If so, summarize any WG discussion and conclusion regarding the IPR disclosures.

Two IPR declarations have been filed on the document; they were
mentioned to the working group, and no concerns were raised.

(9) How solid is the WG consensus behind this document? Does it represent the strong concurrence of a few individuals, with others being silent, or does the WG as a whole understand and agree with it?

Consensus involved most of the group's active participants.

(10) Has anyone threatened an appeal or otherwise indicated extreme discontent? If so, please summarise the areas of conflict in separate email messages to the Responsible Area Director. (It should be in a separate email because this questionnaire is publicly available.)

No.

(11) Identify any ID nits the Document Shepherd has found in this document. (See http://www.ietf.org/tools/idnits/ and the Internet-Drafts Checklist). Boilerplate checks are not enough; this check needs to be thorough.

No nits.

(12) Describe how the document meets any required formal review criteria, such as the MIB Doctor, YANG Doctor, media type, and URI type reviews.

N/A.

(13) Have all references within this document been identified as either normative or informative?

Yes.

(14) Are there normative references to documents that are not ready for advancement or are otherwise in an unclear state? If such normative references exist, what is the plan for their completion?

No.

(15) Are there downward normative references references (see RFC 3967)? If so, list these downward references to support the Area Director in the Last Call procedure.

No.

(16) Will publication of this document change the status of any existing RFCs? Are those RFCs listed on the title page header, listed in the abstract, and discussed in the introduction? If the RFCs are not listed in the Abstract and Introduction, explain why, and point to the part of the document where the relationship of this document to the other RFCs is discussed. If this information is not in the document, explain why the WG considers it unnecessary.

No.

(17) Describe the Document Shepherd's review of the IANA considerations section, especially with regard to its consistency with the body of the document. Confirm that all protocol extensions that the document makes are associated with the appropriate reservations in IANA registries. Confirm that any referenced IANA registries have been clearly identified. Confirm that newly created IANA registries include a detailed specification of the initial contents for the registry, that allocations procedures for future registrations are defined, and a reasonable name for the new registry has been suggested (see RFC 8126).

The IANA registration of a header extension is correct.  No new registries are created.

(18) List any new IANA registries that require Expert Review for future allocations. Provide any public guidance that the IESG would find useful in selecting the IANA Experts for these new registries.

N/A.

(19) Describe reviews and automated checks performed by the Document Shepherd to validate sections of the document written in a formal language, such as XML code, BNF rules, MIB definitions, YANG modules, etc.

N/A.

(20) If the document contains a YANG module, has the module been checked with any of the recommended validation tools (https://trac.ietf.org/trac/ops/wiki/yang-review-tools) for syntax and formatting validation? If there are any resulting errors or warnings, what is the justification for not fixing them at this time? Does the YANG module comply with the Network Management Datastore Architecture (NMDA) as specified in RFC8342?

N/A.