HESP - High Efficiency Streaming Protocol
draft-theo-hesp-00

Document Type Active Internet-Draft (individual)
Author Pieter-Jan Speelmans 
Last updated 2021-05-21
Stream (None)
Intended RFC status (None)
Formats pdf htmlized bibtex
Stream Stream state (No stream defined)
Consensus Boilerplate Unknown
RFC Editor Note (None)
IESG IESG state I-D Exists
Telechat date
Responsible AD (None)
Send notices to (None)
Individual                                             P. Speelmans, Ed.
Internet-Draft                                         THEO Technologies
Intended status: Informational                               21 May 2021
Expires: 22 November 2021

               HESP - High Efficiency Streaming Protocol
                           draft-theo-hesp-00

Abstract

   This document describes a protocol for delivering multimedia data,
   enabling ultra-low latency and fast channel change over HTTP
   networks.  It specifies the data format of the files and the actions
   to be taken by the server (sender) and the clients (receivers) of the
   streams.  It describes version 1 of this protocol.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on 22 November 2021.

Copyright Notice

   Copyright (c) 2021 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents (https://trustee.ietf.org/
   license-info) in effect on the date of publication of this document.
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.

   This document may not be modified, and derivative works of it may not
   be created, except to format it for publication as an RFC or to
   translate it into languages other than English.

Speelmans               Expires 22 November 2021                [Page 1]
Internet-Draft                    HESP                          May 2021

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Overview  . . . . . . . . . . . . . . . . . . . . . . . . . .   4
     2.1.  HESP components . . . . . . . . . . . . . . . . . . . . .   4
     2.2.  Two complementary streams . . . . . . . . . . . . . . . .   4
     2.3.  HESP object model . . . . . . . . . . . . . . . . . . . .   5
     2.4.  Reference flow  . . . . . . . . . . . . . . . . . . . . .   7
   3.  HESP Manifest . . . . . . . . . . . . . . . . . . . . . . . .   8
     3.1.  Timestamps  . . . . . . . . . . . . . . . . . . . . . . .   8
       3.1.1.  Manifest Timestamp and Media Timestamp  . . . . . . .   8
       3.1.2.  Sequence Numbers  . . . . . . . . . . . . . . . . . .   9
       3.1.3.  Calculating the Sequence Number of an Initialization
               Packet  . . . . . . . . . . . . . . . . . . . . . . .   9
     3.2.  Manifest data types . . . . . . . . . . . . . . . . . . .  10
       3.2.1.  Additional JSON data types  . . . . . . . . . . . . .  10
       3.2.2.  ManifestType  . . . . . . . . . . . . . . . . . . . .  11
       3.2.3.  TimeSource  . . . . . . . . . . . . . . . . . . . . .  14
       3.2.4.  ScaledValue . . . . . . . . . . . . . . . . . . . . .  14
       3.2.5.  PresentationType  . . . . . . . . . . . . . . . . . .  14
       3.2.6.  TimeBounds  . . . . . . . . . . . . . . . . . . . . .  18
       3.2.7.  AudioSwitchingSetType . . . . . . . . . . . . . . . .  18
       3.2.8.  SwitchingSetProtection  . . . . . . . . . . . . . . .  23
       3.2.9.  SwitchingSetProtectionSystem  . . . . . . . . . . . .  23
       3.2.10. VideoSwitchingSetType . . . . . . . . . . . . . . . .  24
       3.2.11. MetadataSwitchingSetType  . . . . . . . . . . . . . .  28
       3.2.12. AudioTrackType  . . . . . . . . . . . . . . . . . . .  31
       3.2.13. VideoTrackType  . . . . . . . . . . . . . . . . . . .  36
       3.2.14. Resolution  . . . . . . . . . . . . . . . . . . . . .  42
       3.2.15. MetadataTrackType . . . . . . . . . . . . . . . . . .  42
       3.2.16. SegmentType . . . . . . . . . . . . . . . . . . . . .  46
       3.2.17. PresentationEventType . . . . . . . . . . . . . . . .  46
       3.2.18. PresentationEventTimeBounds . . . . . . . . . . . . .  48
     3.3.  Manifest requests . . . . . . . . . . . . . . . . . . . .  49
       3.3.1.  Manifest responses  . . . . . . . . . . . . . . . . .  49
     3.4.  Addressing of content requests  . . . . . . . . . . . . .  50
       3.4.1.  Content request URL resolution  . . . . . . . . . . .  50
       3.4.2.  Requesting using an identifier and the content request
               URL . . . . . . . . . . . . . . . . . . . . . . . . .  51
     3.5.  Manifest example  . . . . . . . . . . . . . . . . . . . .  51
   4.  Initialization Stream . . . . . . . . . . . . . . . . . . . .  51
     4.1.  Initialization Stream purpose . . . . . . . . . . . . . .  51
     4.2.  Initialization Packet format  . . . . . . . . . . . . . .  52
       4.2.1.  Video Initialization Packet . . . . . . . . . . . . .  52
       4.2.2.  Audio Initialization Packet . . . . . . . . . . . . .  52
       4.2.3.  CMAF header . . . . . . . . . . . . . . . . . . . . .  52
       4.2.4.  Event message information . . . . . . . . . . . . . .  53
     4.3.  Initialization Stream addressing  . . . . . . . . . . . .  53

Speelmans               Expires 22 November 2021                [Page 2]
Internet-Draft                    HESP                          May 2021

       4.3.1.  Initialization Stream requests  . . . . . . . . . . .  53
       4.3.2.  Initialization Stream responses . . . . . . . . . . .  54
   5.  Continuation Stream . . . . . . . . . . . . . . . . . . . . .  55
     5.1.  Continuation Stream format  . . . . . . . . . . . . . . .  55
       5.1.1.  Media content . . . . . . . . . . . . . . . . . . . .  55
     5.2.  Continuation Segment availability . . . . . . . . . . . .  55
     5.3.  Continuation Stream addressing  . . . . . . . . . . . . .  56
       5.3.1.  Continuation Stream URLs  . . . . . . . . . . . . . .  56
       5.3.2.  Continuation Stream requests  . . . . . . . . . . . .  57
       5.3.3.  Continuation Stream responses . . . . . . . . . . . .  57
   6.  Timed metadata  . . . . . . . . . . . . . . . . . . . . . . .  60
     6.1.  Metadata Tracks . . . . . . . . . . . . . . . . . . . . .  60
     6.2.  Metadata events . . . . . . . . . . . . . . . . . . . . .  60
       6.2.1.  In-band events  . . . . . . . . . . . . . . . . . . .  60
       6.2.2.  Out-of-band events  . . . . . . . . . . . . . . . . .  63
   7.  Content protection  . . . . . . . . . . . . . . . . . . . . .  63
     7.1.  Common encryption support . . . . . . . . . . . . . . . .  63
     7.2.  HESP Manifest . . . . . . . . . . . . . . . . . . . . . .  63
     7.3.  CMAF box structure  . . . . . . . . . . . . . . . . . . .  64
       7.3.1.  Initialization Stream . . . . . . . . . . . . . . . .  64
       7.3.2.  Continuation Stream . . . . . . . . . . . . . . . . .  65
   8.  Contributors  . . . . . . . . . . . . . . . . . . . . . . . .  65
   9.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  65
   10. Security Considerations . . . . . . . . . . . . . . . . . . .  66
   11. Normative References  . . . . . . . . . . . . . . . . . . . .  66
   12. Informative References  . . . . . . . . . . . . . . . . . . .  68
   Appendix A.  Example usage  . . . . . . . . . . . . . . . . . . .  68
     A.1.  Manifest  . . . . . . . . . . . . . . . . . . . . . . . .  68
       A.1.1.  Retrieving the Manifest . . . . . . . . . . . . . . .  68
       A.1.2.  Timing information  . . . . . . . . . . . . . . . . .  73
       A.1.3.  Content addressing  . . . . . . . . . . . . . . . . .  73
     A.2.  Initialization Stream . . . . . . . . . . . . . . . . . .  75
       A.2.1.  Retrieving Initialization Packets . . . . . . . . . .  75
       A.2.2.  Parsing offset information  . . . . . . . . . . . . .  75
     A.3.  Continuation Stream . . . . . . . . . . . . . . . . . . .  75
       A.3.1.  Retrieving Continuation Segments  . . . . . . . . . .  76
   Appendix B.  CDNs . . . . . . . . . . . . . . . . . . . . . . . .  76
   Appendix C.  HESP Profiles (using H.264 as video codec) . . . . .  77
     C.1.  Maximal Gain Profile  . . . . . . . . . . . . . . . . . .  77
     C.2.  Compatibility Profile . . . . . . . . . . . . . . . . . .  78
       C.2.1.  Example . . . . . . . . . . . . . . . . . . . . . . .  79
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  80

1.  Introduction

   Viewers are more demanding than ever, calling for a streaming
   protocol that combines ultra-low latency, fast zapping and cost-
   effective scalability.

Speelmans               Expires 22 November 2021                [Page 3]
Internet-Draft                    HESP                          May 2021

   HESP is an HTTP based streaming approach that works with standard
   contribution feeds from (live) productions, standard encoders, albeit
   specifically configured, a HESP compliant packager, regular CDNs and
   an HESP compliant player.

   HESP offers sub second latency, near real-time interactivity, fast
   startup and channel change times and cost-effective scalability up to
   millions of viewers.

   The purpose of this document is to facilitate interoperability
   between HESP implementations by describing the media transmission
   protocol.

   This document describes version 1 of the protocol.

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in [RFC2119].

2.  Overview

   This section contains an overview of the HESP protocol and its
   building blocks.

2.1.  HESP components

   HESP follows a regular approach for online video streaming.  The
   content is ingested and transcoded in the different qualities.  Each
   quality requires two streams.  The encoded streams are packaged by an
   HESP packager and made available via an origin server.  An HESP
   player requests the stream using HTTP requests.  A CDN can be used.

      _________   ________   __________   ________   _____   ________
     |         | |        | |          | |        | |     | |        |
     |  Input  |\| xcoder |\| packager |\| origin |\| CDN |\|  HESP  |
     | Streams |/|        |/|          |/|        |/|     |/| player |
     |_________| |________| |__________| |________| |_____| |________|

                Figure 1: HESP chain from source to playback

2.2.  Two complementary streams

   HESP is based on using two streams for each track, the Initialization
   Stream and the Continuation Stream.  The encoder MUST ensure that the
   corresponding media data of both streams are issued with synchronized
   presentation timestamps.  The packager MUST ensure this data remains
   in sync.

Speelmans               Expires 22 November 2021                [Page 4]
Internet-Draft                    HESP                          May 2021

   The Initialization Stream consists of Initialization Packets.
   Initialization Packets MUST be individually addressable.  An
   Initialization Packet MUST contain an independent media sample and
   the reference to the segment and the position in the segment where
   the Continuation Stream can start.  Since the contained samples are
   independent, playback can start with any Initialization Packet.

   The Continuation Stream can start playback immediately after an
   Initialization Packet, allowing for very fast channel start and
   switch times.  This puts referencing limitations to the Continuation
   Stream.  When a reference is made from the Initialization Stream to a
   Continuation Stream, all data needed to render the sample with the
   subsequent presentation timestamp MUST be available as of the
   reference offset.  The samples in the Initialization Stream and the
   samples in the Continuation Stream MUST be aligned, that is, the
   corresponding media samples of both streams MUST have the same PTS
   and MUST be made available at the same time.  The Continuation Stream
   is addressed using byte range requests.  The Continuation Stream
   SHOULD be published in chunks in order to reduce end-to-end latency.

   The player receives the Initialization Packet, initializes the
   decoder, puts the media data in the decoder buffer, and requests the
   subsequent media data from the Continuation Stream (using HTTP Range
   requests, starting by the range offset given by the Initialization
   Packet.)

   time sequence              T1   T2   T3   T4   T5   T6   T7   T8
                            +----+----+----+----+----+----+----+----+
   initialization stream    | I1 | I2 | I3 | I4 | I5 | I6 | I7 | I8 |
                            +----+----+----+----+----+----+----+----+
                            +----+----+----+----+----+----+----+----+
   continuation stream      | C1 | C2 | C3 | C4 | C5 | C6 | C7 | C8 |
                            +----+----+----+----+----+----+----+----+
                                      +----+----+----+----+----+----+
   playback buffer                    | I3 | C4 | C5 | C6 | C7 | C8 |
                                      +----+----+----+----+----+----+

                     Figure 2: Start of an HESP stream

2.3.  HESP object model

   The object model follows the CMAF media object model [CMAF].

   A Track is used to contain media samples (audio or video) or
   metadata.  It consists of a Continuation Stream and (except for
   metadata Tracks) an Initialization Stream.

Speelmans               Expires 22 November 2021                [Page 5]
Internet-Draft                    HESP                          May 2021

   The Initialization Stream consists of Initialization Packets, where
   each such packet MUST contain a CMAF Header and possibly a CMAF
   Fragment.  Each Initialization Packet MUST be individually
   addressable through Sequence Numbers.  It is further explained in
   Section 4.

   The Continuation Stream consists of Continuation Segments.  These
   Segments MUST consist of CMAF Chunks that can be sent individually to
   clients.  It is further explained in Section 5.

   A Switching Set groups together Tracks that contain the same content,
   but with different encoding parameters (e.g. different resolution,
   bitrate,...) A client is able to seamlessly switch between Tracks of
   a Switching Set as a result.  If multiple Switching Sets contain
   differing content but are aligned in their timings (e.g. multiple
   view perspectives on the same performance or different languages of
   the same audio), we can consider them Aligned Switching Sets.

   A Selection Set groups together multiple Switching Sets of the same
   media type.  HESP currently allows 3 Selection Sets: audio, video and
   metadata Selection Sets.

   A Presentation contains the Selection Sets for a given period of
   time.  Multiple Presentations together form a continuous timeline of
   media content, even though each Presentation might have different
   content, encodings, timestamps,... It can be an advertisement, or a
   part of a show, or the first half of a game,... The Presentation is
   the lowest granularity inside a Manifest.  All Tracks of a
   Presentation MUST have media data for the full duration of the
   Presentation.

   The Manifest informs clients of the aforementioned data structure and
   must be retrieved before any media data.  The format of the Manifest
   is detailed in Section 3.

   A new Manifest is not needed to request a new Segment.  Segment
   addressing can happen automatically within a Presentation for an
   efficient and continuous delivery of the Continuation Stream.  A
   Manifest is only obligated to be updated before the start of a new
   Presentation.  This gives the opportunity for low frequency Manifest
   updates, though this update rate can be freely configured.

   Since Presentations can be of unknown duration, a different mechanism
   is used to signal a new Presentation to a client.  To that extent,
   in-band metadata events are introduced in the Continuation Stream.
   Such an event can signal the client to retrieve a new Manifest, when
   information about an upcoming Presentation becomes available.

Speelmans               Expires 22 November 2021                [Page 6]
Internet-Draft                    HESP                          May 2021

2.4.  Reference flow

            +---------+                                   +---------+
            | Player  |                                   | Origin  |
            +---------+                                   +---------+
                 |                                             |
    _____________|_____________________________________________|_____
   ! LOOP   / all presentations                                |     !
   !_______/     |                                             |     !
   !             |                                             |     !
   !             | request Manifest file                       |     !
   !             |-------------------------------------------->|     !
   !             |                                             |     !
   !             |                             Return Manifest |     !
   !             |<--------------------------------------------|     !
   !             |                                             |     !
   !             | parse Manifest                              |     !
   !             |---------------                              |     !
   !             |              |                              |     !
   !             |<--------------                              |     !
   !             |                                             |     !
   !             |                                             |     !
   !             | Request Initialization Packet               |     !
   !             |-------------------------------------------->|     !
   !             |                                             |     !
   !             |               Return Initialization Packet  |     !
   !             |<--------------------------------------------|     !
   !             |                                             |     !
   !             | Initialize decode pipeline                  |     !
   !             |---------------------------                  |     !
   !             |                          |                  |     !
   !             |<--------------------------                  |     !
   !             |                                             |     !
   !             | Parse position information                  |     !
   !             | of the corresponding data                   |     !
   !             | in the Continuation Stream                  |     !
   !             |--------------------------                   |     !
   !             |                          |                  |     !
   !             |<-------------------------                   |     !
   !             |                                             |     !
   !             |                                             |     !
   !             | Request Continuation Stream:                |     !
   !             | (Segment n) byte range [now, -)             |     !
   !             |-------------------------------------------->|     !
   !             |                                             |     !
   !             |             Return CMAF Fragments until the |     !
   !             |                          end of the Segment |     !
   !             |<--------------------------------------------|     !

Speelmans               Expires 22 November 2021                [Page 7]
Internet-Draft                    HESP                          May 2021

   !   __________|_____________________________________________|__   !
   !  ! LOOP   / all Segments of the Presentation              |  !  !
   !  !_______/  |                                             |  !  !
   !  !          |                                             |  !  !
   !  !          | Request Continuation Stream:                |  !  !
   !  !          | (Segment n+i) without byte range            |  !  !
   !  !          |-------------------------------------------->|  !  !
   !  !          |                                             |  !  !
   !  !          |             Return CMAF Fragments until the |  !  !
   !  !          |                          end of the Segment |  !  !
   !  !          |<--------------------------------------------|  !  !
   !  !          |                                             |  !  !
   !  !__________|_____________________________________________|__!  !
   !             |                                             |     !
   !_____________|_____________________________________________|_____!
                 |                                             !
                 |                                             !

                       Figure 3: HESP reference flow

   Though the manifest file is shared, video, audio and metadata
   (subtitles) media data MUST be distributed separately.  They each
   require separate requests from the client.

3.  HESP Manifest

   The first step for playback of an HESP stream is to fetch a Manifest.
   It contains information on the available Tracks and how to request
   content from every Track's Initialization and Continuation Streams.

3.1.  Timestamps

3.1.1.  Manifest Timestamp and Media Timestamp

   A distinction is made between Manifest Timestamps and Media
   Timestamps in the definitions below.  Media Timestamps MUST represent
   Presentation timestamps as they are given by the media data itself.
   Manifest Timestamps MUST define those same Media Timestamps, but with
   an additional offset such that the timestamps of all Tracks of a
   Presentation are aligned.  This offset MUST be given by the Manifest,
   for each Track or Switching Set.

   For example, consider the first Segment of a Track with Media
   Timestamps starting at 0 seconds.  This Track is part of the second
   Presentation of an HESP stream.  The first Presentation has run for a
   significant amount of time and the second Presentation follows
   immediately after the first Presentation ends.  In this case, the
   Manifest Timestamps of the second Presentation must succeed the

Speelmans               Expires 22 November 2021                [Page 8]
Internet-Draft                    HESP                          May 2021

   Manifest Timestamps of the first Presentation.  As a result, the
   Media Timestamps and Manifest Timestamps of the second Presentation
   now differ.  An offset must be given by the Manifest in order to
   inform clients about this difference between timestamp types for each
   Track of the second Presentation.

   An example of this distinction is given in Appendix A.1.2.

3.1.2.  Sequence Numbers

   Each Initialization Packet belonging to a Track is given a Sequence
   Number.  This is a simple identifier used to retrieve specific
   Initialization Packets by the client.  The Sequence Number is a
   positive integer that MUST be increased by 1 for each subsequent
   Initialization Packet of the same Track.

3.1.3.  Calculating the Sequence Number of an Initialization Packet

   To calculate the Sequence Number of a chosen Initialization Packet,
   the Manifest provides the client multiple values.  Each active
   Presentation MUST contain both its current Manifest Timestamp and for
   each Track, the Sequence Number of the latest Initialization Packet
   and the (constant) frame rate.

   It is given that:

   *  the frame rate MUST be constant for each Track.

   *  Initialization Packets MUST become available in real time.

   *  Sequence Numbers MUST be increased by 1 per Initialization Packet.

   *  the Sequence Number and Manifest Timestamp of the latest
      Initialization Packet for a Track are given by the Manifest.

   As a result, it is possible to derive Sequence Numbers for arbitrary
   Initialization Packets.  By taking the latest Sequence Number of a
   Track, its associated Manifest Timestamp and the frame rate, it is
   possible to calculate the number of frames between the given
   Initialization Packet and the required Initialization Packet in the
   past or future.  This amount can then be subtracted for that latest
   Sequence Number in order to derive the Sequence Number of the desired
   Initialization Packet.

Speelmans               Expires 22 November 2021                [Page 9]
Internet-Draft                    HESP                          May 2021

   For example, suppose the latest Sequence Number would be 103 at
   Manifest Timestamp 00:00:04.120, for a Track with a frame rate of 25
   fps.  To find the Sequence Number at Manifest Timestamp 00:00:01.360,
   the difference between both frames should be calculated.  This is
   2.760 seconds, or 2.760 * 25 = 69 frames.  This means that the
   Sequence Number for the provided Manifest Timestamp is 103 - 69 = 34.

3.2.  Manifest data types

   The Manifest MUST be formatted as a JSON file.  All data types used
   in the information below MUST satisfy the constraints made by the
   JavaScript Object Notation specification [RFC8259].  Some additional
   data types are introduced on top of the specification.

3.2.1.  Additional JSON data types

3.2.1.1.  Integer

   An "integer" is a subset of the "number" type defined in the JSON
   specification, limited to integer numbers.  Floating point numbers do
   belong to the "number" type in JSON, but are not accepted for the
   "integer" type.  For example, even though the floating point number
   "1.0" equals the integer number "1", it SHALL NOT be accepted.

3.2.1.2.  Enumeration

   An enumeration defines a list of constant strings, where a valid
   value of this type MUST equal one of these strings.  The type is
   written as "Enum(x, y, z)", where x, y and z are the constant strings
   that are allowed to be used.

3.2.1.3.  DateTime

   A DateTime MUST be formatted as a string in the format defined by ISO
   8601 [ISO8601]:

   YYYY-MM-DDThh:mm:ss.mmmTZD

   where YYYY = four-digit year
         MM = two-digit month
         DD = two-digit day of the month
         hh = two digits of the hour (00 through 23)
         mm = two digits of a minute (00 through 59)
         ss = two digits of a second (00 through 59)
         mmm = three digits of a millisecond (000 through 999)
         TZD = time zone designator (Z or +hh:mm or -hh:mm)

Speelmans               Expires 22 November 2021               [Page 10]
Internet-Draft                    HESP                          May 2021

3.2.2.  ManifestType

   This data type represents the root of the Manifest.  The structure
   consists of general playback information and a collection of
   Presentations.

   The table below gives the possible fields.  The "Required?" column
   indicates if a field REQUIRED or OPTIONAL.

   +======================+====================+======+================+
   | Attribute            | Type               | Req? | Description    |
   +======================+====================+======+================+
   | availabilityDuration | ScaledValue        | Y    | This value is  |
   |                      |                    |      | used to        |
   |                      |                    |      | define an      |
   |                      |                    |      | interval       |
   |                      |                    |      | [live - n,     |
   |                      |                    |      | live], where   |
   |                      |                    |      | live is the    |
   |                      |                    |      | current live   |
   |                      |                    |      | point and n    |
   |                      |                    |      | is the         |
   |                      |                    |      | availability   |
   |                      |                    |      | duration.      |
   |                      |                    |      | This is the    |
   |                      |                    |      | interval that  |
   |                      |                    |      | is guaranteed  |
   |                      |                    |      | to be          |
   |                      |                    |      | available for  |
   |                      |                    |      | players        |
   |                      |                    |      | during the     |
   |                      |                    |      | live stream.   |
   |                      |                    |      | For VOD        |
   |                      |                    |      | streams, this  |
   |                      |                    |      | value MUST be  |
   |                      |                    |      | ignored.       |
   +----------------------+--------------------+------+----------------+
   | creationDate         | DateTime           | Y    | The timestamp  |
   |                      |                    |      | of the moment  |
   |                      |                    |      | that this      |
   |                      |                    |      | specific       |
   |                      |                    |      | Manifest was   |
   |                      |                    |      | created by     |
   |                      |                    |      | the packager   |
   |                      |                    |      | (in local      |
   |                      |                    |      | packager       |
   |                      |                    |      | time.)         |
   +----------------------+--------------------+------+----------------+

Speelmans               Expires 22 November 2021               [Page 11]
Internet-Draft                    HESP                          May 2021

   | fallbackPollRate     | number             | Y    | This value     |
   |                      |                    |      | defines the    |
   |                      |                    |      | amount of      |
   |                      |                    |      | seconds a      |
   |                      |                    |      | player SHOULD  |
   |                      |                    |      | wait to poll   |
   |                      |                    |      | a new          |
   |                      |                    |      | Manifest, if   |
   |                      |                    |      | it hasn't      |
   |                      |                    |      | requested any  |
   |                      |                    |      | since          |
   |                      |                    |      | retrieving     |
   |                      |                    |      | the current    |
   |                      |                    |      | Manifest.      |
   +----------------------+--------------------+------+----------------+
   | manifestVersion      | Enum("1.0.0")      | Y    | The version    |
   |                      |                    |      | number of the  |
   |                      |                    |      | Manifest.      |
   +----------------------+--------------------+------+----------------+
   | presentations        | PresentationType[] | Y    | Contains a     |
   |                      |                    |      | list of all    |
   |                      |                    |      | Presentations  |
   |                      |                    |      | that are       |
   |                      |                    |      | currently      |
   |                      |                    |      | known.         |
   +----------------------+--------------------+------+----------------+
   | streamType           | Enum("live",       | Y    | This           |
   |                      | "vod")             |      | indicates      |
   |                      |                    |      | whether the    |
   |                      |                    |      | stream is a    |
   |                      |                    |      | live stream    |
   |                      |                    |      | ("live")       |
   |                      |                    |      | which does     |
   |                      |                    |      | not have a     |
   |                      |                    |      | known ending,  |
   |                      |                    |      | or a video on  |
   |                      |                    |      | demand stream  |
   |                      |                    |      | ("vod") with   |
   |                      |                    |      | a known        |
   |                      |                    |      | ending and     |
   |                      |                    |      | duration.      |
   +----------------------+--------------------+------+----------------+
   | activePresentation   | string             | N    | The            |
   |                      |                    |      | identifier of  |
   |                      |                    |      | the currently  |
   |                      |                    |      | active         |
   |                      |                    |      | Presentation.  |
   |                      |                    |      | This value     |

Speelmans               Expires 22 November 2021               [Page 12]
Internet-Draft                    HESP                          May 2021

   |                      |                    |      | MUST be        |
   |                      |                    |      | available if   |
   |                      |                    |      | "streamType"   |
   |                      |                    |      | equals         |
   |                      |                    |      | "live", but    |
   |                      |                    |      | MUST be        |
   |                      |                    |      | ignored if     |
   |                      |                    |      | "streamType"   |
   |                      |                    |      | equals "vod".  |
   +----------------------+--------------------+------+----------------+
   | contentBaseUrl       | string             | N    | This value     |
   |                      |                    |      | contains the   |
   |                      |                    |      | base URL for   |
   |                      |                    |      | all content    |
   |                      |                    |      | requests       |
   |                      |                    |      | relating to    |
   |                      |                    |      | this           |
   |                      |                    |      | Manifest.      |
   |                      |                    |      | See            |
   |                      |                    |      | Section 3.4    |
   |                      |                    |      | for more       |
   |                      |                    |      | information.   |
   +----------------------+--------------------+------+----------------+
   | timeSource           | TimeSource         | N    | Optional       |
   |                      |                    |      | value,         |
   |                      |                    |      | containing     |
   |                      |                    |      | the time       |
   |                      |                    |      | source of the  |
   |                      |                    |      | Manifest.  If  |
   |                      |                    |      | set, the       |
   |                      |                    |      | value          |
   |                      |                    |      | contains a     |
   |                      |                    |      | reference to   |
   |                      |                    |      | a time server  |
   |                      |                    |      | with which     |
   |                      |                    |      | the packager   |
   |                      |                    |      | is synced.     |
   |                      |                    |      | This value     |
   |                      |                    |      | MUST be        |
   |                      |                    |      | ignored if     |
   |                      |                    |      | "streamType"   |
   |                      |                    |      | equals "vod".  |
   +----------------------+--------------------+------+----------------+

                                  Table 1

Speelmans               Expires 22 November 2021               [Page 13]
Internet-Draft                    HESP                          May 2021

3.2.3.  TimeSource

   A TimeSource is used to sync the packager and player internal clocks,
   in order to only make requests for data once it is available.

    +===========+========+==========+================================+
    | Attribute | Type   | Required | Description                    |
    +===========+========+==========+================================+
    | scheme    | string | Y        | A UUID that defines the format |
    |           |        |          | of the time server response.   |
    +-----------+--------+----------+--------------------------------+
    | url       | string | Y        | The URL to be queried for a    |
    |           |        |          | time server response.          |
    +-----------+--------+----------+--------------------------------+

                                 Table 2

3.2.4.  ScaledValue

   In order to avoid rounding issues introduced through floating point
   numbers, this structure defines two integers.  The total value is
   calculated by dividing "value" over "scale".

      +===========+=========+==========+============================+
      | Attribute | Type    | Required | Description                |
      +===========+=========+==========+============================+
      | value     | integer | Y        | The defined integer value. |
      +-----------+---------+----------+----------------------------+
      | scale     | integer | N        | If not defined, the scale  |
      |           |         |          | SHALL equal 1.             |
      +-----------+---------+----------+----------------------------+

                                  Table 3

3.2.5.  PresentationType

   This is the type definition of a Presentation.

   +=============+============================+======+=================+
   | Attribute   | Type                       | Req? | Description     |
   +=============+============================+======+=================+
   | id          | string                     | Y    | The unique      |
   |             |                            |      | identifier for  |
   |             |                            |      | this            |
   |             |                            |      | Presentation.   |
   |             |                            |      | It MUST be      |
   |             |                            |      | unique over     |
   |             |                            |      | all             |

Speelmans               Expires 22 November 2021               [Page 14]
Internet-Draft                    HESP                          May 2021

   |             |                            |      | Presentations   |
   |             |                            |      | of this         |
   |             |                            |      | Manifest.  It   |
   |             |                            |      | MUST NOT        |
   |             |                            |      | change over     |
   |             |                            |      | Manifest        |
   |             |                            |      | updates.        |
   +-------------+----------------------------+------+-----------------+
   | timeBounds  | TimeBounds                 | Y    | The time        |
   |             |                            |      | boundaries of   |
   |             |                            |      | this            |
   |             |                            |      | Presentation,   |
   |             |                            |      | in manifest     |
   |             |                            |      | time.  The      |
   |             |                            |      | start time      |
   |             |                            |      | MUST be         |
   |             |                            |      | announced at    |
   |             |                            |      | least 2         |
   |             |                            |      | seconds before  |
   |             |                            |      | the             |
   |             |                            |      | Presentation    |
   |             |                            |      | is active.  It  |
   |             |                            |      | is possible     |
   |             |                            |      | that the end    |
   |             |                            |      | time is not     |
   |             |                            |      | yet available   |
   |             |                            |      | while the       |
   |             |                            |      | Presentation    |
   |             |                            |      | is active, but  |
   |             |                            |      | it MUST be      |
   |             |                            |      | available at    |
   |             |                            |      | least 2         |
   |             |                            |      | seconds before  |
   |             |                            |      | the actual end  |
   |             |                            |      | of the          |
   |             |                            |      | Presentation.   |
   +-------------+----------------------------+------+-----------------+
   | audio       | AudioSwitchingSetType[]    | N    | The audio       |
   |             |                            |      | Selection Set.  |
   |             |                            |      | It contains     |
   |             |                            |      | all audio       |
   |             |                            |      | Switching Sets  |
   |             |                            |      | of this         |
   |             |                            |      | Presentation.   |
   |             |                            |      | If not          |
   |             |                            |      | defined, this   |
   |             |                            |      | value SHALL     |
   |             |                            |      | equal an empty  |

Speelmans               Expires 22 November 2021               [Page 15]
Internet-Draft                    HESP                          May 2021

   |             |                            |      | list.           |
   +-------------+----------------------------+------+-----------------+
   | baseUrl     | string                     | N    | The base URL    |
   |             |                            |      | of this         |
   |             |                            |      | Presentation.   |
   |             |                            |      | It is part of   |
   |             |                            |      | the content     |
   |             |                            |      | base URLs for   |
   |             |                            |      | all Switching   |
   |             |                            |      | Sets belonging  |
   |             |                            |      | to this         |
   |             |                            |      | Presentation.   |
   |             |                            |      | See             |
   |             |                            |      | Section 3.4     |
   |             |                            |      | for more        |
   |             |                            |      | information.    |
   +-------------+----------------------------+------+-----------------+
   | currentTime | ScaledValue                | N    | The most        |
   |             |                            |      | recent          |
   |             |                            |      | composition     |
   |             |                            |      | timestamp of    |
   |             |                            |      | any Track       |
   |             |                            |      | contained by    |
   |             |                            |      | this            |
   |             |                            |      | Presentation    |
   |             |                            |      | at the time     |
   |             |                            |      | this Manifest   |
   |             |                            |      | is created.     |
   |             |                            |      | It should be    |
   |             |                            |      | specified in    |
   |             |                            |      | manifest time,  |
   |             |                            |      | not in media    |
   |             |                            |      | time (as that   |
   |             |                            |      | could have      |
   |             |                            |      | different       |
   |             |                            |      | offsets from    |
   |             |                            |      | Track to        |
   |             |                            |      | Track.)  If     |
   |             |                            |      | the             |
   |             |                            |      | Presentation    |
   |             |                            |      | is active, the  |
   |             |                            |      | current time    |
   |             |                            |      | MUST be         |
   |             |                            |      | defined.  If    |
   |             |                            |      | the             |
   |             |                            |      | Presentation    |
   |             |                            |      | is not active,  |
   |             |                            |      | this value      |

Speelmans               Expires 22 November 2021               [Page 16]
Internet-Draft                    HESP                          May 2021

   |             |                            |      | MUST be         |
   |             |                            |      | ignored.        |
   +-------------+----------------------------+------+-----------------+
   | events      | PresentationEventType[]    | N    | List of all     |
   |             |                            |      | currently       |
   |             |                            |      | available       |
   |             |                            |      | Presentation    |
   |             |                            |      | Events related  |
   |             |                            |      | to this         |
   |             |                            |      | Presentation.   |
   |             |                            |      | If not          |
   |             |                            |      | defined, this   |
   |             |                            |      | value SHALL     |
   |             |                            |      | equal an empty  |
   |             |                            |      | list.           |
   +-------------+----------------------------+------+-----------------+
   | metadata    | MetadataSwitchingSetType[] | N    | The metadata    |
   |             |                            |      | Selection Set.  |
   |             |                            |      | It contains     |
   |             |                            |      | all metadata    |
   |             |                            |      | Switching Sets  |
   |             |                            |      | of this         |
   |             |                            |      | Presentation.   |
   |             |                            |      | If not          |
   |             |                            |      | defined, this   |
   |             |                            |      | value SHALL     |
   |             |                            |      | equal an empty  |
   |             |                            |      | list.           |
   +-------------+----------------------------+------+-----------------+
   | video       | VideoSwitchingSetType[]    | N    | The video       |
   |             |                            |      | Selection Set.  |
   |             |                            |      | It contains     |
   |             |                            |      | all video       |
   |             |                            |      | Switching Sets  |
   |             |                            |      | of this         |
   |             |                            |      | Presentation.   |
   |             |                            |      | If not          |
   |             |                            |      | defined, this   |
   |             |                            |      | value SHALL     |
   |             |                            |      | equal an empty  |
   |             |                            |      | list.           |
   +-------------+----------------------------+------+-----------------+

                                  Table 4

Speelmans               Expires 22 November 2021               [Page 17]
Internet-Draft                    HESP                          May 2021

3.2.6.  TimeBounds

   A TimeBounds structure denotes a time interval with a start and end
   time.  It is possible that a start or end time are unknown.  The
   behavior in this case depends on the specific usage of the structure.

   These boundaries are inclusive at the start time and exclusive at the
   end time.  I.e. if two TimeBound need to be continuous, then the end
   time of the first one must equal the start time of the second.

      +===========+=========+==========+============================+
      | Attribute | Type    | Required | Description                |
      +===========+=========+==========+============================+
      | startTime | integer | N        | This value, when divided   |
      |           |         |          | by the timescale, denotes  |
      |           |         |          | the start time in seconds. |
      +-----------+---------+----------+----------------------------+
      | endTime   | integer | N        | This value, when divided   |
      |           |         |          | by the timescale, denotes  |
      |           |         |          | the end time in seconds.   |
      +-----------+---------+----------+----------------------------+
      | scale     | integer | N        | If the timescale is not    |
      |           |         |          | defined, it SHALL equal 1. |
      +-----------+---------+----------+----------------------------+

                                  Table 5

3.2.7.  AudioSwitchingSetType

   This is the type definition of an audio Switching Set.

   +=======================+==================+======+=================+
   | Attribute             | Type             | Req? | Description     |
   +=======================+==================+======+=================+
   | id                    | string           | Y    | The unique      |
   |                       |                  |      | identifier for  |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set. It MUST    |
   |                       |                  |      | be unique       |
   |                       |                  |      | within the      |
   |                       |                  |      | Presentation.   |
   +-----------------------+------------------+------+-----------------+
   | language              | string           | Y    | The language    |
   |                       |                  |      | of all audio    |
   |                       |                  |      | Tracks of this  |
   |                       |                  |      | Switching Set.  |
   |                       |                  |      | It MUST be      |
   |                       |                  |      | specified here  |

Speelmans               Expires 22 November 2021               [Page 18]
Internet-Draft                    HESP                          May 2021

   |                       |                  |      | by its ISO      |
   |                       |                  |      | 639-2           |
   |                       |                  |      | [ISO6392]       |
   |                       |                  |      | code.           |
   +-----------------------+------------------+------+-----------------+
   | tracks                | AudioTrackType[] | Y    | The collection  |
   |                       |                  |      | of all Tracks   |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this audio      |
   |                       |                  |      | Switching Set.  |
   +-----------------------+------------------+------+-----------------+
   | alignId               | string           | N    | A unique        |
   |                       |                  |      | identifier      |
   |                       |                  |      | that is set by  |
   |                       |                  |      | all Switching   |
   |                       |                  |      | Sets that are   |
   |                       |                  |      | Aligned         |
   |                       |                  |      | Switching Sets  |
   |                       |                  |      | with each       |
   |                       |                  |      | other.          |
   +-----------------------+------------------+------+-----------------+
   | baseUrl               | string           | N    | The base URL    |
   |                       |                  |      | of this         |
   |                       |                  |      | Switching Set.  |
   |                       |                  |      | It is part of   |
   |                       |                  |      | the content     |
   |                       |                  |      | base URLs for   |
   |                       |                  |      | all Tracks      |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this            |
   |                       |                  |      | Presentation.   |
   |                       |                  |      | See             |
   |                       |                  |      | Section 3.4     |
   |                       |                  |      | for more        |
   |                       |                  |      | information.    |
   +-----------------------+------------------+------+-----------------+
   | channels              | integer          | N    | The audio       |
   |                       |                  |      | channel         |
   |                       |                  |      | configuration   |
   |                       |                  |      | of all audio    |
   |                       |                  |      | Tracks          |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set. It is      |
   |                       |                  |      | defined as the  |
   |                       |                  |      | total number    |
   |                       |                  |      | of audio        |
   |                       |                  |      | channels,       |

Speelmans               Expires 22 November 2021               [Page 19]
Internet-Draft                    HESP                          May 2021

   |                       |                  |      | equal for each  |
   |                       |                  |      | Track.  This    |
   |                       |                  |      | is used for     |
   |                       |                  |      | ABR selection   |
   |                       |                  |      | by the player.  |
   +-----------------------+------------------+------+-----------------+
   | codecs                | string           | N    | The definition  |
   |                       |                  |      | of the sample   |
   |                       |                  |      | type used for   |
   |                       |                  |      | all Tracks      |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set. It MUST    |
   |                       |                  |      | follow the ISO  |
   |                       |                  |      | File Format     |
   |                       |                  |      | Name Space as   |
   |                       |                  |      | defined by      |
   |                       |                  |      | [RFC6381].  If  |
   |                       |                  |      | it is not       |
   |                       |                  |      | defined by      |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set, then it    |
   |                       |                  |      | must be         |
   |                       |                  |      | defined on all  |
   |                       |                  |      | its Tracks      |
   |                       |                  |      | separately.     |
   +-----------------------+------------------+------+-----------------+
   | continuationPattern   | string           | N    | The URL         |
   |                       |                  |      | pattern used    |
   |                       |                  |      | to request      |
   |                       |                  |      | Continuation    |
   |                       |                  |      | Segments        |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Track.     |
   |                       |                  |      | The pattern     |
   |                       |                  |      | MUST include    |
   |                       |                  |      | the             |
   |                       |                  |      | "{segmentId}"   |
   |                       |                  |      | string, see     |
   |                       |                  |      | Section 3.4     |
   |                       |                  |      | for more        |
   |                       |                  |      | information.    |
   |                       |                  |      | If this is not  |
   |                       |                  |      | defined, then   |
   |                       |                  |      | it MUST be      |
   |                       |                  |      | defined on all  |
   |                       |                  |      | Tracks of this  |
   |                       |                  |      | Switching Set   |

Speelmans               Expires 22 November 2021               [Page 20]
Internet-Draft                    HESP                          May 2021

   |                       |                  |      | separately.     |
   +-----------------------+------------------+------+-----------------+
   | frameRate             | integer          | N    | The frame rate  |
   |                       |                  |      | of all audio    |
   |                       |                  |      | Tracks.  It     |
   |                       |                  |      | defines the     |
   |                       |                  |      | amount of       |
   |                       |                  |      | audio samples   |
   |                       |                  |      | in one frame.   |
   |                       |                  |      | If neither the  |
   |                       |                  |      | Track nor the   |
   |                       |                  |      | Switching Set   |
   |                       |                  |      | has this value  |
   |                       |                  |      | set, it SHALL   |
   |                       |                  |      | equal 1024.     |
   +-----------------------+------------------+------+-----------------+
   | initializationPattern | string           | N    | The URL         |
   |                       |                  |      | pattern used    |
   |                       |                  |      | to request      |
   |                       |                  |      | Initialization  |
   |                       |                  |      | Packets         |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Track.     |
   |                       |                  |      | The pattern     |
   |                       |                  |      | MUST include    |
   |                       |                  |      | the "{initId}"  |
   |                       |                  |      | string, see     |
   |                       |                  |      | Section 3.4     |
   |                       |                  |      | for more        |
   |                       |                  |      | information.    |
   |                       |                  |      | If this is not  |
   |                       |                  |      | defined, then   |
   |                       |                  |      | it MUST be      |
   |                       |                  |      | defined on all  |
   |                       |                  |      | Tracks of this  |
   |                       |                  |      | Switching Set   |
   |                       |                  |      | separately.     |
   +-----------------------+------------------+------+-----------------+
   | label                 | string           | N    | A human         |
   |                       |                  |      | readable name   |
   |                       |                  |      | for this        |
   |                       |                  |      | Switching Set.  |
   +-----------------------+------------------+------+-----------------+
   | mediaTimeOffset       | ScaledValue      | N    | The offset to   |
   |                       |                  |      | be added to     |
   |                       |                  |      | Manifest        |
   |                       |                  |      | Timestamps in   |
   |                       |                  |      | order to        |

Speelmans               Expires 22 November 2021               [Page 21]
Internet-Draft                    HESP                          May 2021

   |                       |                  |      | calculate       |
   |                       |                  |      | Media           |
   |                       |                  |      | Timestamps      |
   |                       |                  |      | contained by    |
   |                       |                  |      | Segments of     |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set. If         |
   |                       |                  |      | neither the     |
   |                       |                  |      | Track or the    |
   |                       |                  |      | Switching Set   |
   |                       |                  |      | has this value  |
   |                       |                  |      | set, it SHALL   |
   |                       |                  |      | equal 0.        |
   +-----------------------+------------------+------+-----------------+
   | mimeType              | string           | N    | The MIME type   |
   |                       |                  |      | of all content  |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set. If not     |
   |                       |                  |      | set, the MIME   |
   |                       |                  |      | type SHALL      |
   |                       |                  |      | equal "audio/   |
   |                       |                  |      | mp4".           |
   +-----------------------+------------------+------+-----------------+
   | protection            | SwitchingSet     | N    | The             |
   |                       | Protection       |      | information     |
   |                       |                  |      | related to      |
   |                       |                  |      | content         |
   |                       |                  |      | protection for  |
   |                       |                  |      | all audio       |
   |                       |                  |      | Tracks          |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set.            |
   +-----------------------+------------------+------+-----------------+
   | sampleRate            | integer          | N    | The sample      |
   |                       |                  |      | rate (in Hz)    |
   |                       |                  |      | of all audio    |
   |                       |                  |      | Tracks          |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set. If this    |
   |                       |                  |      | is not          |
   |                       |                  |      | defined, then   |
   |                       |                  |      | every Track     |
   |                       |                  |      | MUST set its    |
   |                       |                  |      | sample rate     |
   |                       |                  |      | separately.     |

Speelmans               Expires 22 November 2021               [Page 22]
Internet-Draft                    HESP                          May 2021

   +-----------------------+------------------+------+-----------------+

                                  Table 6

3.2.8.  SwitchingSetProtection

   The following fields are defined on the SwitchingSetProtection
   structure:

   +===========+================================+======+==============+
   | Attribute | Type                           | Req? | Description  |
   +===========+================================+======+==============+
   | type      | Enum("cenc", "cbcs")           | Y    | The          |
   |           |                                |      | protection   |
   |           |                                |      | scheme used  |
   |           |                                |      | to encrypt   |
   |           |                                |      | this         |
   |           |                                |      | Switching    |
   |           |                                |      | Set.         |
   +-----------+--------------------------------+------+--------------+
   | systems   | SwitchingSetProtectionSystem[] | Y    | Metadata     |
   |           |                                |      | about the    |
   |           |                                |      | DRM systems  |
   |           |                                |      | that can be  |
   |           |                                |      | used to play |
   |           |                                |      | back this    |
   |           |                                |      | Switching    |
   |           |                                |      | Set. This    |
   |           |                                |      | list MUST    |
   |           |                                |      | contain at   |
   |           |                                |      | least one    |
   |           |                                |      | entry.       |
   +-----------+--------------------------------+------+--------------+

                                 Table 7

   More information on content protection can be found in Section 7.

3.2.9.  SwitchingSetProtectionSystem

   The following fields are defined on the SwitchingSetProtectionSystem
   structure:

Speelmans               Expires 22 November 2021               [Page 23]
Internet-Draft                    HESP                          May 2021

   +===========+======+=========+======================================+
   | Attribute |Type  |Required |Description                           |
   +===========+======+=========+======================================+
   | pssh      |string|N        |A Base 64 encoded                     |
   |           |      |         |"ProtectionSystemSpecificHeaderBox".  |
   |           |      |         |If it is not defined in the Manifest, |
   |           |      |         |then such a box MUST be available in  |
   |           |      |         |the Initialization Stream of each     |
   |           |      |         |Track belonging to this Switching Set.|
   +-----------+------+---------+--------------------------------------+
   | schemeId  |string|Y        |A UUID code that identifies the       |
   |           |      |         |protection system used to protect this|
   |           |      |         |Switching Set.                        |
   +-----------+------+---------+--------------------------------------+

                                  Table 8

   Additional attributes MAY be defined by specific protection systems
   such as license acquisition URLs, authorization related URLs,
   specific initialization data (a default key ID),... These attributes
   SHOULD be defined on this SwitchingSetProtectionSystem structure,
   however they will not be defined here.  Depending on the schemeId
   that is given here, a client SHOULD verify that these additional
   attributes exist.

3.2.10.  VideoSwitchingSetType

   This is the type definition of a video Switching Set.

   +=======================+==================+======+=================+
   | Attribute             | Type             | Req? | Description     |
   +=======================+==================+======+=================+
   | id                    | string           | Y    | The unique      |
   |                       |                  |      | identifier for  |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set. It MUST    |
   |                       |                  |      | be unique       |
   |                       |                  |      | within the      |
   |                       |                  |      | Presentation.   |
   +-----------------------+------------------+------+-----------------+
   | tracks                | VideoTrackType[] | Y    | The collection  |
   |                       |                  |      | of all Tracks   |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this video      |
   |                       |                  |      | Switching Set.  |
   +-----------------------+------------------+------+-----------------+
   | alignId               | string           | N    | A unique        |
   |                       |                  |      | identifier      |

Speelmans               Expires 22 November 2021               [Page 24]
Internet-Draft                    HESP                          May 2021

   |                       |                  |      | that is set by  |
   |                       |                  |      | all Switching   |
   |                       |                  |      | Sets that are   |
   |                       |                  |      | Aligned         |
   |                       |                  |      | Switching Sets  |
   |                       |                  |      | with each       |
   |                       |                  |      | other.          |
   +-----------------------+------------------+------+-----------------+
   | baseUrl               | string           | N    | The base URL    |
   |                       |                  |      | of this         |
   |                       |                  |      | Switching Set.  |
   |                       |                  |      | It is part of   |
   |                       |                  |      | the content     |
   |                       |                  |      | base URLs for   |
   |                       |                  |      | all Tracks      |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this            |
   |                       |                  |      | Presentation.   |
   |                       |                  |      | See             |
   |                       |                  |      | Section 3.4     |
   |                       |                  |      | for more        |
   |                       |                  |      | information.    |
   +-----------------------+------------------+------+-----------------+
   | codecs                | string           | N    | The definition  |
   |                       |                  |      | of the sample   |
   |                       |                  |      | type used for   |
   |                       |                  |      | all Tracks      |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set. It needs   |
   |                       |                  |      | to follow the   |
   |                       |                  |      | ISO File        |
   |                       |                  |      | Format Name     |
   |                       |                  |      | Space as        |
   |                       |                  |      | defined by      |
   |                       |                  |      | [RFC6381].  If  |
   |                       |                  |      | it is not       |
   |                       |                  |      | defined by      |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set, then it    |
   |                       |                  |      | must be         |
   |                       |                  |      | defined on all  |
   |                       |                  |      | its Tracks      |
   |                       |                  |      | separately.     |
   +-----------------------+------------------+------+-----------------+
   | continuationPattern   | string           | N    | The URL         |
   |                       |                  |      | pattern used    |
   |                       |                  |      | to request      |

Speelmans               Expires 22 November 2021               [Page 25]
Internet-Draft                    HESP                          May 2021

   |                       |                  |      | Continuation    |
   |                       |                  |      | Segments        |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Track.     |
   |                       |                  |      | The pattern     |
   |                       |                  |      | MUST include    |
   |                       |                  |      | the             |
   |                       |                  |      | "{segmentId}"   |
   |                       |                  |      | string, see     |
   |                       |                  |      | Section 3.4     |
   |                       |                  |      | for more        |
   |                       |                  |      | information.    |
   |                       |                  |      | If this is not  |
   |                       |                  |      | defined, then   |
   |                       |                  |      | it MUST be      |
   |                       |                  |      | defined on all  |
   |                       |                  |      | Tracks of this  |
   |                       |                  |      | Switching Set   |
   |                       |                  |      | separately.     |
   +-----------------------+------------------+------+-----------------+
   | frameRate             | ScaledValue      | N    | The frame rate  |
   |                       |                  |      | of all video    |
   |                       |                  |      | Tracks.  If     |
   |                       |                  |      | this is not     |
   |                       |                  |      | defined, then   |
   |                       |                  |      | every Track     |
   |                       |                  |      | MUST set its    |
   |                       |                  |      | frame rate      |
   |                       |                  |      | separately.     |
   +-----------------------+------------------+------+-----------------+
   | initializationPattern | string           | N    | The URL         |
   |                       |                  |      | pattern used    |
   |                       |                  |      | to request      |
   |                       |                  |      | Initialization  |
   |                       |                  |      | Packets         |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Track.     |
   |                       |                  |      | The pattern     |
   |                       |                  |      | MUST include    |
   |                       |                  |      | the "{initId}"  |
   |                       |                  |      | string, see     |
   |                       |                  |      | Section 3.4     |
   |                       |                  |      | for more        |
   |                       |                  |      | information.    |
   |                       |                  |      | If this is not  |
   |                       |                  |      | defined, then   |
   |                       |                  |      | it MUST be      |
   |                       |                  |      | defined on all  |

Speelmans               Expires 22 November 2021               [Page 26]
Internet-Draft                    HESP                          May 2021

   |                       |                  |      | Tracks of this  |
   |                       |                  |      | Switching Set   |
   |                       |                  |      | separately.     |
   +-----------------------+------------------+------+-----------------+
   | label                 | string           | N    | An optional     |
   |                       |                  |      | human readable  |
   |                       |                  |      | name for this   |
   |                       |                  |      | Switching Set.  |
   +-----------------------+------------------+------+-----------------+
   | mediaTimeOffset       | ScaledValue      | N    | The offset to   |
   |                       |                  |      | be added to     |
   |                       |                  |      | Manifest        |
   |                       |                  |      | Timestamps in   |
   |                       |                  |      | order to        |
   |                       |                  |      | calculate       |
   |                       |                  |      | Media           |
   |                       |                  |      | Timestamps      |
   |                       |                  |      | contained by    |
   |                       |                  |      | Segments of     |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set. If         |
   |                       |                  |      | neither the     |
   |                       |                  |      | Track or the    |
   |                       |                  |      | Switching Set   |
   |                       |                  |      | has this value  |
   |                       |                  |      | set, it SHALL   |
   |                       |                  |      | equal 0.        |
   +-----------------------+------------------+------+-----------------+
   | mimeType              | string           | N    | The MIME type   |
   |                       |                  |      | of all content  |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set. If not     |
   |                       |                  |      | set, the MIME   |
   |                       |                  |      | type SHALL      |
   |                       |                  |      | equal "video/   |
   |                       |                  |      | mp4".           |
   +-----------------------+------------------+------+-----------------+
   | protection            | SwitchingSet     | N    | The             |
   |                       | Protection       |      | information     |
   |                       |                  |      | related to      |
   |                       |                  |      | content         |
   |                       |                  |      | protection for  |
   |                       |                  |      | all Tracks      |
   |                       |                  |      | belonging to    |
   |                       |                  |      | this Switching  |
   |                       |                  |      | Set.            |
   +-----------------------+------------------+------+-----------------+

Speelmans               Expires 22 November 2021               [Page 27]
Internet-Draft                    HESP                          May 2021

                                  Table 9

3.2.11.  MetadataSwitchingSetType

   This is the type definition of a metadata Switching Set.

   +=====================+=====================+======+================+
   | Attribute           | Type                | Req? | Description    |
   +=====================+=====================+======+================+
   | id                  | string              | Y    | The unique     |
   |                     |                     |      | identifier     |
   |                     |                     |      | for this       |
   |                     |                     |      | Switching      |
   |                     |                     |      | Set. It MUST   |
   |                     |                     |      | be unique      |
   |                     |                     |      | within the     |
   |                     |                     |      | Presentation.  |
   +---------------------+---------------------+------+----------------+
   | mimeType            | string              | Y    | The MIME type  |
   |                     |                     |      | of all         |
   |                     |                     |      | content        |
   |                     |                     |      | belonging to   |
   |                     |                     |      | this           |
   |                     |                     |      | Switching      |
   |                     |                     |      | Set.           |
   +---------------------+---------------------+------+----------------+
   | tracks              | MetadataTrackType[] | Y    | The            |
   |                     |                     |      | collection of  |
   |                     |                     |      | all Tracks     |
   |                     |                     |      | belonging to   |
   |                     |                     |      | this metadata  |
   |                     |                     |      | Switching      |
   |                     |                     |      | Set.           |
   +---------------------+---------------------+------+----------------+
   | schemeId            | string              | Y    | An identifier  |
   |                     |                     |      | that denotes   |
   |                     |                     |      | the type of    |
   |                     |                     |      | metadata       |
   |                     |                     |      | contained by   |
   |                     |                     |      | this metadata  |
   |                     |                     |      | Switching      |
   |                     |                     |      | Set. Client    |
   |                     |                     |      | behavior may   |
   |                     |                     |      | differ         |
   |                     |                     |      | depending on   |
   |                     |                     |      | the vendor     |
   |                     |                     |      | specific       |
   |                     |                     |      | identifier     |

Speelmans               Expires 22 November 2021               [Page 28]
Internet-Draft                    HESP                          May 2021

   |                     |                     |      | set here.      |
   +---------------------+---------------------+------+----------------+
   | alignId             | string              | N    | A unique       |
   |                     |                     |      | identifier     |
   |                     |                     |      | that is set    |
   |                     |                     |      | by all         |
   |                     |                     |      | Switching      |
   |                     |                     |      | Sets that are  |
   |                     |                     |      | Aligned        |
   |                     |                     |      | Switching      |
   |                     |                     |      | Sets with      |
   |                     |                     |      | each other.    |
   +---------------------+---------------------+------+----------------+
   | baseUrl             | string              | N    | The base URL   |
   |                     |                     |      | of this        |
   |                     |                     |      | Switching      |
   |                     |                     |      | Set. It is     |
   |                     |                     |      | part of the    |
   |                     |                     |      | content base   |
   |                     |                     |      | URLs for all   |
   |                     |                     |      | Tracks         |
   |                     |                     |      | belonging to   |
   |                     |                     |      | this           |
   |                     |                     |      | Presentation.  |
   |                     |                     |      | See            |
   |                     |                     |      | Section 3.4    |
   |                     |                     |      | for more       |
   |                     |                     |      | information.   |
   +---------------------+---------------------+------+----------------+
   | continuationPattern | string              | N    | The URL        |
   |                     |                     |      | pattern used   |
   |                     |                     |      | to request     |
   |                     |                     |      | continuation   |
   |                     |                     |      | segments       |
   |                     |                     |      | belonging to   |
   |                     |                     |      | this media     |
   |                     |                     |      | Track.  The    |
   |                     |                     |      | pattern needs  |
   |                     |                     |      | to include     |
   |                     |                     |      | the            |
   |                     |                     |      | "{segmentId}"  |
   |                     |                     |      | string, see    |
   |                     |                     |      | Section 3.4    |
   |                     |                     |      | for more       |
   |                     |                     |      | information.   |
   |                     |                     |      | If it is not   |
   |                     |                     |      | defined by     |
   |                     |                     |      | this           |

Speelmans               Expires 22 November 2021               [Page 29]
Internet-Draft                    HESP                          May 2021

   |                     |                     |      | Switching      |
   |                     |                     |      | Set, then it   |
   |                     |                     |      | must be        |
   |                     |                     |      | defined on     |
   |                     |                     |      | all its        |
   |                     |                     |      | Tracks         |
   |                     |                     |      | separately.    |
   +---------------------+---------------------+------+----------------+
   | label               | string              | N    | An optional    |
   |                     |                     |      | human          |
   |                     |                     |      | readable name  |
   |                     |                     |      | for this       |
   |                     |                     |      | Switching      |
   |                     |                     |      | Set.           |
   +---------------------+---------------------+------+----------------+
   | language            | string              | N    | The language   |
   |                     |                     |      | of all Tracks  |
   |                     |                     |      | of the         |
   |                     |                     |      | metadata       |
   |                     |                     |      | Switching      |
   |                     |                     |      | Set. It MUST   |
   |                     |                     |      | be specified   |
   |                     |                     |      | here by its    |
   |                     |                     |      | ISO 639-2      |
   |                     |                     |      | [ISO6392]      |
   |                     |                     |      | code.          |
   +---------------------+---------------------+------+----------------+
   | mediaTimeOffset     | ScaledValue         | N    | The offset to  |
   |                     |                     |      | be added to    |
   |                     |                     |      | Manifest       |
   |                     |                     |      | Timestamps in  |
   |                     |                     |      | order to       |
   |                     |                     |      | calculate      |
   |                     |                     |      | Media          |
   |                     |                     |      | Timestamps     |
   |                     |                     |      | contained by   |
   |                     |                     |      | Segments of    |
   |                     |                     |      | this           |
   |                     |                     |      | Switching      |
   |                     |                     |      | Set. If        |
   |                     |                     |      | neither the    |
   |                     |                     |      | Track or the   |
   |                     |                     |      | Switching Set  |
   |                     |                     |      | has this       |
   |                     |                     |      | value set, it  |
   |                     |                     |      | SHALL equal    |
   |                     |                     |      | 0.             |
   +---------------------+---------------------+------+----------------+

Speelmans               Expires 22 November 2021               [Page 30]
Internet-Draft                    HESP                          May 2021

                                  Table 10

3.2.12.  AudioTrackType

   This is the type definition of an audio Track.

   +=======================+===============+======+====================+
   | Attribute             | Type          | Req? | Description        |
   +=======================+===============+======+====================+
   | bandwidth             | number        | Y    | The peak bitrate   |
   |                       |               |      | of this Track.     |
   |                       |               |      | It is denoted in   |
   |                       |               |      | bits per second.   |
   |                       |               |      | The measured       |
   |                       |               |      | bitrates of all    |
   |                       |               |      | Segments of the    |
   |                       |               |      | Track MUST NOT     |
   |                       |               |      | exceed this        |
   |                       |               |      | value.  This       |
   |                       |               |      | value is used to   |
   |                       |               |      | aid in ABR         |
   |                       |               |      | decisions by the   |
   |                       |               |      | player.            |
   +-----------------------+---------------+------+--------------------+
   | id                    | string        | Y    | The unique         |
   |                       |               |      | identifier for     |
   |                       |               |      | this Track.  It    |
   |                       |               |      | MUST be unique     |
   |                       |               |      | within its         |
   |                       |               |      | Switching Set.     |
   +-----------------------+---------------+------+--------------------+
   | segments              | SegmentType[] | Y    | The metadata of    |
   |                       |               |      | the Segments that  |
   |                       |               |      | are contained by   |
   |                       |               |      | this Track at the  |
   |                       |               |      | moment of          |
   |                       |               |      | Manifest           |
   |                       |               |      | creation.          |
   |                       |               |      | If a segment       |
   |                       |               |      | duration is set    |
   |                       |               |      | in the Track of    |
   |                       |               |      | this segment,      |
   |                       |               |      | then only the      |
   |                       |               |      | active segment     |
   |                       |               |      | MUST be shown      |
   |                       |               |      | here.  If a        |
   |                       |               |      | segment duration   |
   |                       |               |      | is not set, then   |

Speelmans               Expires 22 November 2021               [Page 31]
Internet-Draft                    HESP                          May 2021

   |                       |               |      | each segment MUST  |
   |                       |               |      | be announced by    |
   |                       |               |      | the Manifest       |
   |                       |               |      | before it is       |
   |                       |               |      | active.  More      |
   |                       |               |      | information about  |
   |                       |               |      | Segment            |
   |                       |               |      | availability is    |
   |                       |               |      | given in           |
   |                       |               |      | Section 5.2.       |
   +-----------------------+---------------+------+--------------------+
   | activeSegment         | integer       | N    | The identifier of  |
   |                       |               |      | the segment that   |
   |                       |               |      | is currently       |
   |                       |               |      | active.            |
   |                       |               |      | If this Track's    |
   |                       |               |      | Presentation was   |
   |                       |               |      | previously active  |
   |                       |               |      | or is currently    |
   |                       |               |      | active, then the   |
   |                       |               |      | most recent        |
   |                       |               |      | Continuation       |
   |                       |               |      | Segment's          |
   |                       |               |      | identifier MUST    |
   |                       |               |      | be shown here.     |
   |                       |               |      | If the Track's     |
   |                       |               |      | Presentation has   |
   |                       |               |      | not yet been       |
   |                       |               |      | active (i.e. no    |
   |                       |               |      | Continuation       |
   |                       |               |      | Segments are yet   |
   |                       |               |      | available), then   |
   |                       |               |      | this value MUST    |
   |                       |               |      | be ignored by the  |
   |                       |               |      | client.            |
   +-----------------------+---------------+------+--------------------+
   | activeSequenceNumber  | integer       | N    | The active         |
   |                       |               |      | Sequence Number    |
   |                       |               |      | of this Track.     |
   |                       |               |      | It MUST denote     |
   |                       |               |      | the Sequence       |
   |                       |               |      | Number of the      |
   |                       |               |      | Initialization     |
   |                       |               |      | Packet of this     |
   |                       |               |      | Track that was     |
   |                       |               |      | most recently      |
   |                       |               |      | published, at the  |
   |                       |               |      | time of creation   |

Speelmans               Expires 22 November 2021               [Page 32]
Internet-Draft                    HESP                          May 2021

   |                       |               |      | of this Manifest.  |
   |                       |               |      | If this Track's    |
   |                       |               |      | Presentation was   |
   |                       |               |      | previously active  |
   |                       |               |      | or is currently    |
   |                       |               |      | active, then the   |
   |                       |               |      | most recent        |
   |                       |               |      | Initialization     |
   |                       |               |      | Packet's Sequence  |
   |                       |               |      | Number MUST be     |
   |                       |               |      | shown here.  If    |
   |                       |               |      | the Track's        |
   |                       |               |      | Presentation has   |
   |                       |               |      | not yet been       |
   |                       |               |      | active (i.e. no    |
   |                       |               |      | Initialization     |
   |                       |               |      | Packets are yet    |
   |                       |               |      | available), then   |
   |                       |               |      | this value MUST    |
   |                       |               |      | be ignored by the  |
   |                       |               |      | client.            |
   +-----------------------+---------------+------+--------------------+
   | averageBandwidth      | number        | N    | The average        |
   |                       |               |      | bitrate of this    |
   |                       |               |      | Track.  It is      |
   |                       |               |      | denoted in bits    |
   |                       |               |      | per second.        |
   |                       |               |      | If given, it is    |
   |                       |               |      | expected that      |
   |                       |               |      | over a duration    |
   |                       |               |      | of 10 minutes,     |
   |                       |               |      | the average        |
   |                       |               |      | bitrate of this    |
   |                       |               |      | Track SHALL be     |
   |                       |               |      | within 5% of the   |
   |                       |               |      | given value.       |
   |                       |               |      | This value is      |
   |                       |               |      | used to aid in     |
   |                       |               |      | ABR decisions by   |
   |                       |               |      | the player.        |
   +-----------------------+---------------+------+--------------------+
   | baseUrl               | string        | N    | The base URL of    |
   |                       |               |      | this Track.  It    |
   |                       |               |      | is part of the     |
   |                       |               |      | content base URLs  |
   |                       |               |      | used to request    |
   |                       |               |      | this Track's       |
   |                       |               |      | Initialization     |

Speelmans               Expires 22 November 2021               [Page 33]
Internet-Draft                    HESP                          May 2021

   |                       |               |      | Packets and        |
   |                       |               |      | Continuation       |
   |                       |               |      | Segments.  See     |
   |                       |               |      | Section 3.4 for    |
   |                       |               |      | more information.  |
   +-----------------------+---------------+------+--------------------+
   | channels              | integer       | N    | The audio channel  |
   |                       |               |      | configuration of   |
   |                       |               |      | this audio Track,  |
   |                       |               |      | defined as the     |
   |                       |               |      | total number of    |
   |                       |               |      | audio channels in  |
   |                       |               |      | the media data.    |
   |                       |               |      | This is used for   |
   |                       |               |      | ABR selection by   |
   |                       |               |      | the player.        |
   +-----------------------+---------------+------+--------------------+
   | codecs                | string        | N    | The definition of  |
   |                       |               |      | the sample type    |
   |                       |               |      | used for this      |
   |                       |               |      | Track.  It MUST    |
   |                       |               |      | follow the ISO     |
   |                       |               |      | File Format Name   |
   |                       |               |      | Space as defined   |
   |                       |               |      | by [RFC6381].  If  |
   |                       |               |      | it is not defined  |
   |                       |               |      | by the Switching   |
   |                       |               |      | Set, then it MUST  |
   |                       |               |      | be defined here.   |
   +-----------------------+---------------+------+--------------------+
   | continuationPattern   | string        | N    | The URL pattern    |
   |                       |               |      | used to request    |
   |                       |               |      | Continuation       |
   |                       |               |      | Segments           |
   |                       |               |      | belonging to this  |
   |                       |               |      | Track.  The        |
   |                       |               |      | pattern needs to   |
   |                       |               |      | include the        |
   |                       |               |      | "{segmentId}"      |
   |                       |               |      | string, see        |
   |                       |               |      | Section 3.4 for    |
   |                       |               |      | more information.  |
   |                       |               |      | If the value is    |
   |                       |               |      | not set for the    |
   |                       |               |      | Track's Switching  |
   |                       |               |      | Set, then it MUST  |
   |                       |               |      | be set here.       |
   +-----------------------+---------------+------+--------------------+

Speelmans               Expires 22 November 2021               [Page 34]
Internet-Draft                    HESP                          May 2021

   | frameRate             | integer       | N    | The frame rate of  |
   |                       |               |      | the audio Track.   |
   |                       |               |      | It defines the     |
   |                       |               |      | amount of audio    |
   |                       |               |      | samples in one     |
   |                       |               |      | frame.  If         |
   |                       |               |      | neither the Track  |
   |                       |               |      | nor the Switching  |
   |                       |               |      | Set has this       |
   |                       |               |      | value set, it      |
   |                       |               |      | SHALL equal 1024.  |
   +-----------------------+---------------+------+--------------------+
   | label                 | string        | N    | A human readable   |
   |                       |               |      | name for this      |
   |                       |               |      | Track.             |
   +-----------------------+---------------+------+--------------------+
   | initializationPattern | string        | N    | The URL pattern    |
   |                       |               |      | used to request    |
   |                       |               |      | Initialization     |
   |                       |               |      | Packets belonging  |
   |                       |               |      | to this media      |
   |                       |               |      | Track.  The        |
   |                       |               |      | pattern needs to   |
   |                       |               |      | include the        |
   |                       |               |      | "{initId}"         |
   |                       |               |      | string, see        |
   |                       |               |      | Section 3.4 for    |
   |                       |               |      | more information.  |
   |                       |               |      | If the value was   |
   |                       |               |      | not set for the    |
   |                       |               |      | Track's Switching  |
   |                       |               |      | Set, then it MUST  |
   |                       |               |      | be set here.       |
   +-----------------------+---------------+------+--------------------+
   | mediaTimeOffset       | ScaledValue   | N    | The offset to be   |
   |                       |               |      | added to Manifest  |
   |                       |               |      | Timestamps in      |
   |                       |               |      | order to           |
   |                       |               |      | calculate Media    |
   |                       |               |      | Timestamps         |
   |                       |               |      | contained by       |
   |                       |               |      | segments of this   |
   |                       |               |      | Track.  If         |
   |                       |               |      | neither the Track  |
   |                       |               |      | or the Switching   |
   |                       |               |      | Set has this       |
   |                       |               |      | value set, it      |
   |                       |               |      | SHALL equal 0.     |

Speelmans               Expires 22 November 2021               [Page 35]
Internet-Draft                    HESP                          May 2021

   +-----------------------+---------------+------+--------------------+
   | sampleRate            | number        | N    | The sample rate    |
   |                       |               |      | (in Hz) of this    |
   |                       |               |      | audio Track.  If   |
   |                       |               |      | it is not defined  |
   |                       |               |      | by the Switching   |
   |                       |               |      | Set, then it MUST  |
   |                       |               |      | be defined here.   |
   +-----------------------+---------------+------+--------------------+
   | segmentDuration       | ScaledValue   | N    | The duration of    |
   |                       |               |      | all Segments       |
   |                       |               |      | contained by this  |
   |                       |               |      | Track.  If this    |
   |                       |               |      | value is set,      |
   |                       |               |      | then every n       |
   |                       |               |      | seconds a new      |
   |                       |               |      | Segment SHOULD be  |
   |                       |               |      | available.  More   |
   |                       |               |      | information about  |
   |                       |               |      | the availability   |
   |                       |               |      | is given in        |
   |                       |               |      | Section 5.2.  If   |
   |                       |               |      | not set, then      |
   |                       |               |      | each segment MUST  |
   |                       |               |      | be individually    |
   |                       |               |      | defined by the     |
   |                       |               |      | "segments" field.  |
   +-----------------------+---------------+------+--------------------+

                                  Table 11

   For any attributes that also exist on AudioSwitchingSetType, if both
   AudioSwitchingSetType and AudioTrackType have a value for this
   attribute, then only the value set by AudioTrackType must be
   considered (except for the identifier and label.)

3.2.13.  VideoTrackType

   This is the type definition of a video Track.

   +=======================+===============+======+====================+
   | Attribute             | Type          | Req? | Description        |
   +=======================+===============+======+====================+
   | bandwidth             | number        | Y    | The peak bitrate   |
   |                       |               |      | of this Track.     |
   |                       |               |      | It is denoted in   |
   |                       |               |      | bits per second.   |
   |                       |               |      | The measured       |

Speelmans               Expires 22 November 2021               [Page 36]
Internet-Draft                    HESP                          May 2021

   |                       |               |      | bitrates of all    |
   |                       |               |      | Segments of the    |
   |                       |               |      | Track MUST NOT     |
   |                       |               |      | exceed this        |
   |                       |               |      | value.  This       |
   |                       |               |      | value is used to   |
   |                       |               |      | aid in ABR         |
   |                       |               |      | decisions by the   |
   |                       |               |      | player.            |
   +-----------------------+---------------+------+--------------------+
   | id                    | string        | Y    | The unique         |
   |                       |               |      | identifier for     |
   |                       |               |      | this Track.  It    |
   |                       |               |      | MUST be unique     |
   |                       |               |      | within its         |
   |                       |               |      | Switching Set.     |
   +-----------------------+---------------+------+--------------------+
   | resolution            | Resolution    | Y    | The resolution of  |
   |                       |               |      | this video Track.  |
   +-----------------------+---------------+------+--------------------+
   | segments              | SegmentType[] | Y    | The metadata of    |
   |                       |               |      | the Segments that  |
   |                       |               |      | are contained by   |
   |                       |               |      | this Track at the  |
   |                       |               |      | moment of          |
   |                       |               |      | Manifest           |
   |                       |               |      | creation.          |
   |                       |               |      | If a segment       |
   |                       |               |      | duration is set    |
   |                       |               |      | in the Track of    |
   |                       |               |      | this segment,      |
   |                       |               |      | then only the      |
   |                       |               |      | active segment     |
   |                       |               |      | MUST be shown      |
   |                       |               |      | here.  If a        |
   |                       |               |      | segment duration   |
   |                       |               |      | is not set, then   |
   |                       |               |      | each segment MUST  |
   |                       |               |      | be announced by    |
   |                       |               |      | the Manifest       |
   |                       |               |      | before it is       |
   |                       |               |      | active.  More      |
   |                       |               |      | information about  |
   |                       |               |      | Segment            |
   |                       |               |      | availability is    |
   |                       |               |      | given in           |
   |                       |               |      | Section 5.2.       |
   +-----------------------+---------------+------+--------------------+

Speelmans               Expires 22 November 2021               [Page 37]
Internet-Draft                    HESP                          May 2021

   | activeSegment         | integer       | N    | The identifier of  |
   |                       |               |      | the segment that   |
   |                       |               |      | is currently       |
   |                       |               |      | active.            |
   |                       |               |      | If this Track's    |
   |                       |               |      | Presentation was   |
   |                       |               |      | previously active  |
   |                       |               |      | or is currently    |
   |                       |               |      | active, then the   |
   |                       |               |      | most recent        |
   |                       |               |      | Continuation       |
   |                       |               |      | Segment's          |
   |                       |               |      | identifier MUST    |
   |                       |               |      | be shown here.     |
   |                       |               |      | If the Track's     |
   |                       |               |      | Presentation has   |
   |                       |               |      | not yet been       |
   |                       |               |      | active (i.e. no    |
   |                       |               |      | Continuation       |
   |                       |               |      | Segments are yet   |
   |                       |               |      | available), then   |
   |                       |               |      | this value MUST    |
   |                       |               |      | be ignored by the  |
   |                       |               |      | client.            |
   +-----------------------+---------------+------+--------------------+
   | activeSequenceNumber  | integer       | N    | The active         |
   |                       |               |      | Sequence Number    |
   |                       |               |      | of this Track.     |
   |                       |               |      | It MUST denote     |
   |                       |               |      | the Sequence       |
   |                       |               |      | Number of the      |
   |                       |               |      | Initialization     |
   |                       |               |      | Packet of this     |
   |                       |               |      | Track that was     |
   |                       |               |      | most recently      |
   |                       |               |      | published, at the  |
   |                       |               |      | time of creation   |
   |                       |               |      | of this Manifest.  |
   |                       |               |      | If this Track's    |
   |                       |               |      | Presentation was   |
   |                       |               |      | previously active  |
   |                       |               |      | or is currently    |
   |                       |               |      | active, then the   |
   |                       |               |      | most recent        |
   |                       |               |      | Initialization     |
   |                       |               |      | Packet's Sequence  |
   |                       |               |      | Number MUST be     |
   |                       |               |      | shown here.  If    |

Speelmans               Expires 22 November 2021               [Page 38]
Internet-Draft                    HESP                          May 2021

   |                       |               |      | the Track's        |
   |                       |               |      | Presentation has   |
   |                       |               |      | not yet been       |
   |                       |               |      | active (i.e. no    |
   |                       |               |      | Initialization     |
   |                       |               |      | Packets are yet    |
   |                       |               |      | available), then   |
   |                       |               |      | this value MUST    |
   |                       |               |      | be ignored by the  |
   |                       |               |      | client.            |
   +-----------------------+---------------+------+--------------------+
   | averageBandwidth      | number        | N    | The average        |
   |                       |               |      | bitrate of this    |
   |                       |               |      | Track.  It is      |
   |                       |               |      | denoted in bits    |
   |                       |               |      | per second.        |
   |                       |               |      | If given, it is    |
   |                       |               |      | expected that      |
   |                       |               |      | over a duration    |
   |                       |               |      | of 10 minutes,     |
   |                       |               |      | the average        |
   |                       |               |      | bitrate of this    |
   |                       |               |      | Track SHALL be     |
   |                       |               |      | within 5% of the   |
   |                       |               |      | given value.       |
   |                       |               |      | This value is      |
   |                       |               |      | used to aid in     |
   |                       |               |      | ABR decisions by   |
   |                       |               |      | the player.        |
   +-----------------------+---------------+------+--------------------+
   | baseUrl               | string        | N    | The base URL of    |
   |                       |               |      | this Track.  It    |
   |                       |               |      | is part of the     |
   |                       |               |      | content base URLs  |
   |                       |               |      | used to request    |
   |                       |               |      | this Track's       |
   |                       |               |      | Initialization     |
   |                       |               |      | Packets and        |
   |                       |               |      | Continuation       |
   |                       |               |      | Segments.  See     |
   |                       |               |      | Section 3.4 for    |
   |                       |               |      | more information.  |
   +-----------------------+---------------+------+--------------------+
   | codecs                | string        | N    | The definition of  |
   |                       |               |      | the sample type    |
   |                       |               |      | used for this      |
   |                       |               |      | Track.  It MUST    |
   |                       |               |      | follow the ISO     |

Speelmans               Expires 22 November 2021               [Page 39]
Internet-Draft                    HESP                          May 2021

   |                       |               |      | File Format Name   |
   |                       |               |      | Space as defined   |
   |                       |               |      | by [RFC6381].  If  |
   |                       |               |      | it is not defined  |
   |                       |               |      | by the Switching   |
   |                       |               |      | Set, then it MUST  |
   |                       |               |      | be defined here.   |
   +-----------------------+---------------+------+--------------------+
   | continuationPattern   | string        | N    | The URL pattern    |
   |                       |               |      | used to request    |
   |                       |               |      | Continuation       |
   |                       |               |      | Segments           |
   |                       |               |      | belonging to this  |
   |                       |               |      | Track.  The        |
   |                       |               |      | pattern needs to   |
   |                       |               |      | include the        |
   |                       |               |      | "{segmentId}"      |
   |                       |               |      | string, see        |
   |                       |               |      | Section 3.4 for    |
   |                       |               |      | more information.  |
   |                       |               |      | If the value is    |
   |                       |               |      | not set for the    |
   |                       |               |      | Track's Switching  |
   |                       |               |      | Set, then it MUST  |
   |                       |               |      | be set here.       |
   +-----------------------+---------------+------+--------------------+
   | frameRate             | ScaledValue   | N    | The frame rate of  |
   |                       |               |      | this video Track.  |
   |                       |               |      | If it is not       |
   |                       |               |      | defined by the     |
   |                       |               |      | Switching Set,     |
   |                       |               |      | then it must be    |
   |                       |               |      | defined here.      |
   +-----------------------+---------------+------+--------------------+
   | label                 | string        | N    | A human readable   |
   |                       |               |      | name for this      |
   |                       |               |      | Track.             |
   +-----------------------+---------------+------+--------------------+
   | initializationPattern | string        | N    | The URL pattern    |
   |                       |               |      | used to request    |
   |                       |               |      | Initialization     |
   |                       |               |      | Packets belonging  |
   |                       |               |      | to this media      |
   |                       |               |      | Track.  The        |
   |                       |               |      | pattern needs to   |
   |                       |               |      | include the        |
   |                       |               |      | "{initId}"         |
   |                       |               |      | string, see        |

Speelmans               Expires 22 November 2021               [Page 40]
Internet-Draft                    HESP                          May 2021

   |                       |               |      | Section 3.4 for    |
   |                       |               |      | more information.  |
   |                       |               |      | If the value was   |
   |                       |               |      | not set for the    |
   |                       |               |      | Track's Switching  |
   |                       |               |      | Set, then it MUST  |
   |                       |               |      | be set here.       |
   +-----------------------+---------------+------+--------------------+
   | mediaTimeOffset       | ScaledValue   | N    | The offset to be   |
   |                       |               |      | added to Manifest  |
   |                       |               |      | Timestamps in      |
   |                       |               |      | order to           |
   |                       |               |      | calculate Media    |
   |                       |               |      | Timestamps         |
   |                       |               |      | contained by       |
   |                       |               |      | segments of this   |
   |                       |               |      | Track.  If         |
   |                       |               |      | neither the Track  |
   |                       |               |      | or the Switching   |
   |                       |               |      | Set has this       |
   |                       |               |      | value set, it      |
   |                       |               |      | SHALL equal 0.     |
   +-----------------------+---------------+------+--------------------+
   | segmentDuration       | ScaledValue   | N    | The duration of    |
   |                       |               |      | all Segments       |
   |                       |               |      | contained by this  |
   |                       |               |      | Track.  If this    |
   |                       |               |      | value is set,      |
   |                       |               |      | then every n       |
   |                       |               |      | seconds a new      |
   |                       |               |      | Segment SHOULD be  |
   |                       |               |      | available.  More   |
   |                       |               |      | information about  |
   |                       |               |      | the availability   |
   |                       |               |      | is given in        |
   |                       |               |      | Section 5.2.  If   |
   |                       |               |      | not set, then      |
   |                       |               |      | each segment MUST  |
   |                       |               |      | be individually    |
   |                       |               |      | defined by the     |
   |                       |               |      | "segments" field.  |
   +-----------------------+---------------+------+--------------------+

                                  Table 12

Speelmans               Expires 22 November 2021               [Page 41]
Internet-Draft                    HESP                          May 2021

   For any attributes that also exist on VideoSwitchingSetType, if both
   VideoSwitchingSetType and VideoTrackType have a value for this
   attribute, then only the value set by VideoTrackType must be
   considered (except for the identifier and label.)

3.2.14.  Resolution

   A Resolution contains the following elements:

    +===========+========+==========+=================================+
    | Attribute | Type   | Required | Description                     |
    +===========+========+==========+=================================+
    | width     | number | Y        | The width of the picture.       |
    +-----------+--------+----------+---------------------------------+
    | height    | number | Y        | The height of the picture.      |
    +-----------+--------+----------+---------------------------------+
    | sarWidth  | number | N        | The width of the sample aspect  |
    |           |        |          | ratio of the resolution.  If it |
    |           |        |          | is not set, it SHALL equal 1.   |
    +-----------+--------+----------+---------------------------------+
    | sarHeight | number | N        | The height of the sample aspect |
    |           |        |          | ratio of the resolution.  If it |
    |           |        |          | is not set, it SHALL equal 1.   |
    +-----------+--------+----------+---------------------------------+

                                  Table 13

   The display aspect ratio belonging to this Resolution can be
   calculated by using this sample aspect ratio width and height, and
   the picture width and height as follows:

   dar = (darWidth, darHeight)
   darWidth = sarWidth * width
   darHeight = sarHeight * height

3.2.15.  MetadataTrackType

   This is the type definition of a metadata Track.

   +=====================+===============+======+======================+
   | Attribute           | Type          | Req? | Description          |
   +=====================+===============+======+======================+
   | id                  | string        | Y    | The unique           |
   |                     |               |      | identifier for this  |
   |                     |               |      | Track.  It MUST be   |
   |                     |               |      | unique within its    |
   |                     |               |      | Switching Set.       |
   +---------------------+---------------+------+----------------------+

Speelmans               Expires 22 November 2021               [Page 42]
Internet-Draft                    HESP                          May 2021

   | segments            | SegmentType[] | Y    | The metadata of the  |
   |                     |               |      | Segments that are    |
   |                     |               |      | contained by this    |
   |                     |               |      | Track at the moment  |
   |                     |               |      | of Manifest          |
   |                     |               |      | creation.            |
   |                     |               |      | If a segment         |
   |                     |               |      | duration is set in   |
   |                     |               |      | the Track of this    |
   |                     |               |      | segment, then only   |
   |                     |               |      | the active segment   |
   |                     |               |      | MUST be shown here.  |
   |                     |               |      | If a segment         |
   |                     |               |      | duration is not      |
   |                     |               |      | set, then each       |
   |                     |               |      | segment MUST be      |
   |                     |               |      | announced by the     |
   |                     |               |      | Manifest before it   |
   |                     |               |      | is active.  More     |
   |                     |               |      | information about    |
   |                     |               |      | Segment              |
   |                     |               |      | availability is      |
   |                     |               |      | given in             |
   |                     |               |      | Section 5.2.         |
   +---------------------+---------------+------+----------------------+
   | activeSegment       | integer       | N    | The identifier of    |
   |                     |               |      | the segment that is  |
   |                     |               |      | currently active.    |
   |                     |               |      | If this Track's      |
   |                     |               |      | Presentation was     |
   |                     |               |      | previously active    |
   |                     |               |      | or is currently      |
   |                     |               |      | active, then the     |
   |                     |               |      | most recent          |
   |                     |               |      | Continuation         |
   |                     |               |      | Segment's            |
   |                     |               |      | identifier MUST be   |
   |                     |               |      | shown here.  If the  |
   |                     |               |      | Track's              |
   |                     |               |      | Presentation has     |
   |                     |               |      | not yet been active  |
   |                     |               |      | (i.e. no             |
   |                     |               |      | Continuation         |
   |                     |               |      | Segments are yet     |
   |                     |               |      | available), then     |
   |                     |               |      | this value MUST be   |
   |                     |               |      | ignored by the       |
   |                     |               |      | client.              |

Speelmans               Expires 22 November 2021               [Page 43]
Internet-Draft                    HESP                          May 2021

   +---------------------+---------------+------+----------------------+
   | averageBandwidth    | number        | N    | The average bitrate  |
   |                     |               |      | of this Track.  It   |
   |                     |               |      | is denoted in bits   |
   |                     |               |      | per second.          |
   |                     |               |      | If given, it is      |
   |                     |               |      | expected that over   |
   |                     |               |      | a duration of 10     |
   |                     |               |      | minutes, the         |
   |                     |               |      | average bitrate of   |
   |                     |               |      | this Track SHALL be  |
   |                     |               |      | within 5% of the     |
   |                     |               |      | given value.  This   |
   |                     |               |      | value is used to     |
   |                     |               |      | aid in ABR           |
   |                     |               |      | decisions by the     |
   |                     |               |      | player.              |
   +---------------------+---------------+------+----------------------+
   | bandwidth           | number        | N    | The peak bitrate of  |
   |                     |               |      | this Track.  It is   |
   |                     |               |      | denoted in bits per  |
   |                     |               |      | second.  The         |
   |                     |               |      | measured bitrates    |
   |                     |               |      | of all Segments of   |
   |                     |               |      | the Track MUST NOT   |
   |                     |               |      | exceed this value.   |
   |                     |               |      | This value is used   |
   |                     |               |      | to aid in ABR        |
   |                     |               |      | decisions by the     |
   |                     |               |      | player.              |
   +---------------------+---------------+------+----------------------+
   | baseUrl             | string        | N    | The base URL of      |
   |                     |               |      | this Track.  It is   |
   |                     |               |      | part of the content  |
   |                     |               |      | base URLs used to    |
   |                     |               |      | request this         |
   |                     |               |      | Track's              |
   |                     |               |      | Continuation         |
   |                     |               |      | Segments.  See       |
   |                     |               |      | Section 3.4 for      |
   |                     |               |      | more information.    |
   +---------------------+---------------+------+----------------------+
   | codecs              | string        | N    | The definition of    |
   |                     |               |      | the sample type      |
   |                     |               |      | used for this        |
   |                     |               |      | Track.  It MUST      |
   |                     |               |      | follow the ISO File  |
   |                     |               |      | Format Name Space    |

Speelmans               Expires 22 November 2021               [Page 44]
Internet-Draft                    HESP                          May 2021

   |                     |               |      | as defined by        |
   |                     |               |      | [RFC6381].  If it    |
   |                     |               |      | is not defined by    |
   |                     |               |      | the Switching Set,   |
   |                     |               |      | then it MUST be      |
   |                     |               |      | defined here.        |
   +---------------------+---------------+------+----------------------+
   | continuationPattern | string        | N    | The URL pattern      |
   |                     |               |      | used to request      |
   |                     |               |      | Continuation         |
   |                     |               |      | Segments belonging   |
   |                     |               |      | to this Track.  The  |
   |                     |               |      | pattern needs to     |
   |                     |               |      | include the          |
   |                     |               |      | "{segmentId}"        |
   |                     |               |      | string, see          |
   |                     |               |      | Section 3.4 for      |
   |                     |               |      | more information.    |
   |                     |               |      | If the value is not  |
   |                     |               |      | set for the Track's  |
   |                     |               |      | Switching Set, then  |
   |                     |               |      | it MUST be set       |
   |                     |               |      | here.                |
   +---------------------+---------------+------+----------------------+
   | label               | string        | N    | A human readable     |
   |                     |               |      | name for this        |
   |                     |               |      | Track.               |
   +---------------------+---------------+------+----------------------+
   | mediaTimeOffset     | ScaledValue   | N    | The offset to be     |
   |                     |               |      | added to Manifest    |
   |                     |               |      | Timestamps in order  |
   |                     |               |      | to calculate Media   |
   |                     |               |      | Timestamps           |
   |                     |               |      | contained by         |
   |                     |               |      | segments of this     |
   |                     |               |      | Track.  If neither   |
   |                     |               |      | the Track or the     |
   |                     |               |      | Switching Set has    |
   |                     |               |      | this value set, it   |
   |                     |               |      | SHALL equal 0.       |
   +---------------------+---------------+------+----------------------+
   | segmentDuration     | ScaledValue   | N    | The duration of all  |
   |                     |               |      | Segments contained   |
   |                     |               |      | by this Track.  If   |
   |                     |               |      | this value is set,   |
   |                     |               |      | then every n         |
   |                     |               |      | seconds a new        |
   |                     |               |      | Segment SHOULD be    |

Speelmans               Expires 22 November 2021               [Page 45]
Internet-Draft                    HESP                          May 2021

   |                     |               |      | available.  More     |
   |                     |               |      | information about    |
   |                     |               |      | the availability is  |
   |                     |               |      | given in             |
   |                     |               |      | Section 5.2.  If     |
   |                     |               |      | not set, then each   |
   |                     |               |      | segment MUST be      |
   |                     |               |      | individually         |
   |                     |               |      | defined by the       |
   |                     |               |      | "segments" field.    |
   +---------------------+---------------+------+----------------------+

                                  Table 14

3.2.16.  SegmentType

   This is the type definition of a Segment.

   +============+============+==========+=============================+
   | Attribute  | Type       | Required | Description                 |
   +============+============+==========+=============================+
   | id         | integer    | Y        | The unique identifier for   |
   |            |            |          | this Segment within the     |
   |            |            |          | Track.  This identifier     |
   |            |            |          | MUST be incremented by 1    |
   |            |            |          | for every new Segment of    |
   |            |            |          | the same Track.             |
   +------------+------------+----------+-----------------------------+
   | timeBounds | TimeBounds | N        | The time boundaries of this |
   |            |            |          | Segment.  If this Segment's |
   |            |            |          | Track's does not have a     |
   |            |            |          | constant segment duration,  |
   |            |            |          | then this value MUST be set |
   |            |            |          | and the segment's duration  |
   |            |            |          | MUST be available at least  |
   |            |            |          | 2 seconds before the actual |
   |            |            |          | end of the segment.         |
   +------------+------------+----------+-----------------------------+

                                 Table 15

3.2.17.  PresentationEventType

   Timed metadata events can be embedded in the Manifest by using
   Presentation Events.  Each PresentationEventType contains the
   following elements:

Speelmans               Expires 22 November 2021               [Page 46]
Internet-Draft                    HESP                          May 2021

   +============+=============================+======+=================+
   | Attribute  | Type                        | Req? | Description     |
   +============+=============================+======+=================+
   | data       | string                      | Y    | The event       |
   |            |                             |      | payload.        |
   +------------+-----------------------------+------+-----------------+
   | id         | string                      | Y    | An unique       |
   |            |                             |      | identifier for  |
   |            |                             |      | this event.     |
   |            |                             |      | It MUST be      |
   |            |                             |      | unique within   |
   |            |                             |      | this            |
   |            |                             |      | Presentation's  |
   |            |                             |      | events.         |
   +------------+-----------------------------+------+-----------------+
   | timeBounds | PresentationEventTimeBounds | Y    | The time        |
   |            |                             |      | boundaries for  |
   |            |                             |      | the event,      |
   |            |                             |      | during which    |
   |            |                             |      | the event       |
   |            |                             |      | SHALL be        |
   |            |                             |      | active.         |
   +------------+-----------------------------+------+-----------------+
   | encoding   | Enum("identity", "base64",  | N    | The content     |
   |            | "json")                     |      | encoding of     |
   |            |                             |      | the event       |
   |            |                             |      | data.           |
   |            |                             |      | "identity"      |
   |            |                             |      | signifies that  |
   |            |                             |      | the encoding    |
   |            |                             |      | is plaintext.   |
   |            |                             |      | "base64"        |
   |            |                             |      | signifies       |
   |            |                             |      | Base64          |
   |            |                             |      | encoding.       |
   |            |                             |      | "json"          |
   |            |                             |      | signifies that  |
   |            |                             |      | the payload     |
   |            |                             |      | given by        |
   |            |                             |      | "data" is a     |
   |            |                             |      | valid JSON      |
   |            |                             |      | document.  If   |
   |            |                             |      | this JSON       |
   |            |                             |      | document is     |
   |            |                             |      | not valid,      |
   |            |                             |      | this event      |
   |            |                             |      | MUST be         |
   |            |                             |      | ignored.  If    |

Speelmans               Expires 22 November 2021               [Page 47]
Internet-Draft                    HESP                          May 2021

   |            |                             |      | not set, the    |
   |            |                             |      | encoding SHALL  |
   |            |                             |      | default to      |
   |            |                             |      | "identity".     |
   +------------+-----------------------------+------+-----------------+

                                  Table 16

3.2.18.  PresentationEventTimeBounds

   The time boundaries of a PresentationEventType are defined as
   follows:

   +=================+=========+==========+===========================+
   | Attribute       | Type    | Required | Description               |
   +=================+=========+==========+===========================+
   | startTimeOffset | integer | N        | The scaled start time     |
   |                 |         |          | offset of the event,      |
   |                 |         |          | defined in seconds        |
   |                 |         |          | divided by the timescale. |
   |                 |         |          | The actual start time of  |
   |                 |         |          | the event can be          |
   |                 |         |          | calculated by dividing    |
   |                 |         |          | this value by the scale   |
   |                 |         |          | and adding the resulting  |
   |                 |         |          | value to the start time   |
   |                 |         |          | of the Presentation.  It  |
   |                 |         |          | is defined as seconds     |
   |                 |         |          | divided by the timescale. |
   |                 |         |          | If it is not set, it      |
   |                 |         |          | SHALL equal 0.            |
   +-----------------+---------+----------+---------------------------+
   | duration        | integer | N        | The scaled duration of    |
   |                 |         |          | the event, defined in     |
   |                 |         |          | seconds divided by the    |
   |                 |         |          | timescale.  If it is not  |
   |                 |         |          | set, it SHALL equal 0.    |
   +-----------------+---------+----------+---------------------------+
   | scale           | integer | N        | The timescale used for    |
   |                 |         |          | these time bounds.  If it |
   |                 |         |          | is not set, it SHALL be   |
   |                 |         |          | equal to 1.               |
   +-----------------+---------+----------+---------------------------+

                                 Table 17

Speelmans               Expires 22 November 2021               [Page 48]
Internet-Draft                    HESP                          May 2021

3.3.  Manifest requests

   The Manifest SHOULD be available through an HTTP GET request.  The
   URL of this Manifest can be personalized by the user.

     +==================================+========+==================+
     | Request path                     | Method | Summary          |
     +==================================+========+==================+
     | (chosen by the stream publisher) | GET    | Retrieve the     |
     |                                  |        | stream Manifest. |
     +----------------------------------+--------+------------------+

                       Table 18: Manifest Requests

3.3.1.  Manifest responses

         +==============+========================================+
         | Success      |                                        |
         +==============+========================================+
         | Status code  | 200                                    |
         +--------------+----------------------------------------+
         | Content Type | JSON (MIME type MUST equal             |
         |              | "application/vnd.theo.hesp+json")      |
         +--------------+----------------------------------------+
         | Content      | Manifest data                          |
         +--------------+----------------------------------------+
         | Description  | When the given Manifest exists on the  |
         |              | server, the Manifest data is returned. |
         +--------------+----------------------------------------+

                   Table 19: Manifest Responses: Success

    +=============+=========+=========+==============================+
    | Status code | Content | Content | Description                  |
    |             |         | type    |                              |
    +=============+=========+=========+==============================+
    | 404         | -       | -       | The Manifest is not          |
    |             |         |         | available for the given URL. |
    +-------------+---------+---------+------------------------------+

                   Table 20: Manifest Responses: Error

   If an error occurs, the player SHOULD attempt to retry the request.
   In case of consecutive unsuccessful requests, the media player SHOULD
   assume the content is unavailable and cease playback.

Speelmans               Expires 22 November 2021               [Page 49]
Internet-Draft                    HESP                          May 2021

3.4.  Addressing of content requests

   Several elements of the Manifest contain a "baseUrl" attribute.
   These attributes are used to construct the URLs of media content (and
   metadata.)

3.4.1.  Content request URL resolution

   A content request URL for a certain Track SHALL be constructed by
   applying relative resolution (as explained in Section 5.2 of RFC 3986
   [RFC3986]) to each defined "baseUrl" or "contentBaseUrl" attribute
   relating to that Track, starting from the root of the Manifest.  The
   URL of the Manifest SHALL be as a base for the first resolution,
   after which the target URL of the previous resolution SHALL be used
   as the base URL of the next resolution.

   This means that the content request URL for a Track's Initialization
   Stream is constructed as follows (in pseudocode):

   T = manifestUrl
   if isDefined(Manifest.contentBaseUrl):
         T = resolve(Manifest.contentBaseUrl, T)
   if isDefined(Presentation.baseUrl):
         T = resolve(Presentation.baseUrl, T)
   if isDefined(SwitchingSet.baseUrl):
         T = resolve(SwitchingSet.baseUrl, T)
   if isDefined(Track.baseUrl):
         T = resolve(Track.baseUrl, T)
   contentRequestURL = resolve(initializationPattern, T)

   where "initializationPattern" is the attribute given by either
   SwitchingSet or Track (with Track taking precedence if given by both)
   and "resolve(R, B)" is relative resolution, where R is a URI
   reference and B is a base URI.

   The content request URL for the Track's Continuation Stream can be
   constructed in the same manner, if the "initializationPattern" is
   replaced with the "continuationPattern".  Both URLs MUST be unique
   for each unique Track.

   An example of content addressing is given in Appendix A.1.3.

Speelmans               Expires 22 November 2021               [Page 50]
Internet-Draft                    HESP                          May 2021

3.4.2.  Requesting using an identifier and the content request URL

   The content request URL MUST include the "{initId}" pattern for
   Initialization Stream URLs and the "{segmentId}" pattern for
   Continuation Stream URLs.  These patterns are replaced in the actual
   content requests: "{initId}" is replaced with the requested Sequence
   Number of the Initialization Stream and "{segmentId}" is replaced
   with the requested segment identifier of the Continuation Stream.

   In order to add leading zeros to the identifier, the following can be
   added to the patterns "initId" and "segmentId": ":0(n)d" where "(n)"
   is the minimal amount of characters the resulting identifier has (if
   the identifier is already longer, it will not be altered.)

   For example, for a continuation segment with identifier 100, a
   following content request URL "https://www.example.com/s-1/
   content-{segmentId:06d}.mp4" will resolve to
   "https://www.example.com/s-1/content-000100.mp4".  While the content
   request URL "https://www.example.com/s-1/content-{segmentId:02d}.mp4"
   resolves to "https://www.example.com/s-1/content-100.mp4"

   Further details about these requests are given in Section 4 and
   Section 5.

3.5.  Manifest example

   A full Manifest example, together with information on content
   addressing and timing information is given by Appendix A.1.

4.  Initialization Stream

   The player uses the information in the HESP Manifest to fetch an
   Initialization Packet.

4.1.  Initialization Stream purpose

   The Initialization Stream is a stream containing only independent
   samples.  This stream is not regularly used by a single client, nor
   completely streamed.  The purpose of the Initialization Stream is to
   make available, upon request of the client, the most recent
   independent frame, packaged in an Initialization Packet, so that
   decoding can start at a certain position, for example to start
   playback at the live edge, seek to a specific point in time, or
   switch to an alternative Track.

   Additionally, the Initialization Packet contains information on the
   position of the following frame in the Continuation Stream, in order
   to achieve regular media playback.

Speelmans               Expires 22 November 2021               [Page 51]
Internet-Draft                    HESP                          May 2021

4.2.  Initialization Packet format

4.2.1.  Video Initialization Packet

   A video Initialization Packet MUST contain:

   *  A CMAF header, i.e. all information required to initialize the
      media decoder, stored in ISO Base Media File Format [ISOBMFF]
      boxes as specified by CMAF [CMAF].  The minimal required contents
      are defined in Section 4.2.3

      -  An additional in-band metadata event containing extra HESP
         information, included in the aforementioned CMAF segment.  The
         definition of such an event is denoted in Section 4.2.4.

   *  A CMAF segment, i.e. an independent media sample and timing
      information related to this sample, stored in ISOBMFF [ISOBMFF]
      boxes as specified by CMAF [CMAF].

4.2.2.  Audio Initialization Packet

   An audio Initialization Packet MUST contain:

   *  A CMAF header, i.e. all information required to initialize the
      media decoder, stored in ISO Base Media File Format [ISOBMFF]
      boxes as specified by CMAF [CMAF].  The minimal required contents
      are defined below.

   *  An additional in-band metadata event containing extra HESP
      information, included in the aforementioned CMAF segment.  The
      definition of such an event is denoted below.

   It MAY also contain an independent (audio) media sample, as is
   defined by the video Initialization Packet.  However, it is heavily
   discouraged as it leads to more storage costs for the provider
   without any significant advantage.

4.2.3.  CMAF header

   A CMAF header is defined in ISO/IEC 23000-19:2020 [CMAF] as: "a
   sequence of CMAF constrained ISOBMFF boxes that do not reference any
   media samples (3.3.15), but are associated with a CMAF track (3.2.1)
   and necessary for the decoding of its CMAF fragments (3.1.1)."  As
   such, the header MUST NOT contain any samples.

   This means that for the ISO Base Media File Format, at least the
   following boxes MUST be given:

Speelmans               Expires 22 November 2021               [Page 52]
Internet-Draft                    HESP                          May 2021

   *  An "ftyp" box.

   *  A "moov" box.

   *  One or more tracks and accompanying boxes, though "stbl" and other
      related boxes cannot give information about any actual sample
      entries.

4.2.4.  Event message information

   In order to pass information about the Continuation Stream, an in-
   band event in the form of an "emsg" box must be added to the
   Initialization Packet.  The format of this box is defined at
   Section 6.2.1.1.  The "message_data" field MUST contain the following
   two values (see Table 31):

   *  the Continuation Segment index ("index"), which defines the index
      of the Continuation Segment which contains the next media sample.

   *  the Continuation Segment byte offset ("offset"), which defines the
      position of the next media sample in the Continuation Segment.  It
      MAY be unset if the next media sample is located at the start of
      the segment, in which case a byte range request is not needed.

   The "message_data" field of the "emsg" box could for example look
   like this:

   "{"index":1,"offset":1234}"

4.3.  Initialization Stream addressing

   Section 3.4 defines how to create the correct URLs and how to request
   Initialization Packets.  The request content URL is used together
   with a desired Sequence Number in order to retrieve an Initialization
   Packet.

   Additionally, "{initId}" can also be replaced with the string "now".
   If this URL is requested, the most recently available Initialization
   Packet MUST be returned.

4.3.1.  Initialization Stream requests

   Initialization Packets MUST be available through a basic GET request.
   They MUST be able to be retrieved through two methods:

Speelmans               Expires 22 November 2021               [Page 53]
Internet-Draft                    HESP                          May 2021

     +========================+========+============================+
     | Request path           | Method | Summary                    |
     +========================+========+============================+
     | Initialization content | GET    | Retrieve the most recent   |
     | request URL            |        | Initialization Packet from |
     | "{initId}" is replaced |        | the given track.           |
     | with the string "now"  |        |                            |
     +------------------------+--------+----------------------------+
     | Initialization content | GET    | Retrieve the               |
     | request URL            |        | Initialization Packet with |
     | "{initId}" is replaced |        | the given Sequence Number  |
     | with a Sequence Number |        | from the given track.      |
     +------------------------+--------+----------------------------+

                 Table 21: Initialization Stream requests

4.3.2.  Initialization Stream responses

       +==============+============================================+
       | Success      |                                            |
       +==============+============================================+
       | Status Code  | 200                                        |
       +--------------+--------------------------------------------+
       | Content type | "video/mp4" (video) or "audio/mp4" (audio) |
       +--------------+--------------------------------------------+
       | Description  | When the requested Initialization Packet   |
       |              | exists on the media server.                |
       +--------------+--------------------------------------------+

             Table 22: Initialization Stream responses: success

     +========+=========+=========+=================================+
     | Status | Content | Content | Description                     |
     | code   |         | type    |                                 |
     +========+=========+=========+=================================+
     | 404    | -       | -       | The Track does not exist on the |
     |        |         |         | media server or the requested   |
     |        |         |         | Initialization Packet does not  |
     |        |         |         | exist on the media server.      |
     +--------+---------+---------+---------------------------------+

             Table 23: Initialization Stream responses: error

Speelmans               Expires 22 November 2021               [Page 54]
Internet-Draft                    HESP                          May 2021

5.  Continuation Stream

   The client uses the information given by the Manifest and the
   Initialization Packet to request the Continuation Stream.  This low
   latency stream is used for the bulk of media playback of HESP.  The
   Continuation Stream is an independently playable CMAF stream.

5.1.  Continuation Stream format

5.1.1.  Media content

   The Continuation Stream is packaged as a regular CMAF stream, albeit
   with some encoding constraints, depending on the chosen profile (see
   Appendix C.)

   Maximum Gain Profile

   *  Samples MUST only contain references to the one sample preceding
      it.

   *  Each CMAF chunk MUST contain at most one sample (for lowest
      possible latency.)

   *  Long CMAF Segments (values of multiple minutes are possible.)

   Compatibility Profile

   *  Regular sized CMAF Segments (recommendation: between 1 and 30
      seconds)

   *  Chunk sizes range between one sample and one sub GOP

   *  The following GOP structure MUST be followed: I B ... B P B ... B
      P B ... B P

   *  The number of B frames MAY vary from 0 (same behavior as Maximum
      Gain Profile) to 4 (recommended maximum)

   *  Each sub GOP (B ... B P) MUST only reference one previous frame
      (allowing injection of keyframes)

5.2.  Continuation Segment availability

   Continuation Segments must together create a continuous stream of
   media data.  There MUST NOT exist gaps in the time boundaries of
   subsequent Continuation Segments.

Speelmans               Expires 22 November 2021               [Page 55]
Internet-Draft                    HESP                          May 2021

   There exist two ways to signal Segment availability.  Either the
   Track has a "segmentDuration" set in the Manifest, which represents
   the constant duration of each Segment.  In this case, a client can
   derive on its own when to start requesting a new Segment.  A new
   Segment SHOULD be published every n seconds, where n equals
   "segmentDuration".  This new Segment MUST become available within 100
   milliseconds from that point.  It is not allowed to drift, i.e. if a
   Segment is only published at n+100ms, then the next segment MUST be
   available at 2n+100ms at the latest.

   The duration of the last Segment of a Track MAY be shorter than this
   constant "segmentDuration".  As a new Manifest must be retrieved near
   the end of a Presentation, a client should be able to start
   requesting content of the new Presentation in a timely manner,
   regardless of the length of this last Segment.

   The other option is for the Track to specify all its Segments
   outright in the "segments" field of the Manifest.  As the Manifest
   does not get regularly retrieved, it is REQUIRED to signal a Manifest
   update (see Section 6.2.1.2) at the end of each Segment.  It is
   recommended to only use this option in the "Maximum Gain Profile", as
   it can otherwise significantly increase the amount of Manifest
   requests necessary.

5.3.  Continuation Stream addressing

   To start playback of a certain Track, a client MUST request the
   Continuation Segments of the chosen Track(s) using the information
   given by (already obtained) the Manifest and Initialization Packet.

   Requests for Content Segments are to be made using HTTP GET requests.
   The first request, following an Initialization Packet, fetches the
   Segment indicated by the Initialization Packet.  A byte range header
   should be used to specify the range of data that needs to be
   requested.  This information is also given by the Initialization
   Packet.

   A client then automatically calculates the URL of the next Segment as
   indicated in the Manifest and requests the next segment using a
   regular HTTP GET request.

5.3.1.  Continuation Stream URLs

   Section 3.4 defines how to create the correct URLs and how to request
   a continuation segment.  The request content URL is used together
   with a desired segment identifier in order to retrieve the segment.

Speelmans               Expires 22 November 2021               [Page 56]
Internet-Draft                    HESP                          May 2021

5.3.2.  Continuation Stream requests

   If retrieved through the metadata given by the Initialization Packet,
   Continuation Segments must be fetched using a range request.  The
   requested end of the range should be a very large value to ensure the
   entire Segment is retrieved.  If this offset equals 0, the client
   SHOULD use a non-range request.

    +===========================+========+===========================+
    | Request path              | Method | Summary                   |
    +===========================+========+===========================+
    | Continuation content      | GET    | Retrieve the Continuation |
    | request URL               |        | Segment with the          |
    | "{segmentId}" is replaced |        | requested identifier from |
    | with a segment identifier |        | the given track.          |
    +---------------------------+--------+---------------------------+

                  Table 24: Continuation Stream requests

5.3.3.  Continuation Stream responses

   A distinction is made between how responses should be returned,
   depending on the version of the HTTP protocol used.

5.3.3.1.  HTTP/1.1 successful response

   If a HTTP Range request is sent, then a 206 Partial Content response
   MUST be returned upon a successful response.  The response MUST use
   Chunked Transfer Coding [RFC7230] to ensure timely delivery of media
   data.

Speelmans               Expires 22 November 2021               [Page 57]
Internet-Draft                    HESP                          May 2021

        +=============+==========================================+
        | Success     |                                          |
        +=============+==========================================+
        | Status code | 206                                      |
        +-------------+------------------------------------------+
        | Content     | "video/mp4" (video) or "audio/mp4"       |
        | Type        | (audio) or any type (metadata)           |
        +-------------+------------------------------------------+
        | Transfer    | chunked                                  |
        | Encoding    |                                          |
        +-------------+------------------------------------------+
        | Description | The requested range of Segment data is   |
        |             | returned from the server.  Depending on  |
        |             | the byte range requested, the connection |
        |             | is kept open to retrieve live data.      |
        +-------------+------------------------------------------+

             Table 25: Continuation Stream responses: success
                         (Range header is given)

   For requests without a Range header, a 200 OK response MUST be
   returned upon success.

    +=============+===================================================+
    | Success     |                                                   |
    +=============+===================================================+
    | Status code | 200                                               |
    +-------------+---------------------------------------------------+
    | Content     | "video/mp4" (video) or "audio/mp4" (audio) or any |
    | Type        | type (metadata)                                   |
    +-------------+---------------------------------------------------+
    | Transfer    | chunked                                           |
    | Encoding    |                                                   |
    +-------------+---------------------------------------------------+
    | Description | The Segment data is returned from the server.     |
    |             | Depending on the availability of the Segment, the |
    |             | connection is kept open to retrieve live data.    |
    +-------------+---------------------------------------------------+

       Table 26: Continuation Stream responses: success (Range header
                               is not given)

5.3.3.2.  HTTP/2 successful response

   HTTP/2 uses frame based transmission and cannot not use Chunked
   Transfer Coding.  As such, this header is not given here.

Speelmans               Expires 22 November 2021               [Page 58]
Internet-Draft                    HESP                          May 2021

        +=============+==========================================+
        | Success     |                                          |
        +=============+==========================================+
        | Status code | 206                                      |
        +-------------+------------------------------------------+
        | Content     | "video/mp4" (video) or "audio/mp4"       |
        | Type        | (audio) or any type (metadata)           |
        +-------------+------------------------------------------+
        | Description | The requested range of Segment data is   |
        |             | returned from the server.  Depending on  |
        |             | the byte range requested, the connection |
        |             | is kept open to retrieve live data.      |
        +-------------+------------------------------------------+

             Table 27: Continuation Stream responses: success
                         (Range header is given)

   For requests without a Range header, a 200 OK response MUST be
   returned upon success.

    +=============+===================================================+
    | Success     |                                                   |
    +=============+===================================================+
    | Status code | 200                                               |
    +-------------+---------------------------------------------------+
    | Content     | "video/mp4" (video) or "audio/mp4" (audio) or any |
    | Type        | type (metadata)                                   |
    +-------------+---------------------------------------------------+
    | Description | The Segment data is returned from the server.     |
    |             | Depending on the availability of the Segment, the |
    |             | connection is kept open to retrieve live data.    |
    +-------------+---------------------------------------------------+

       Table 28: Continuation Stream responses: success (Range header
                               is not given)

5.3.3.3.  Response errors

        +========+=========+=========+============================+
        | Status | Content | Content | Description                |
        | code   |         | type    |                            |
        +========+=========+=========+============================+
        | 404    | -       | -       | The Track does not exist   |
        |        |         |         | on the media server or the |
        |        |         |         | requested Segment does not |
        |        |         |         | exist on the media server. |
        +--------+---------+---------+----------------------------+
        | 416    | -       | -       | The requested byte range   |

Speelmans               Expires 22 November 2021               [Page 59]
Internet-Draft                    HESP                          May 2021

        |        |         |         | could not be fulfilled.    |
        +--------+---------+---------+----------------------------+

               Table 29: Continuation Stream responses: error

6.  Timed metadata

   HESP has support for timed metadata through two methods: metadata
   Tracks for continuous, often segmented metadata and metadata events
   for sporadic metadata updates.

6.1.  Metadata Tracks

   Metadata Tracks function similarly to media Tracks, however without
   the need for an Initialization Stream.  The attributes for metadata
   Tracks in the Manifest can be found at Section 3.2.15.  The
   addressing happens similarly to the addressing of Continuation
   segments: a "continuationPattern" is given for each metadata Track,
   either directly set on the Track or on the Switching Set of the
   Track.  Each metadata Segment contains a numerical identifier that
   increments by one for each new Segment of the Track.  This identifier
   together with the addressing pattern creates the URL used to retrieve
   the contents of the Segment.

   The Manifest contains the duration of each metadata Segment, either
   stated individually per Segment, or through the "segmentDuration"
   attribute set on the metadata Track.  It is possible that chunked
   encoding is used here to ensure that the contents are delivered as
   soon as possible, but this is not a requirement.  This can be useful
   for e.g. subtitles alongside live content.

6.2.  Metadata events

   Metadata events can be used to signal information that does not
   become available in regular intervals.  These events can either be
   transmitted in-band, where it MUST be added to the video or audio
   Continuation Streams, or out-of-band, in which case details about the
   metadata event MUST be available in the Manifest.

6.2.1.  In-band events

   In order to deliver events in-band, root level Event Message ("emsg")
   boxes MUST be added to ongoing media Continuation Streams.  The
   definition of an "emsg" box can be found in the CMAF [CMAF]
   specification.

Speelmans               Expires 22 November 2021               [Page 60]
Internet-Draft                    HESP                          May 2021

   These boxes should be appended to each Track of a Presentation in
   order to ensure all viewers can receive such an event.  It is
   recommended to use out-of-band events if the included data is
   significantly large.

   The HESP specification defines a few in-band events that it leverages
   for client initialization and for Manifest updates.  The structure of
   these events is given below.

6.2.1.1.  Initialization data

   This event is appended to each Initialization Packet in order for the
   client to request the correct Continuation Segments.  It is an 'emsg'
   box with the following REQUIRED values:

      +=========================+==================================+
      | Attribute               | Value                            |
      +=========================+==================================+
      | version                 | 0                                |
      +-------------------------+----------------------------------+
      | scheme_id_uri           | "urn:theo:hesp:2020"             |
      +-------------------------+----------------------------------+
      | value                   | "initdata"                       |
      +-------------------------+----------------------------------+
      | timescale               | MUST match the timescale of the  |
      |                         | Initialization Packet in case of |
      |                         | video or MUST equal 1 otherwise. |
      +-------------------------+----------------------------------+
      | presentation_time_delta | 0                                |
      +-------------------------+----------------------------------+
      | event_duration          | MUST match the duration of the   |
      |                         | Initialization Packet in case of |
      |                         | video or MUST equal 0 otherwise. |
      +-------------------------+----------------------------------+
      | id                      | can be freely set (and MUST be   |
      |                         | ignored by the player)           |
      +-------------------------+----------------------------------+
      | message_data            | MUST contain the data defined by |
      |                         | Table 31, formatted as JSON      |
      +-------------------------+----------------------------------+

       Table 30: "emsg" box containing an initialization data event

Speelmans               Expires 22 November 2021               [Page 61]
Internet-Draft                    HESP                          May 2021

    +===========+=========+==========+================================+
    | Attribute | Type    | Required | Description                    |
    +===========+=========+==========+================================+
    | index     | integer | Y        | The Continuation Segment index |
    |           |         |          | (see Section 4.2.4.)           |
    +-----------+---------+----------+--------------------------------+
    | offset    | integer | N        | The Continuation Segment byte  |
    |           |         |          | offset (see Section 4.2.4), it |
    |           |         |          | SHALL be 0 if not given here.  |
    +-----------+---------+----------+--------------------------------+

     Table 31: "message_data" contents of an initialization data event

6.2.1.2.  Manifest update

   This event is used to signal the client that a new Manifest must be
   retrieved.  This can occur for a multitude of reasons, such as before
   a Presentation change, availability of new out-of-band metadata,...
   It is an "emsg" box with the following REQUIRED values:

       +=========================+================================+
       | Attribute               | Value                          |
       +=========================+================================+
       | version                 | 0                              |
       +-------------------------+--------------------------------+
       | scheme_id_uri           | "urn:theo:hesp:2020"           |
       +-------------------------+--------------------------------+
       | value                   | "manifestupdate"               |
       +-------------------------+--------------------------------+
       | timescale               | 1                              |
       +-------------------------+--------------------------------+
       | presentation_time_delta | 0                              |
       +-------------------------+--------------------------------+
       | event_duration          | 0                              |
       +-------------------------+--------------------------------+
       | id                      | MAY be freely set (and MUST be |
       |                         | ignored by the player)         |
       +-------------------------+--------------------------------+
       | message_data            | MUST contain the data defined  |
       |                         | by Table 33, formatted as JSON |
       +-------------------------+--------------------------------+

         Table 32: "emsg" box containing a Manifest update event

Speelmans               Expires 22 November 2021               [Page 62]
Internet-Draft                    HESP                          May 2021

       +===========+========+==========+===========================+
       | Attribute | Type   | Required | Description               |
       +===========+========+==========+===========================+
       | url       | string | N        | the URL of an alternative |
       |           |        |          | location of the Manifest  |
       +-----------+--------+----------+---------------------------+

         Table 33: "message_data" contents of a Manifest data event

   If the event contains a URL, then this Manifest request MUST be made
   to this address.  Further Manifest requests do not have this
   requirement.

6.2.2.  Out-of-band events

   Out-of-band events can be added to the Manifest through Presentation
   Events.  The definition of a Presentation Event is given in
   Section 3.2.17.  The contents of the event data are not predefined.
   It is possible to include arbitrary data as Base64 encoded data and
   plaintext data can be included as is.  If needed, the data can simply
   be an URL that needs to be resolved separately, but this is up to the
   publisher of this data.

7.  Content protection

   HESP has support for content protection.  It allows DRM systems to be
   implemented through the common encryption standard.

7.1.  Common encryption support

   As HESP has the requirement for media to be structured as ISOBMFF
   [ISOBMFF], common encryption [CENC] is to be used to encrypt that
   media content.

   Common encryption specifies 4 protection schemes that may be used:
   "cenc", "cbc1", "cens" and "cbcs".  For HESP, either the AES-CBC
   subsample pattern encryption scheme ("cbcs") or the AES-CTR full
   sample pattern encryption scheme ("cenc") MUST be used.

7.2.  HESP Manifest

   The HESP Manifest has a "SwitchingSetProtection" structure to set up
   common encryption for audio and video Switching Sets.  The definition
   of this structure can be found in Chapter 3.  As a result of this
   being set on a Switching Set level:

Speelmans               Expires 22 November 2021               [Page 63]
Internet-Draft                    HESP                          May 2021

   *  all Tracks belonging to such a Switching Set MUST be encrypted
      with the same content key.  Aligned Switching Sets can be used to
      ensure that a client can still switch through Tracks of different
      Switching Sets.

   *  audio and video data SHOULD be encrypted with different content
      keys.  This is a recommendation as both often have different
      encryption strength requirements.

   The "SwitchingSetProtection" structure MAY contain a
   "ProtectionSystemSpecificHeaderBox" ("pssh"), which can also be
   contained by the Initialization Stream.  Note that this box MUST be
   given by at least one of both options in order for a license request
   to be made.  If both exist for a certain Switching Set, the "pssh"
   box from the Initialization Stream MUST be disregarded.  This allows
   for easier altering of license information after a stream has been
   created or published.

7.3.  CMAF box structure

   The media of a protected stream needs to contain certain ISOBMFF
   boxes in order to be compliant with CENC (and CMAF).  HESP requires
   the same boxes to be present in media streams.  A brief overview of
   these boxes is given below.  More information on this requirement can
   be found in the CENC [CENC] and CMAF [CMAF] specifications.

7.3.1.  Initialization Stream

   Initialization Packets MUST contain the following boxes:

   *  "TrackEncryptionBox" ("tenc"): This box contains default
      parameters regarding sample encryption.

   *  "SchemeTypeBox" ("schm"): This box identifies the protection
      scheme. "scheme_type" MUST be set to "cbcs" or "cenc".

   *  "ProtectionSystemSpecificHeaderBox" ("pssh"): If the Manifest does
      not contain a "pssh" box that applies to this Track, then this box
      must be included in each Initialization Packet of each Track of
      the protected Switching Set. If the Manifest does contain a "pssh"
      box that applies to this Track, then this box MUST be disregarded.

   In order to signal that the Track is encrypted, the stream type MUST
   equal "encv" for video Tracks and "enca" for audio Tracks.

Speelmans               Expires 22 November 2021               [Page 64]
Internet-Draft                    HESP                          May 2021

   Additionally, video Initialization Packets contain a sample.  If that
   sample is encrypted, then the same requirements for CMAF fragments of
   a Continuation Segment also apply here: a "senc" box MUST be
   included, "pssh", "saiz" and "saio" boxes MAY be included.  The next
   section contains more information about these requirements.

7.3.2.  Continuation Stream

   CMAF fragments of a Continuation Segment MUST contain the following
   box:

   *  "SampleEncryptionBox" ("senc"): This box is used to store
      initialization vector data and information on subsample
      encryption.  As each chunk in HESP must currently contain at most
      one sample, "sample_count" SHALL always 1.

   The following boxes MAY be included:

   *  "ProtectionSystemSpecificHeaderBox" ("pssh"): If any updates need
      to be made to the underlying licensing system, a "pssh" box MAY be
      included.

   *  "SampleAuxiliaryInformationSizesBox" ("saiz"): This box is used to
      store the size of per-sample auxiliary information.  It is only
      REQUIRED if such per-sample information exists.

   *  "SampleAuxiliaryInformationOffsetsBox" ("saio"): This box is used
      to store the offsets of per-sample auxiliary information.  It is
      only REQUIRED if such per-sample information exists.

8.  Contributors

   Significant contributions to the design of this protocol were made by
   Egon Okerman, Samie Beheydt and Johan Vounckx.

9.  IANA Considerations

   This memo requests that the following MIME type [RFC2046] be
   registered with the IANA:

   |  Type name: "application"
   |  
   |  Subtype name: "vnd.theo.hesp+json"
   |  
   |  Required parameters: (none)
   |  
   |  Optional parameters: (none)
   |  

Speelmans               Expires 22 November 2021               [Page 65]
Internet-Draft                    HESP                          May 2021

   |  Encoding considerations: encoded as text.
   |  
   |  Security considerations: See Section 10.
   |  
   |  Compression: this media type does not employ compression.
   |  
   |  Interoperability considerations: There are no byte-ordering
   |  issues, since files are 7- or 8-bit text.  Applications could
   |  encounter unrecognized tags, which SHOULD be ignored.

10.  Security Considerations

   Since the protocol uses HTTP to transmit data, the regular HTTP
   security considerations apply.  See section 15 of RFC 7230 [RFC7230].

   Clients SHOULD take care when parsing files received from a server so
   that non-compliant files are rejected.  Clients SHOULD range-check
   responses to prevent buffer overflows.  See also the Security
   Considerations section of RFC 3986 [RFC3986].  Clients SHOULD load
   resources identified by URI lazily to avoid contributing to denial-
   of-service attacks.

   HTTP requests often include session state ("cookies"), which may
   contain private user data.  Implementations MUST follow cookie
   restriction and expiry rules specified by RFC 6265 [RFC6265].  See
   also the Security Considerations section of RFC 6265, and RFC 2964
   [RFC2964].

   Encryption keys are specified by URI.  The delivery of these keys
   SHOULD be secured by a mechanism such as HTTP over TLS [RFC8446]
   (formerly SSL) in conjunction with a secure realm or a session
   cookie.

11.  Normative References

   [CENC]     International Organization for Standardization,
              "Information technology - MPEG systems technologies - Part
              7: Common encryption in ISO base media file format files",
              December 2020, <https://www.iso.org/standard/68042.html>.

   [CMAF]     International Organization for Standardization,
              "Information technology - Multimedia application format
              (MPEG-A) - Part 19: Common media application format (CMAF)
              for segmented media", March 2020,
              <https://www.iso.org/standard/79106.html>.

Speelmans               Expires 22 November 2021               [Page 66]
Internet-Draft                    HESP                          May 2021

   [ISO6392]  International Organization for Standardization, "Codes for
              the representation of names of languages - Part 2: Alpha-3
              code", November 1998,
              <https://www.iso.org/standard/4767.html>.

   [ISO8601]  International Organization for Standardization, "Date and
              time - Representations for information interchange",
              February 2019, <https://www.iso.org/standard/70907.html>.

   [ISOBMFF]  International Organization for Standardization,
              "Information technology - Coding of audio-visual objects -
              Part 12: ISO base media file format", December 2020,
              <https://www.iso.org/standard/74428.html>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC2964]  Moore, K. and N. Freed, "Use of HTTP State Management",
              BCP 44, RFC 2964, DOI 10.17487/RFC2964, October 2000,
              <https://www.rfc-editor.org/info/rfc2964>.

   [RFC3986]  Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
              Resource Identifier (URI): Generic Syntax", STD 66,
              RFC 3986, DOI 10.17487/RFC3986, January 2005,
              <https://www.rfc-editor.org/info/rfc3986>.

   [RFC6265]  Barth, A., "HTTP State Management Mechanism", RFC 6265,
              DOI 10.17487/RFC6265, April 2011,
              <https://www.rfc-editor.org/info/rfc6265>.

   [RFC6381]  Gellens, R., Singer, D., and P. Frojdh, "The 'Codecs' and
              'Profiles' Parameters for "Bucket" Media Types", RFC 6381,
              DOI 10.17487/RFC6381, August 2011,
              <https://www.rfc-editor.org/info/rfc6381>.

   [RFC7230]  Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer
              Protocol (HTTP/1.1): Message Syntax and Routing",
              RFC 7230, DOI 10.17487/RFC7230, June 2014,
              <https://www.rfc-editor.org/info/rfc7230>.

   [RFC8259]  Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
              Interchange Format", STD 90, RFC 8259,
              DOI 10.17487/RFC8259, December 2017,
              <https://www.rfc-editor.org/info/rfc8259>.

Speelmans               Expires 22 November 2021               [Page 67]
Internet-Draft                    HESP                          May 2021

   [RFC8446]  Rescorla, E., "The Transport Layer Security (TLS) Protocol
              Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018,
              <https://www.rfc-editor.org/info/rfc8446>.

12.  Informative References

   [RFC2046]  Freed, N. and N. Borenstein, "Multipurpose Internet Mail
              Extensions (MIME) Part Two: Media Types", RFC 2046,
              DOI 10.17487/RFC2046, November 1996,
              <https://www.rfc-editor.org/info/rfc2046>.

Appendix A.  Example usage

A.1.  Manifest

   For the initial step, the client retrieves a Manifest.

A.1.1.  Retrieving the Manifest

   The URL of the Manifest is given out of band to the client.  The
   client sends out a GET request.  In this case, let's suppose the
   Manifest is available at "https://example.com/stream1/manifest.json".
   The client then makes such a request:

   GET /stream1/manifest.json HTTP/1.1
   Host: example.com
   Accept: application/vnd.theo.hesp+json

   The server responds with the following headers:

   HTTP/1.1 200 OK
   Content-Type: application/vnd.theo.hesp+json; charset=utf-8
   Content-Length: 6867
   Date: Wed, 31 Mar 2021 08:00:00 GMT

   and the following body:

   {
      "activePresentation":"1",
      "availabilityDuration":{
         "value":1500
      },
      "creationDate":"2021-03-31T08:00:00.000Z",
      "fallbackPollRate":300,
      "manifestVersion":"1.0.0",
      "streamType":"live",
      "presentations":[
         {

Speelmans               Expires 22 November 2021               [Page 68]
Internet-Draft                    HESP                          May 2021

            "id":"0",
            "timeBounds":{
               "startTime":0,
               "endTime":972000000,
               "scale":90000
            },
            "audio":[
               {
                  "id":"main-audio",
                  "language":"eng",
                  "baseUrl":"audio/",
                  "channels":2,
                  "codecs":"mp4a.40.2",
                  "continuationPattern":"content-{segmentId}.mp4",
                  "initializationPattern":"init-{initId}.mp4",
                  "sampleRate":48000,
                  "tracks":[
                     {
                        "id":"96kbps",
                        "activeSegment":1799,
                        "activeSequenceNumber":506249,
                        "averageBandwidth":96000,
                        "bandwidth":96000,
                        "baseUrl":"96k/",
                        "segmentDuration":{
                           "value":540000,
                           "scale":90000
                        },
                        "segments":[
                           {
                              "id":1799,
                              "timeBounds":{
                                 "startTime":971460000,
                                 "scale":90000
                              }
                           }
                        ]
                     }
                  ]
               }
            ],
            "video":[
               {
                  "id":"main-video",
                  "baseUrl":"video/",
                  "frameRate":{
                     "value":25
                  },

Speelmans               Expires 22 November 2021               [Page 69]
Internet-Draft                    HESP                          May 2021

                  "continuationPattern":"content-{segmentId}.mp4",
                  "initializationPattern":"init-{initId}.mp4",
                  "tracks":[
                     {
                        "id":"720p",
                        "activeSegment":1799,
                        "activeSequenceNumber":269999,
                        "bandwidth":3000000,
                        "baseUrl":"720p/",
                        "codecs":"avc1.4d001f",
                        "resolution":{
                           "width":1280,
                           "height":720
                        },
                        "segmentDuration":{
                           "value":540000,
                           "scale":90000
                        },
                        "segments":[
                           {
                              "id":1799,
                              "timeBounds":{
                                 "startTime":971460000,
                                 "scale":90000
                              }
                           }
                        ]
                     }
                  ]
               }
            ]
         },
         {
            "id":"1",
            "timeBounds":{
               "startTime":972000000,
               "scale":90000
            },
            "currentTime":{
               "value":1080000000,
               "scale":90000
            },
            "baseUrl":"https://otherexample.com/s2/",
            "audio":[
               {
                  "id":"main-audio",
                  "language":"eng",
                  "baseUrl":"audio/",

Speelmans               Expires 22 November 2021               [Page 70]
Internet-Draft                    HESP                          May 2021

                  "channels":2,
                  "codecs":"mp4a.40.2",
                  "sampleRate":48000,
                  "mediaTimeOffset":{
                     "value":-972000000,
                     "scale":90000
                  },
                  "tracks":[
                     {
                        "id":"128kbps",
                        "activeSegment":200,
                        "activeSequenceNumber":56250,
                        "averageBandwidth":128000,
                        "bandwidth":128000,
                        "continuationPattern":
                           "128k-content-{segmentId}.mp4",
                        "initializationPattern":
                           "128k-init-{initId}.mp4",
                        "segmentDuration":{
                           "value":540000,
                           "scale":90000
                        },
                        "segments":[
                           {
                              "id":200,
                              "timeBounds":{
                                 "startTime":1080000000,
                                 "scale":90000
                              }
                           }
                        ]
                     }
                  ]
               }
            ],
            "video":[
               {
                  "id":"main-video",
                  "baseUrl":"video/",
                  "frameRate":{
                     "value":25
                  },
                  "tracks":[
                     {
                        "id":"720p",
                        "activeSegment":200,
                        "activeSequenceNumber":29999,
                        "bandwidth":3000000,

Speelmans               Expires 22 November 2021               [Page 71]
Internet-Draft                    HESP                          May 2021

                        "codecs":"avc1.4d001f",
                        "continuationPattern":
                           "720p-content-{segmentId}.mp4",
                        "initializationPattern":
                           "720p-init-{initId}.mp4",
                        "resolution":{
                           "width":1280,
                           "height":720
                        },
                        "segmentDuration":{
                           "value":540000,
                           "scale":90000
                        },
                        "segments":[
                           {
                              "id":200,
                              "timeBounds":{
                                 "startTime":1080000000,
                                 "scale":90000
                              }
                           }
                        ]
                     },
                     {
                        "id":"1080p",
                        "activeSegment":200,
                        "activeSequenceNumber":29999,
                        "bandwidth":5000000,
                        "codecs":"avc1.4d001f",
                        "continuationPattern":
                           "1080p-content-{segmentId}.mp4",
                        "initializationPattern":
                           "1080p-init-{initId}.mp4",
                        "resolution":{
                           "width":1920,
                           "height":1080
                        },
                        "segmentDuration":{
                           "value":540000,
                           "scale":90000
                        },
                        "segments":[
                           {
                              "id":200,
                              "timeBounds":{
                                 "startTime":1080000000,
                                 "scale":90000
                              }

Speelmans               Expires 22 November 2021               [Page 72]
Internet-Draft                    HESP                          May 2021

                           }
                        ]
                     }
                  ]
               }
            ]
         }
      ]
   }

A.1.2.  Timing information

   In the above Manifest, we have two Presentations:

   *  The first Presentation with ID "0" is not active at the current
      time, though it was active previously.  It contains one audio
      Track and one video Track.  It is 3 hours long in total and starts
      at 00:00:00.000 (Manifest Timestamp.)  No Track has a
      mediaTimeOffset defined, so the Manifest Timestamps match the
      Media Timestamps.

   *  The second Presentation with ID "1" is active at the current time.
      It contains one audio Track and two video Tracks.  Its total
      duration is yet to be defined.  Its start time is 03:00:00.000 in
      Manifest Time and its current time (at the moment the manifest was
      retrieved) is 03:20:00.000.  All Tracks have the same
      mediaTimeOffset defined.  The Media Timestamps for this
      Presentation start at 00:00:00.000.  That also means that for all
      the currently active Segments, the media data will contain a
      starting timestamp of 00:20:00.000.

   The availabilityDuration of the Manifest is 25 minutes.  As only 20
   minutes of the second Presentation have elapsed, some Segments of the
   first Presentation are still available.  Therefore, the first
   Presentation must still be included in the Manifest.  Once 5 minutes
   have passed, the first Presentation can be left out of the Manifest.

A.1.3.  Content addressing

   The client uses the Manifest to derive the content request URLs of
   each Track.

   For the audio Track of the first Presentation, the following parts
   are found:

Speelmans               Expires 22 November 2021               [Page 73]
Internet-Draft                    HESP                          May 2021

   AudioSwitchingSet.baseUrl = "audio/"
   AudioTrack.baseUrl = "96k/"
   AudioSwitchingSet.initializationPattern = "init-{initId}.mp4"
   AudioSwitchingSet.continuationPattern = "content-{segmentId}.mp4"

   The Manifest URL ("https://example.com/stream1/manifest.json") is
   used as the base for resolution, then relative resolution is applied
   to each of the parts above.

   T = resolve(AudioSwitchingSet.baseUrl, manifestUrl)
     = "https://example.com/stream1/audio/"
   T = resolve(AudioTrack.baseUrl, T)
     = "https://example.com/stream1/audio/96k/"
   initBaseUrl = resolve(AudioSwitchingSet.initializationPattern, T)
     = "https://example.com/stream1/audio/96k/init-{initId}.mp4"
   contBaseUrl = resolve(AudioSwitchingSet.continuationPattern, T)
     = "https://example.com/stream1/audio/96k/content-{segmentId}.mp4"

   The final content request URLs are
   "https://example.com/stream1/audio/96k/init-{initId}.mp4" and
   "https://example.com/stream1/audio/96k/content-{segmentId}.mp4" for
   the Initialization Stream and Continuation Stream respectively.

   For the audio Track of the second Presentation, the following parts
   are found:

   Presentation.baseUrl = "https://otherexample.com/s2/"
   AudioSwitchingSet.baseUrl = "audio/"
   AudioTrack.initializationPattern = "128k-init-{initId}.mp4"
   AudioTrack.continuationPattern = "128k-content-{segmentId}.mp4"

   The Manifest URL ("https://example.com/stream1/manifest.json") is
   used as the base for resolution, then relative resolution is applied
   to each of the parts above.

   T = resolve(Presentation.baseUrl, manifestUrl)
     = "https://otherexample.com/s2/"
   T = resolve(AudioSwitchingSet.baseUrl, T)
     = "https://otherexample.com/s2/audio/"
   initBaseUrl = resolve(AudioTrack.initializationPattern, T)
     = "https://otherexample.com/s2/audio/128k-init-{initId}.mp4"
   contBaseUrl = resolve(AudioTrack.continuationPattern, T)
     = "https://otherexample.com/s2/audio/128k-content-{segmentId}.mp4"

   The final content request URLs are "https://otherexample.com/s2/
   audio/128k-init-{initId}.mp4" and "https://otherexample.com/s2/
   audio/128k-content-{segmentId}.mp4" for the Initialization Stream and
   Continuation Stream respectively.

Speelmans               Expires 22 November 2021               [Page 74]
Internet-Draft                    HESP                          May 2021

A.2.  Initialization Stream

   In the second step, the client uses the Manifest to retrieve and then
   parse an Initialization Packet.

A.2.1.  Retrieving Initialization Packets

   The client decides to retrieve the only audio Track of the active
   Presentation.  The URL pattern for the Initialization Stream of this
   Track is "https://otherexample.com/s2/audio/128k-init-{initId}.mp4".

   The client opts to retrieve the most recent Initialization Packet and
   sends out a request:

   GET /s2/128k-init-now.mp4 HTTP/1.1
   Host: otherexample.com
   Accept: */*

   The server responds with the following headers

   HTTP/1.1 200 OK
   Content-Type: audio/mp4
   Content-Length: 742
   Date: Wed, 31 Mar 2021 08:00:02 GMT

   and a binary body containing the ISOBMFF media data of the audio
   Initialization Packet.

   The client repeats this to retrieve one of the video Tracks.

A.2.2.  Parsing offset information

   The client parses the ISOBMFF boxes in the audio Initialization
   Packet.  Once the "emsg" box with schemeId "urn:theo:hesp:2020" and
   value "initdata" is found, its "message_data" field is parsed.

   In this case, "message_data" equals "{"index":200,"offset":63275}".
   The following media data for the chosen audio Track is available in
   the Continuation Segment with ID 200 and at a byte offset of 63275.

A.3.  Continuation Stream

   In the final step, the client uses the already retrieved information
   to retrieve Continuation Segments and reaches playback.

Speelmans               Expires 22 November 2021               [Page 75]
Internet-Draft                    HESP                          May 2021

A.3.1.  Retrieving Continuation Segments

   The client retrieves the Continuation Segment of the chosen audio
   Track.  The URL pattern for the Continuation Stream of this Track is
   "https://otherexample.com/s2/audio/128k-content-{segmentId}.mp4".

   The client sends out a request for the segment with ID 200, starting
   at offset 63275:

   GET /s2/128k-content-200.mp4 HTTP/1.1
   Host: otherexample.com
   Accept: */*
   Range: bytes=63275-9007199254740991

   As discussed in Section 5.3.2, a sufficiently large end byte value is
   chosen to ensure the full range is retrieved.

   The server responds with the following headers

   HTTP/1.1 200 OK
   Content-Type: audio/mp4
   Content-Range: bytes 63275-9007199254740991/9007199254677716
   Transfer-Encoding: chunked
   Date: Wed, 31 Mar 2021 08:00:03 GMT

   and a (chunked) binary body containing the ISOBMFF media data of the
   Continuation Segment.

   The client repeats this for the chosen video Track and uses the
   retrieved media data to reach playback.  The Manifest that was
   retrieved previously contains sufficient information to retrieve new
   Continuation Segments from this point on.

Appendix B.  CDNs

   A Content Delivery Network (CDN) MAY be used to increase the
   scalability and cacheability for delivering HESP.

   While the HESP protocol makes use of HTTP/1.1 and delivery should be
   possible over most HTTP CDNs, care must be taken to ensure that the
   CDN has all the required features.

   To correctly handle HESP, the CDN MUST either support HTTP/1.1 with
   chunked transfer encoding or support HTTP/2.  It MUST also support
   Range Requests as well as the caching of the partial object
   responses.  This ensures that HESP requests and responses pass
   correctly through the CDN and that responses can be cached for future
   use.

Speelmans               Expires 22 November 2021               [Page 76]
Internet-Draft                    HESP                          May 2021

   The CDN SHOULD support collapsing multiple HTTP/1.1 Range Requests
   when they have overlapping byte-ranges into a single request.  This
   ensures that two requests with byte ranges which are partially
   overlapping only require a single request to the media server for the
   overlapping part.  This reduces the number of concurrent requests
   arriving on the media server.

Appendix C.  HESP Profiles (using H.264 as video codec)

   In this annex, we describe two possible profiles for video Tracks of
   HESP streams.  Both profiles place certain requirements on the
   underlying H.264 encoding of the HESP stream.

C.1.  Maximal Gain Profile

   The goal of this profile is to allow the stream to reach the lowest
   latency and zapping times possible using the HESP protocol.  In this
   profile, it should be ensured that an Initialization Packet exists
   for each frame of the Continuation Stream.  As a result, a client can
   start playback, seek, switch between Tracks,... at any time position
   of the stream.

   The Initialization Stream for Tracks of the Maximal Gain Profile must
   satisfy the following requirements:

   *  The frame rate of the Initialization Stream MUST match the
      Continuation Stream.

   *  Each media sample of the Initialization Stream MUST be independent
      (i.e. an I frame in H.264) and individually addressable (the
      latter is currently always true for HESP.)

   The Continuation Stream for Tracks of the Maximal Gain Profile must
   satisfy the following requirements:

   *  Each media sample of the Continuation Stream MUST be either
      independent (i.e. an I frame in H.264) or dependent on the media
      sample directly preceding it in decode order (i.e. a P frame
      referencing only the previous frame.)

   *  Each CMAF Fragment of the Continuation Segment MUST only contain
      one media sample.

   *  Each Continuation Segment SHOULD be significantly long (values of
      multiple minutes are possible and encouraged.)

Speelmans               Expires 22 November 2021               [Page 77]
Internet-Draft                    HESP                          May 2021

                  +----+----+----+----+----+----+----+----+----+----+
   time positions | 00 | 01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 |
                  +----+----+----+----+----+----+----+----+----+----+

                  +----+----+----+----+----+----+----+----+----+----+
   initialization | IDR| IDR| IDR| IDR| IDR| IDR| IDR| IDR| IDR| IDR|
   stream         | 00 | 01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 |
                  +----+----+----+----+----+----+----+----+----+----+
                  | REF| REF| REF| REF| REF| REF| REF| REF| REF| REF|
                  |  a |  b |  c |  d |  e |  f |  g |  h |  i |  j |
                  +----+----+----+----+----+----+----+----+----+----+

                  +----+----+----+----+----+----+----+----+----+----+
   continuation   | I,P| I,P| I,P| I,P| I,P| I,P| I,P| I,P| I,P| I,P|
   stream         | 00 | 01 | 02 | 03 | 04 | 05 | 06 | 07 | 08 | 09 |
                  +----+----+----+----+----+----+----+----+----+----+
                       |    |    |
                       |    |     \___ position c
                       |    |
                       |     \___ position b
                       |
                        \___ position a

                       Figure 4: Maximum Gain Profile

C.2.  Compatibility Profile

   The goal of this profile is to allow media data from other HTTP-based
   adaptive bitrate protocols to be reused.  This comes at the cost of
   some optimizations made by the previous profile.  In this profile, it
   is not possible to start playback at any time position of the stream,
   as it is not possible for Initialization Packets to refer to every
   sample of a Continuation Segment.

   The Initialization Stream for Tracks of the Compatibility Profile
   must satisfy the following requirements:

   *  Each media sample of the Initialization Stream MUST be independent
      (i.e. an I frame in H.264) and individually addressable (the
      latter is currently always true for HESP.)

Speelmans               Expires 22 November 2021               [Page 78]
Internet-Draft                    HESP                          May 2021

   *  Initialization Packets MUST contain a reference to the subsequent
      sample in the Continuation Segment, where this subsequent sample
      MUST be either independent (i.e. an I frame in H.264) or dependent
      on the media sample directly preceding it in decode order (i.e. a
      P frame referencing only the previous frame.)  If the subsequent
      sample does not meet this constraint, then this Initialization
      Packet MUST NOT be published.  Instead, the last valid
      Initialization Packet MUST be returned if this Sequence Number is
      queried.

   The Continuation Stream for Tracks of the Compatibility Profile must
   satisfy the following requirements:

   *  Continuation Segments SHOULD NOT exceed a duration of 30 seconds.

   *  Each CMAF Fragment of the Continuation Segment MUST contain at
      most the amount of media samples of a sub GOP (defined below.)

   *  The following GOP structure must be followed in the underlying
      H.264 stream: I, B (repeated n times), P, B (repeated n times), P,
      where n lies between 0 and 4.

   *  Each sub GOP (B ... B P) MUST depend on at most one previous frame
      (allowing for keyframe insertion.)

C.2.1.  Example

   Figure 5 depicts a part of H.264 output of 7 frames, sorted in decode
   order.  The dependencies of each frame are shown with arrows.

                      ______________________________________
                      |                     |       |       |
                      V                     |       |       |
          +----+  +----+  +----+  +----+  +----+  +----+  +----+
          |####|  |%%%%|  |%%%%|  |%%%%|  |@@@@|  |@@@@|  |@@@@|
          | I1 |  | P4 |  | B2 |  | B3 |  | P7 |  | B5 |  | B6 |
          +----+  +----+  +----+  +----+  +----+  +----+  +----+
              ^     |  ^     |       |        ^      |       |
              |_____|  |     |       |        |______|       |
              |        |     |       |        |              |
              |________|_____|       |        |______________|
              |                      |
              |______________________|

                Figure 5: Continuation stream with sub GOPs

Speelmans               Expires 22 November 2021               [Page 79]
Internet-Draft                    HESP                          May 2021

C.2.1.1.  Sub GOPs

   A "sub GOP" defines a set of B and P frames that only depend on one
   previous frame.

   In Figure 5, there are 3 sub GOPs:

   *  sub GOP 1 ("####") contains only a single I frame.

   *  sub GOP 2 ("%%%%") contains a P frame (P4) that only depends on
      I1, all B frames depend on I1 and P4.

   *  sub GOP 3 ("@@@@") contains a P frame (P7) that only depends on
      P4, all B frames depend on P7 and P4.

C.2.1.2.  Initialization Packets

   An Initialization Packet can be published if the subsequent media
   sample of the Continuation Stream depends on at most one previous
   frame.

   For Figure 5, this is the case at the following positions:

   *  position 1: an Initialization Packet can be published.  It will
      contain IDR1 (a keyframe matching the timestamp of I1) and will
      reference P4, the subsequent media sample of the Continuation
      Stream that only depends on I1.  On the client side, the media
      data will be decoded with IDR1 inserted at the location of I1.

   *  position 4: an Initialization Packet can be published.  It will
      contain IDR4 (a keyframe matching the timestamp of P4) and will
      reference P7, the subsequent media sample of the Continuation
      Stream that only depends on P4.  On the client side, the media
      data will be decoded with IDR4 inserted at the location of P4.

   A client requesting an Initialization Packet at other time positions
   must receive the most recent valid Initialization Packet.  That means
   that a request for an Initialization Packet at position 2 in Figure 5
   must return the Initialization Packet at position 1.

Author's Address

   Pieter-Jan Speelmans (editor)
   THEO Technologies
   Leuven
   Belgium

   Email: pieter-jan.speelmans@theoplayer.com

Speelmans               Expires 22 November 2021               [Page 80]