INTERNET-DRAFT J. Lazzaro
June 22, 2004 J. Wawrzynek
Expires: December 22, 2004 UC Berkeley
RTP Payload Format for MIDI
<draft-ietf-avt-rtp-midi-format-05.txt>
Status of this Memo
By submitting this Internet-Draft, I certify that any applicable
patent or other IPR claims of which I am aware have been disclosed,
and any of which I become aware will be disclosed, in accordance with
RFC 3668.
By submitting this Internet-Draft, I accept the provisions of Section 3
of RFC 3667 (BCP 78).
Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF), its areas, and its working groups. Note that other groups
may also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference material
or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at http://
www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on December 22, 2004.
Copyright Notice
Copyright (C) The Internet Society (2004). All Rights Reserved.
Lazzaro/Wawrzynek [Page 1]
INTERNET-DRAFT 22 June 2004
Abstract
This memo describes an RTP payload format for the MIDI command
language. The format encodes all commands that may legally appear
on a MIDI 1.0 DIN cable. The format is suitable for interactive
applications (such as the remote operation of musical instruments)
and content-delivery applications (such as file streaming). The
format may be used over unicast and multicast UDP as well as TCP,
and defines tools for graceful recovery from packet loss. Stream
behavior, including the MIDI rendering method, may be customized
during session setup. The format also serves as a mode for the
mpeg4-generic format, to support the MPEG 4 Audio Object Types for
General MIDI, Downloadable Sounds Level 2, and Structured Audio.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 6
2. Packet Format . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1 RTP Header . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 MIDI Payload . . . . . . . . . . . . . . . . . . . . . . . 8
3. MIDI Command Section . . . . . . . . . . . . . . . . . . . . . . 10
3.1 Timestamps . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Command Coding . . . . . . . . . . . . . . . . . . . . . . 12
4. The Recovery Journal System . . . . . . . . . . . . . . . . . . . 18
5. Recovery Journal Format . . . . . . . . . . . . . . . . . . . . . 20
6. Session Description Protocol . . . . . . . . . . . . . . . . . . 23
6.1 Session Descriptions for Native Streams . . . . . . . . . . 24
6.2 Session Descriptions for mpeg4-generic Streams . . . . . . 26
6.3 Session Configuration Tools . . . . . . . . . . . . . . . . 28
7. Extensibility . . . . . . . . . . . . . . . . . . . . . . . . . . 29
8. Congestion Control . . . . . . . . . . . . . . . . . . . . . . . 30
A. The Recovery Journal Channel Chapters . . . . . . . . . . . . . . 31
A.1 Recovery Journal Definitions . . . . . . . . . . . . . . . 31
A.2 Chapter P: MIDI Program Change . . . . . . . . . . . . . . 35
A.3 Chapter C: MIDI Control Change . . . . . . . . . . . . . . 36
A.3.1 Log Inclusion Rules . . . . . . . . . . . . . . . . 36
A.3.2 Controller Log Format . . . . . . . . . . . . . . . 38
A.3.3 Log List Coding Rules . . . . . . . . . . . . . . . 39
A.3.4 The Parameter System . . . . . . . . . . . . . . . . 40
A.4 Chapter M: MIDI Parameter System . . . . . . . . . . . . . 42
A.4.1 Log Inclusion Rules . . . . . . . . . . . . . . . . 43
A.4.2 Log Coding Rules . . . . . . . . . . . . . . . . . . 44
A.4.2.1 The Value Tool . . . . . . . . . . . . . . . 46
A.4.2.2 The Count Tool . . . . . . . . . . . . . . . 49
A.5 Chapter W: MIDI Pitch Wheel . . . . . . . . . . . . . . . . 50
Lazzaro/Wawrzynek [Page 2]
INTERNET-DRAFT 22 June 2004
A.6 Chapter N: MIDI NoteOff and NoteOn . . . . . . . . . . . . 51
A.6.1 Header Structure . . . . . . . . . . . . . . . . . . 52
A.6.2 Note Structures . . . . . . . . . . . . . . . . . . 53
A.7 Chapter E: MIDI Note Command Extras . . . . . . . . . . . . 54
A.7.1 Note Log Format . . . . . . . . . . . . . . . . . . 55
A.7.2 Log Inclusion Rules . . . . . . . . . . . . . . . . 55
A.8 Chapter T: MIDI Channel Aftertouch . . . . . . . . . . . . 56
A.9 Chapter A: MIDI Poly Aftertouch . . . . . . . . . . . . . . 57
B. The Recovery Journal System Chapters . . . . . . . . . . . . . . 58
B.1 System Chapter D: Simple System Commands . . . . . . . . . 58
B.1.1 Undefined System Commands . . . . . . . . . . . 59
B.2 System Chapter V: Active Sense Command . . . . . . . . . . 62
B.3 System Chapter Q: Sequencer State Commands . . . . . . . . 62
B.3.1 Non-compliant Sequencers . . . . . . . . . . . 64
B.4 System Chapter F: MIDI Time Code . . . . . . . . . . . . . 65
B.4.1 Partial Frames . . . . . . . . . . . . . . . . . . 67
B.5 System Chapter X: System Exclusive . . . . . . . . . . . . 68
B.5.1 Chapter Format . . . . . . . . . . . . . . . . 69
B.5.2 Log Inclusion Semantics . . . . . . . . . . . . 71
B.5.3 TCOUNT and COUNT fields . . . . . . . . . . . . 73
C. SDP Session Configuration Tools . . . . . . . . . . . . . . . . . 74
C.1 The Journalling System . . . . . . . . . . . . . . . . . . 75
C.1.1 The j_sec Parameter . . . . . . . . . . . . . . . . 75
C.1.2 The j_update Parameter . . . . . . . . . . . . . . . 76
C.1.2.1 The anchor Sending Policy . . . . . . . . . . 77
C.1.2.2 The closed-loop Sending Policy . . . . . . . 77
C.1.2.3 The open-loop Sending Policy . . . . . . . . 80
C.1.3 Chapter Inclusion Parameters . . . . . . . . . . . . 82
C.2 Command Execution Semantics . . . . . . . . . . . . . . . . 88
C.2.1 The async Algorithm . . . . . . . . . . . . . . . . 88
C.2.2 The buffer Algorithm . . . . . . . . . . . . . . . . 89
C.3 Packet Timing Tools . . . . . . . . . . . . . . . . . . . . 91
C.3.1 ptime and maxptime . . . . . . . . . . . . . . . . . 91
C.3.2 The guardtime Parameter . . . . . . . . . . . . . . 92
C.3.3 MIDI Time Code Issues . . . . . . . . . . . . . . . 93
C.4 Multiple Streams . . . . . . . . . . . . . . . . . . . . . 94
C.4.1 The musicport Parameter . . . . . . . . . . . . . . 94
C.4.2 The zerosync Parameter . . . . . . . . . . . . . . . 96
C.5 MIDI Rendering . . . . . . . . . . . . . . . . . . . . . . 100
C.5.1 The multimode Parameter . . . . . . . . . . . . . . 101
C.5.2 The dirsense Parameter . . . . . . . . . . . . . . . 101
C.5.3 The rinit Parameter . . . . . . . . . . . . . . . . 102
C.5.4 Encoding rinit Data Objects . . . . . . . . . . . . 104
C.5.5 MIDI Channel Mapping . . . . . . . . . . . . . . . . 104
C.5.5.1 smf_info . . . . . . . . . . . . . . . . . . 105
C.5.5.2 smf_inline, smf_url, smf_cid . . . . . . . . 106
C.5.5.3 chanmask . . . . . . . . . . . . . . . . . . 107
C.5.6 The audio/asc MIME Type . . . . . . . . . . . . . . 108
Lazzaro/Wawrzynek [Page 3]
INTERNET-DRAFT 22 June 2004
C.6 Interoperability . . . . . . . . . . . . . . . . . . . . . 110
C.6.1 Transport and Recovery Journal . . . . . . . . . . . 112
C.6.2 Command Timestamp Semantics . . . . . . . . . . . . 114
C.6.3 Packet Timing Tools . . . . . . . . . . . . . . . . 115
C.6.4 Multiple Streams . . . . . . . . . . . . . . . . . . 116
C.6.5 MIDI Rendering . . . . . . . . . . . . . . . . . . . 117
D. Parameter Syntax Definitions . . . . . . . . . . . . . . . . . . 120
E. A MIDI Overview for Networking Specialists . . . . . . . . . . . 125
E.1 Commands Types . . . . . . . . . . . . . . . . . . . . . . 126
E.2 Running Status . . . . . . . . . . . . . . . . . . . . . . 126
E.3 Command Timing . . . . . . . . . . . . . . . . . . . . . . 126
F. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 127
G. Security Considerations . . . . . . . . . . . . . . . . . . . . . 128
H. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . . 129
H.1 rtp-midi MIME Registration . . . . . . . . . . . . . . . . 129
H.2 mpeg4-generic MIME Registration . . . . . . . . . . . . . . 132
H.3 asc MIME Registration . . . . . . . . . . . . . . . . . . . 134
I. References . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
I.1 Normative References . . . . . . . . . . . . . . . . . . . 136
I.2 Informative References . . . . . . . . . . . . . . . . . . 137
J. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 138
K. Intellectual Property Rights Statement . . . . . . . . . . . . . 138
L. Full Copyright Statement . . . . . . . . . . . . . . . . . . . . 139
M. Change Log for <draft-ietf-avt-rtp-midi-format-05.txt> . . . . . 140
Lazzaro/Wawrzynek [Page 4]
INTERNET-DRAFT 22 June 2004
1. Introduction
The Internet Engineering Task Force (IETF) has developed a set of
focused tools for multimedia networking ([2] [6] [16] [17]). These
tools can be combined in different ways to support a variety of real-
time applications over Internet Protocol (IP) networks.
For example, a telephony application might use the Session Initiation
Protocol (SIP, [16]) to set up a phone call. Call setup would include
negotiations to agree on a common audio codec [13]. Negotiations would
use the Session Description Protocol (SDP, [6]) to describe candidate
codecs.
After a call is set up, audio data would flow between the parties using
the Real Time Protocol (RTP, [2]) under the Audio/Visual Profile (AVP,
[3]). The tools used in this telephony example (SIP, SDP, RTP/AVP)
might be combined in a different way to support a content streaming
application, perhaps in conjunction with other tools (such as the Real
Time Streaming Protocol (RTSP, [17])).
The MIDI command language [1] is widely used in musical applications
that are analogous to the examples described above. On stage and in the
recording studio, MIDI is used for the interactive remote control of
musical instruments, an application similar in spirit to telephony. On
web pages, Standard MIDI Files (SMFs, [1]) rendered using the General
MIDI standard [1] provide a low-bandwidth substitute for audio
streaming.
This memo is motivated by a simple premise: if MIDI performances could
be sent as RTP streams that are managed by IETF session tools, a
hybridization of the MIDI and IETF application domains may occur.
For example, interoperable MIDI networking may foster network music
performance applications, in which a group of musicians, located at
different physical locations, interact over a network to perform as they
would if located in the same room [14]. As another example, the
streaming community may begin to use MIDI for low-bitrate audio coding,
perhaps in conjunction with normative sound synthesis methods [5]. As
another example, manufacturers of professional audio equipment and
electronic musical instruments may consider adopting the IETF multimedia
stack (IP, SIP, RTP) as the networking layer for a MIDI control plane.
To provide a foundation for RTP MIDI applications, this memo extends two
of the IETF tools (RTP and SDP) to support MIDI. Sections 2-5 and
Appendices A-B extend RTP/AVP by adding a MIDI payload format. Section
6 and Appendices C-D extend SDP by adding session configuration tools to
customize the stream behavior (including the MIDI rendering method)
during session setup.
Lazzaro/Wawrzynek [Page 5]
INTERNET-DRAFT 22 June 2004
Some applications may require MIDI media delivery at a certain service
quality level (latency, jitter, packet loss, etc). RTP itself does not
provide service guarantees. However, applications may use lower-layer
network protocols to configure the quality of the transport services
that RTP uses. These protocols may act to reserve network resources for
RTP flows [20], or may simply direct RTP traffic onto a dedicated "media
network" in a local installation. Note that RTP and the MIDI payload
format do provide tools that applications may use to achieve the best
possible real-time performance at a given service level.
This memo normatively defines the syntax and semantics of the MIDI
payload format. However, this memo does not define algorithms for
sending and receiving packets. An ancillary document [19] provides
informative guidance on algorithms. Supplemental information may be
found in related conference publications [14] [15].
Throughout this memo, the phrase "native stream" refers to a stream that
uses the rtp-midi MIME type. The phrase "mpeg4-generic stream" refers
to a stream that uses the mpeg4-generic MIME type (in mode rtp-midi) to
operate in an MPEG 4 environment [4]. Section 6 describes this
distinction in detail.
1.1 Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119 [11].
2. Packet Format
In this section, we introduce the format of RTP MIDI packets. The
description includes some background information on RTP/AVP, for the
benefit of MIDI implementors new to IETF tools. Implementors should
consult [2,3] for an authoritative description of RTP/AVP.
This memo assumes the reader is familiar with MIDI syntax and semantics.
Appendix E provides a MIDI overview, at a level of detail sufficient to
understand most of this memo. Implementors should consult [1] for an
authoritative description of MIDI.
The MIDI payload format maps a MIDI command stream (16 voice channels +
systems) onto an RTP stream. An RTP media stream is a sequence of
logical packets that share a common format. Each packet consists of two
parts: the RTP header and the MIDI payload. Figure 1 shows this format
(vertical space delineates the header and payload).
Lazzaro/Wawrzynek [Page 6]
INTERNET-DRAFT 22 June 2004
We describe RTP packets as "logical" packets to highlight the fact that
RTP itself is not a network-layer protocol. Instead, RTP packets are
mapped onto network protocols (such as unicast UDP, multicast UDP, or
TCP) by an application [18]. The interleaved mode of the Real Time
Streaming Protocol (RTSP, [17]) is an example of an RTP mapping to TCP
transport, as is [22].
2.1 RTP Header
[2] provides a complete description of the RTP header fields. In this
section, we clarify the role of a few RTP header fields for MIDI
applications. All fields are coded in network byte order (big-endian).
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| V |P|X| CC |M| PT | Sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MIDI command section ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Journal section ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1 -- Packet format
The behavior of the 1-bit M field depends on the MIME type of the
stream. For native streams, the M bit MUST be set to 1 if the MIDI
command section has a non-zero LEN field, and MUST be set to 0
otherwise. For mpeg4-generic streams, the M bit MUST be set to 1 for
all packets (to conform to [4]).
The 16-bit sequence number field is initialized to a randomly chosen
value, and is incremented by one (modulo 2^16) for each packet sent in
the stream. A related quantity, the 32-bit extended packet sequence
number, may be computed by tracking rollovers of the 16-bit sequence
number. Note that different receivers of the same stream may compute
different extended packet sequence numbers, depending on when the
receiver joined the session.
Lazzaro/Wawrzynek [Page 7]
INTERNET-DRAFT 22 June 2004
The 32-bit timestamp field sets the base timestamp value for the packet.
The payload codes MIDI command timing relative to this value. The
timestamp units are set during session configuration by the clock rate
parameter of the rtpmap attribute (see Section 6.1). For example, if
the clock rate has a value of 44100 Hz, two packets whose base timestamp
values differ by 2 seconds have RTP timestamp fields that differ by
88200. By default the timestamp field is initialized to a randomly
chosen value (see Appendix C.4.2 for an exception).
Timestamps of consecutive packets do not necessarily increment at a
fixed rate, because RTP MIDI packets are not necessarily sent at a fixed
rate. The degree of packet transmission regularity reflects the
underlying application dynamics. Interactive applications may vary the
packet sending rate to track the gestural rate of a human performer,
whereas content-streaming applications may send packets at a fixed rate.
Therefore, the timestamps for two sequential RTP packets may be
identical, or the second packet may have a timestamp arbitrarily larger
than the first packet (modulo 2^32). Section 3 places additional
restrictions on the RTP timestamps for two sequential RTP packets, as
does the guardtime MIME parameter (Appendix C.3.2).
The media time coded by a packet is computed by subtracting the last
command timestamp in the MIDI command section from the RTP timestamp
(modulo 2^32). If the MIDI list of the MIDI command section of a packet
is empty, the media time coded by the packet is 0 ms. Appendix C.3.1
discusses media time issues in detail.
2.2 MIDI Payload
The payload (Figure 1) MUST begin with the MIDI command section. The
MIDI command section codes a (possibly empty) list of timestamped MIDI
commands, and provides the essential service of the payload format.
The payload MAY also contain a journal section. The journal section
provides resiliency by coding the recent history of the stream. A flag
in the MIDI command section codes the presence of a journal section in
the payload.
Section 3 defines the MIDI command section. Sections 4-5 and Appendices
A-B define the recovery journal, the default format for the journal
section. Here, we describe how these payload sections operate in a
stream.
The journalling method for a stream is set at the start of a session and
MUST NOT be changed thereafter. A stream may be set to use the recovery
journal, to use an alternative journal format (none are defined in this
memo), or to not use a journal.
Lazzaro/Wawrzynek [Page 8]
INTERNET-DRAFT 22 June 2004
The default journalling method of a stream is inferred from its
transport type. Streams that use unreliable transport (such as UDP)
default to using the recovery journal. Streams that use reliable
transport (such as TCP) default to not using a journal. Appendix C.1.1
defines session configuration tools for overriding these defaults. For
all types of transport, a sender MUST transmit an RTP packet stream with
consecutive sequence numbers (modulo 2^16).
If a stream uses the recovery journal, every payload in the stream MUST
include a journal section. If a stream does not use journalling, a
journal section MUST NOT appear in a stream payload. If a stream uses
an alternative journal format, the specification for the journal format
defines an inclusion policy.
The payload of a stream encodes data for a single MIDI command name
space (16 voice channels + systems). Applications may use several
streams in a session. Session configuration tools for multi-stream
sessions are defined in Appendix C.4.
In some applications, a receiver renders MIDI commands into audio (or
into control actions, such as the rewind of a tape deck or the dimming
of stage lights). In other applications, a receiver presents a MIDI
stream to software programs via an Application Programmer Interface
(API). Appendix C.5 defines session configuration tools to specify what
receivers should do with a MIDI command stream.
If a stream is sent over UDP transport, the Maximum Transmission Unit
(MTU) of the underlying network limits the practical size of the payload
section (for example, an Ethernet MTU is 1500 octets). Note that MTU
size restrictions do not apply to RTP packets sent over TCP streams.
The session configuration tools defined in Appendix C.4 may be used to
split a dense MIDI name space into several UDP streams, so that the
payload fits comfortably into an MTU.
Lazzaro/Wawrzynek [Page 9]
INTERNET-DRAFT 22 June 2004
3. MIDI Command Section
Figure 2 shows the format of the MIDI command section.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|B|J|Z|P|LEN... | MIDI list ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2 -- MIDI command section
The MIDI command section begins with a variable-length header.
The header field LEN codes the number of octets in the MIDI list that
follows the header. If the header flag B is 0, the header is one octet
long, and LEN is a 4-bit field, supporting a maximum MIDI list length of
15 octets. If B is 1, the header is two octets long, and LEN is a
12-bit field, supporting a maximum MIDI list length of 4095 octets. A
LEN value of 0 is legal, and codes an empty MIDI list
If the J header bit is set to 1, a journal section MUST appear after
MIDI command section in the payload. If the J header bit is set to 0,
the payload MUST NOT contain a journal section.
We define the semantics of the P header bit in Section 3.2.
If the LEN header field is nonzero, the MIDI list has the structure
shown in Figure 3.
Lazzaro/Wawrzynek [Page 10]
INTERNET-DRAFT 22 June 2004
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Delta Time 0 (if Z = 1) | MIDI Command 0 ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Delta Time 1 ... | MIDI Command 1 ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Delta Time 2 ... | MIDI Command 2 ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ..... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Delta Time N ... | MIDI Command N (may be empty) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3 -- MIDI list structure
If the header flag Z is 1, the MIDI list begins with a complete MIDI
command (MIDI Command 0) preceded by a delta time (Delta Time 0). If Z
is 0, the Delta Time 0 field is not present in the MIDI list, and MIDI
Command 0 has an implicit delta time of 0. The MIDI list structure may
also optionally encode a list of N additional complete MIDI commands.
Each additional command MUST be preceded by a delta time. We discuss
exceptions to the "command fields code complete MIDI commands" rule in
Section 3.2.
The final MIDI Command field in the MIDI list MAY be empty. Senders may
use this feature to precisely set the media time of a packet.
3.1 Timestamps
The RTP MIDI delta time syntax is a modified form of the MIDI File delta
time syntax [1]. RTP MIDI delta times use 1-4 octet fields to encode
32-bit unsigned integers. Figure 4 shows the encoded and decoded forms
of delta times. Note that delta time values may be legally encoded in
multiple formats; for example, there are four legal ways to encode the
zero delta time (0x00, 0x8000, 0x800000, 0x80000000).
RTP MIDI uses delta times to encode a timestamp for each MIDI command.
The timestamp for MIDI Command K is the summation (modulo 2^32) of the
RTP timestamp and decoded delta times 0 through K. This cumulative
coding technique, borrowed from MIDI File delta time coding, is
efficient because it reduces the number of multi-octet delta times.
All command timestamps in a packet MUST be less than or equal to the RTP
timestamp of the next packet in the stream (modulo 2^32).
Lazzaro/Wawrzynek [Page 11]
INTERNET-DRAFT 22 June 2004
By default, a command timestamp indicates the execution time for the
command. The difference between two timestamps indicates the time delay
between the execution of the commands. This difference may be zero,
coding simultaneous execution.
This interpretation of timestamps works well for transcoding a Standard
MIDI File (SMF) into an RTP MIDI stream, as SMFs code a timestamp for
each MIDI command stored in the file. To transcode an SMF that uses
metric time markers, use the SMF tempo map (encoded in the SMF as meta-
events) to convert metric SMF timestamp units into seconds-based RTP
timestamp units.
Other interpretations of timestamps may work better for transcoding a
MIDI source that uses implicit command timing (such as MIDI 1.0 DIN
cables) into an RTP MIDI stream. Appendix C.2 describes session
configuration tools for selecting the optimal timestamp interpretation
for a stream.
One-Octet Delta Time:
Encoded form: 0ddddddd
Decoded form: 00000000 00000000 00000000 0ddddddd
Two-Octet Delta Time:
Encoded form: 1ccccccc 0ddddddd
Decoded form: 00000000 00000000 00cccccc cddddddd
Three-Octet Delta Time:
Encoded form: 1bbbbbbb 1ccccccc 0ddddddd
Decoded form: 00000000 000bbbbb bbcccccc cddddddd
Four-Octet Delta Time:
Encoded form: 1aaaaaaa 1bbbbbbb 1ccccccc 0ddddddd
Decoded form: 0000aaaa aaabbbbb bbcccccc cddddddd
Figure 4 -- Decoding delta time formats
3.2 Command Coding
Each non-empty MIDI Command field in the MIDI list codes one of the MIDI
command types that may legally appear on a MIDI 1.0 DIN cable. Standard
MIDI File meta-events do not fit this definition and MUST NOT appear in
Lazzaro/Wawrzynek [Page 12]
INTERNET-DRAFT 22 June 2004
the MIDI list. As a rule, each MIDI Command field codes a complete
command, in the binary command format defined in [1]. In the remainder
of this section, we describe exceptions to this rule.
The first MIDI channel command in the MIDI list MUST include a status
octet. Running status coding, as defined in [1], MAY be used for all
subsequent MIDI channel commands in the list. As in [1], System Common
and System Exclusive messages (0xF0 ... 0xF7) cancel the running status
state, but System Real-time messages (0xF8 ... 0xFF) do not affect the
running status state. All System commands in the MIDI list MUST include
a status octet.
Although the first channel command in the MIDI list MUST include a
status octet, the corresponding command in the original MIDI source data
stream might not have a status octet (in this case, the source would be
coding the command using running status). If the status octet of the
first channel command in the MIDI list does not appear in the source
data stream, the P (phantom) header bit MUST be set to 1. In all other
cases, the P bit MUST be set to 0.
As receivers MUST be able to decode running status, sender implementors
should feel free to use running status to improve bandwidth efficiency.
However, senders SHOULD NOT introduce timing jitter into an existing
MIDI command stream through an inappropriate use or removal of running
status coding.
On a MIDI 1.0 DIN cable [1], a System Real-time command may be embedded
inside of another "host" MIDI command. This syntactic construction is
not supported in the payload format: a MIDI Command field in the MIDI
list codes exactly one MIDI command (partially or completely).
To encode an embedded System Real-time command, senders MUST extract the
command from its host, and code it in the MIDI list as a separate
command. The host command and System Real-time command SHOULD appear in
the same MIDI list. The delta time of the System Real-time command
SHOULD result in a command timestamp that encodes the System Real-time
command placement in its original embedded position.
Two methods are provided for encoding MIDI System Exclusive (SysEx)
commands in the MIDI list. A SysEx command may be encoded in a MIDI
Command field verbatim: a 0xF0 octet, followed by an arbitrary number of
data octets, followed by a 0xF7 octet.
Alternatively, a SysEx command may be encoded as multiple segments. The
command is divided into two or more SysEx command segments; each segment
is encoded in its own MIDI Command field in the MIDI list.
Lazzaro/Wawrzynek [Page 13]
INTERNET-DRAFT 22 June 2004
The payload format supports segmentation in order to encode SysEx
commands that encode information in the temporal pattern of data octets.
By encoding these commands as a series of segments, each data octet may
be associated with a distinct delta time. Segmentation also supports
the coding of large SysEx commands across several packets.
To segment a SysEx command, first partition its data octet list into two
or more sublists. The last sublist MAY be empty (i.e. contain no
octets); all other sublists MUST contain at least one data octet. To
complete the segmentation, add the status octets defined in Figure 5 to
the head and tail of the first, last, and any "middle" sublists. Figure
6 shows example segmentations of a SysEx command.
A sender MAY cancel a segmented SysEx command transmission that is in
progress, by sending the "cancel" sublist shown in Figure 5. A "cancel"
sublist MAY follow a "first" or "middle" sublist in the transmission,
but MUST NOT follow a "last" sublist. The cancel sublist MAY contain
data octets or MAY be empty.
The cancellation feature lets a sender start the transmission of a
command before the MIDI source has sent the entire command, even if the
RTP stream is configured to block a subset of SysEx commands (for
example, commands larger than a certain size; Appendix C.1.3 defines
configuration tools for blocking SysEx commands). If a sender
determines that the command whose transmission is in progress should not
appear on the RTP stream, it cancels the command.
A cancelled transmission SHOULD be ignored by the receiver, if possible.
However, if a receiver has forwarded earlier command segments to a MIDI
destination that is not able to ignore commands, the receiver SHOULD
forward the data octets coded in the "cancel" sublist, and end the
forwarded SysEx command with an 0xF7.
Appendix C.1.3 defines configuration tools that may be used to prohibit
SysEx command cancellation.
The relative ordering of SysEx command segments in a MIDI list must
match the relative ordering of the sublists in the original SysEx
command. By default, commands other than System Real-time MIDI commands
MUST NOT appear between SysEx command segments (Appendix C.1.3 defines
configuration tools to change this default, to let other commands types
appear between segments). If the command segments of a SysEx command
are placed in the MIDI lists of two or more RTP packets, the segment
ordering rules apply to the concatenation of all affected MIDI lists.
Lazzaro/Wawrzynek [Page 14]
INTERNET-DRAFT 22 June 2004
-----------------------------------------------------------
| Sublist Position | Head Status Octet | Tail Status Octet |
|-----------------------------------------------------------|
| first | 0xF0 | 0xF0 |
|-----------------------------------------------------------|
| middle | 0xF7 | 0xF0 |
|-----------------------------------------------------------|
| last | 0xF7 | 0xF7 |
|-----------------------------------------------------------|
| cancel | 0xF7 | 0xF4 |
-----------------------------------------------------------
Figure 5 -- Command segmentation status octets
[1] permits 0xF7 octets that are not part of a (0xF0, 0xF7) pair to
appear on a MIDI 1.0 DIN cable. Unpaired 0xF7 octets have no semantic
meaning in MIDI, apart from cancelling running status.
Unpaired 0xF7 octets MUST NOT be appear in the MIDI list of the MIDI
Command section. We impose this restriction to avoid interference with
the command segmentation coding defined in Figure 5.
SysEx commands carried on a MIDI 1.0 DIN cable may use the "dropped
0xF7" construction [1]. In this coding method, the 0xF7 octet is
dropped from the end of the SysEx command, and the status octet of the
next MIDI command acts both to terminate the SysEx command and start the
next command. To encode this construction in the payload format, follow
these steps:
o Determine the appropriate delta times for the SysEx command and
the command that follows the SysEx command.
o Insert the "dropped" 0xF7 octet at the end of the SysEx command,
to form the standard SysEx syntax.
o Code both commands into the MIDI list using the rules above.
o Replace the 0xF7 octet that terminates the verbatim SysEx
encoding or the last segment of the segmented SysEx encoding
with a 0xF5 octet. This substitution informs the receiver
of the original dropped 0xF7 coding.
[1] reserves the undefined System Common opcodes 0xF4 and 0xF5 and the
undefined System Real-time opcodes 0xF9 and 0xFD for future use. By
default, undefined opcodes MUST NOT appear in a MIDI Command field in
the MIDI list, with the exception of the 0xF5 octets used to code the
"dropped 0xF7" construction and the 0xF4 octets used by SysEx "cancel"
Lazzaro/Wawrzynek [Page 15]
INTERNET-DRAFT 22 June 2004
sublists.
During session configuration, a stream may be customized to transport
undefined opcodes (Appendix C.1.3). For this case, we now define how
senders encode undefined commands in the MIDI list.
An undefined System Real-time command MUST be coded using the System
Real-time rules. For an undefined System Common command, we assume the
command begins with an 0xF4 or 0xF5 status octet, followed by zero or
more data octets, potentially followed by trailing status octet(s). To
encode the command, senders MUST first remove all trailing status octets
from the command. Then, senders MUST terminate the command with an 0xF7
octet, and place the modified command into the MIDI Command field.
Lazzaro/Wawrzynek [Page 16]
INTERNET-DRAFT 22 June 2004
Original SysEx command:
0xF0 0x01 0x02 0x03 0x04 0x05 0x06 0x07 0x08 0xF7
A two-segment segmentation:
0xF0 0x01 0x02 0x03 0x04 0xF0
0xF7 0x05 0x06 0x07 0x08 0xF7
A different two-segment segmentation:
0xF0 0x01 0xF0
0xF7 0x02 0x03 0x04 0x05 0x06 0x07 0x08 0xF7
A three-segment segmentation:
0xF0 0x01 0x02 0xF0
0xF7 0x03 0x04 0xF0
0xF7 0x05 0x06 0x07 0x08 0xF7
The segmentation with the largest number of segments:
0xF0 0x01 0xF0
0xF7 0x02 0xF0
0xF7 0x03 0xF0
0xF7 0x04 0xF0
0xF7 0x05 0xF0
0xF7 0x06 0xF0
0xF7 0x07 0xF0
0xF7 0x08 0xF0
0xF7 0xF7
Figure 6 -- Example segmentations
Lazzaro/Wawrzynek [Page 17]
INTERNET-DRAFT 22 June 2004
4. The Recovery Journal System
The recovery journal is the default resiliency tool for unreliable
transport. In this section, we normatively define the roles that
senders and receivers play in the recovery journal system.
MIDI is a fragile code. A single lost command in a MIDI command stream
may produce an artifact in the rendered performance. We normatively
classify rendering artifacts into two categories:
o Transient artifacts. Transient artifacts produce immediate
but short-term glitches in the performance. For example, a lost
NoteOn (0x9) command produces a transient artifact: one note
fails to play, but the artifact does not extend beyond the end
of that note.
o Indefinite artifacts. Indefinite artifacts produce long-lasting
errors in the rendered performance. For example, a lost NoteOff
(0x8) command may produce an indefinite artifact: the note that
should have been ended by the lost NoteOff command may sustain
indefinitely. As a second example, the loss of a Control Change
(0xB) command for controller number 7 (Channel Volume) may
produce an indefinite artifact: after the loss, all notes on
the channel may play too softly or too loudly.
The purpose of the recovery journal system is to satisfy the recovery
journal mandate: the MIDI performance rendered from an RTP MIDI stream
sent over unreliable transport MUST NOT contain indefinite artifacts.
The recovery journal system does not use packet retransmission to
satisfy this mandate. Instead, each packet includes a special section,
called the recovery journal.
The recovery journal codes the history of the stream, back to an earlier
packet called the checkpoint packet. The range of coverage for the
journal is called the checkpoint history. The recovery journal codes
the information necessary to recover from the loss of an arbitrary
number of packets in the checkpoint history. Appendix A.1 normatively
defines the checkpoint packet and the checkpoint history.
When a receiver detects a packet loss, it compares its own knowledge
about the history of the stream with the history information coded in
the recovery journal of the packet that ends the loss event. By noting
the differences in these two versions of the past, a receiver is able to
transform all indefinite artifacts in the rendered performance into
transient artifacts, by executing MIDI commands to repair the stream.
Lazzaro/Wawrzynek [Page 18]
INTERNET-DRAFT 22 June 2004
We now state the normative role for senders in the recovery journal
system.
Senders prepare a recovery journal for every packet in the stream. In
doing so, senders choose the checkpoint packet identity for the journal.
Senders make this choice by applying a sending policy. Appendix C.1.2
normatively defines three sending policies: "closed-loop", "open-loop",
and "anchor".
By default, senders MUST use the closed-loop sending policy. If the
session description overrides this default policy, by using the MIME
parameter j_update defined in Appendix C.1.2, senders MUST use the
specified policy.
After choosing the checkpoint packet identity for a packet, the sender
creates the recovery journal. By default, this journal MUST conform to
the normative semantics in Section 5 and Appendices A-B in this memo.
In Appendix C.1.3, we define MIME parameters that modify the normative
semantics for recovery journals. If the session description uses these
parameters, the journal created by the sender MUST conform to the
modified semantics.
Next, we state the normative role for receivers in the recovery journal
system.
A receiver MUST detect each RTP sequence number break in a stream. If
the sequence number break is due to a packet loss event (as defined in
[2]) the receiver MUST repair all indefinite artifacts in the rendered
MIDI performance caused by the loss. If the sequence number break is
due to an out-of-order packet (as defined in [2]) the receiver MUST NOT
take actions that introduce indefinite artifacts (ignoring the out-of-
order packet is a safe option).
Receivers take special precautions when entering or exiting a session.
A receiver MUST process the first received packet in a stream as if it
were a packet that ends a loss event. Upon exiting a session, a
receiver MUST ensure that the rendered MIDI performance does not end
with indefinite artifacts.
Receivers are under no obligation to perform indefinite artifact repairs
at the moment a packet arrives. A receiver that uses a playout buffer
may choose to wait until the moment of rendering before processing the
recovery journal, as the "lost" packet may be a late packet that arrives
in time to use.
Lazzaro/Wawrzynek [Page 19]
INTERNET-DRAFT 22 June 2004
Next, we state the normative role for the creator of the session
description in the recovery journal system. Depending on the
application, the sender, the receivers, and other parties may take part
in creating or approving the session description.
A session description that specifies the default closed-loop sending
policy and the default recovery journal semantics satisfies the recovery
journal mandate. However, these default behaviors may not be
appropriate for all sessions. If the creators of a session description
use the parameters defined in Appendix C.1 to override these defaults,
the creators MUST ensure that the parameters define a system that
satisfy the recovery journal mandate.
Finally, we note that this memo does not specify sender or receiver
recovery journal algorithms. Implementations are free to use any
algorithm that conforms to the requirements in this section. The non-
normative [19] discusses sender and receiver algorithm design.
5. Recovery Journal Format
This section introduces the structure of the recovery journal, and
defines the bitfields of recovery journal headers. Appendices A-B
complete the bitfield definition of the recovery journal.
The recovery journal has a three-level structure:
o Top-level header.
o Channel and system journal headers. Encodes recovery
information for a single voice channel (channel journal) or
for all systems commands (system journal).
o Chapters. Describes recovery information for a single MIDI
command type.
Figure 7 shows the top-level structure of the recovery journal. The
recovery journals consists of a 3-octet header, followed by an optional
system journal (labeled S-journal in Figure 7) and an optional list of
channel journals. Figure 8 shows the recovery journal header format.
Lazzaro/Wawrzynek [Page 20]
INTERNET-DRAFT 22 June 2004
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Recovery journal header | S-journal ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Channel journals ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 7 -- Top-level recovery journal format
0 1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|Y|A|R|TOTCHAN| Checkpoint Packet Seqnum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 8 -- Recovery journal header
If the Y header bit is set to 1, the system journal appears in recovery
journal, directly following the recovery journal header.
If the A header bit is set to 1, the recovery journal ends with a list
of (TOTCHAN + 1) channel journals (the 4-bit TOTCHAN header field is
interpreted as an unsigned integer).
A MIDI channel MAY be represented by (at most) one channel journal in a
recovery journal. Channel journals MUST appear in the recovery journal
in ascending channel-number order.
If A and Y are both zero, the recovery journal only contains its 3-octet
header, and is considered to be an "empty" journal.
The S (single-packet loss) bit appears in most recovery journal
structures, including the recovery journal header. The S bit helps
receivers efficiently parse the recovery journal in the common case of
the loss of a single packet. Appendix A.1 defines S bit semantics.
The R header bit is reserved. The semantics for R bits are uniform
throughout the recovery journal, and are defined in Appendix A.1.
The 16-bit Checkpoint Packet Seqnum header field codes the sequence
number of the checkpoint packet for this journal, in network byte order
(big-endian). The choice of the checkpoint packet sets the depth of the
checkpoint history for the journal (defined in Appendix A.1).
Lazzaro/Wawrzynek [Page 21]
INTERNET-DRAFT 22 June 2004
Receivers may use the Checkpoint Packet Seqnum field of the packet that
ends a loss event to verify that the journal checkpoint history covers
the entire loss event. The checkpoint history covers the loss event if
the Checkpoint Packet Seqnum field is less than or equal to one plus the
highest RTP sequence number previously received on the stream (modulo
2^16).
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| CHAN |R| LENGTH |P|C|M|W|N|E|T|A| Chapters ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 9 -- Channel journal format
Figure 9 shows the structure of a channel journal: a 3-octet header,
followed by a list of leaf elements called channel chapters. A channel
journal encodes information about MIDI commands on the MIDI channel
coded by the 4-bit CHAN header field.
The 10-bit LENGTH field codes the length of the channel journal. The
semantics for LENGTH fields are uniform throughout the recovery journal,
and are defined in Appendix A.1.
The third octet of the channel journal header is the Table of Contents
(TOC) of the channel journal. The TOC is a set of bits that encode the
presence of a chapter in the journal. Each chapter contains information
about a certain class of MIDI channel command:
o Chapter P: MIDI Program Change (0xC)
o Chapter C: MIDI Control Change (0xB)
o Chapter M: MIDI Parameter System (part of 0xB)
o Chapter W: MIDI Pitch Wheel (0xE)
o Chapter N: MIDI NoteOff (0x8), NoteOn (0x9)
o Chapter E: MIDI Note Command Extras (0x8, 0x9)
o Chapter T: MIDI Channel Aftertouch (0xD)
o Chapter A: MIDI Poly Aftertouch (0xA)
Chapters appear in a list following the header, in order of their
appearance in the TOC. Appendices A.2-9 describe the bitfield format
for each chapter, and define the conditions under which a chapter type
MUST appear in the recovery journal. If any chapter types are required
for a channel, an associated channel journal MUST appear in the recovery
journal.
Lazzaro/Wawrzynek [Page 22]
INTERNET-DRAFT 22 June 2004
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|D|V|Q|F|X| LENGTH | System chapters ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 10 -- System journal format
Figure 10 shows the structure of the system journal: a 2-octet header,
followed by a list of system chapters. Each chapter codes information
about a specific class of MIDI Systems command:
o Chapter D: Song Select (0xF3), Tune Request (0xF6), Reset (0xFF),
undefined System commands (0xF4, 0xF5, 0xF9, 0xFD)
o Chapter V: Active Sense (0xFE)
o Chapter Q: Sequencer State (0xF2, 0xF8, 0xF9, 0xFA, 0xFB, 0xFC)
o Chapter F: MTC Tape Position (0xF1, 0xF0 0x7F 0xcc 0x01 0x01)
o Chapter X: System Exclusive (all other 0xF0)
The 10-bit LENGTH field codes the size of the system journal, and
conforms to semantics described in Appendix A.1.
The D, V, Q, F, and X header bits form a Table of Contents (TOC) for the
system journal. A TOC bit that is set to 1 codes the presence of a
chapter in the journal. Chapters appear in a list following the header,
in the order of their appearance in the TOC.
Appendix B describes the bitfield format for the system chapters, and
define the conditions under which a chapter type MUST appear in the
recovery journal. If any system chapter type is required to appear in
the recovery journal, the system journal MUST appear in the recovery
journal.
6. Session Description Protocol
RTP does not perform session management. Instead, RTP is designed to
work together with tools that perform session management, such as the
Session Initiation Protocol (SIP, [16]) and the Real Time Streaming
Protocol (RTSP, [17]).
RTP interacts with session management tools via another standard, the
Session Description Protocol (SDP, [6]). SDP is a textual format for
specifying session descriptions. Session descriptions specify the
network transport and media encoding for RTP streams. Session
management tools coordinate the exchange of session descriptions between
participants, and may also support negotiations [13].
Lazzaro/Wawrzynek [Page 23]
INTERNET-DRAFT 22 June 2004
Below, we show session description examples for native (Section 6.1) and
mpeg4-generic (Section 6.2) streams. In Section 6.3, we introduce
session configuration tools that may be used to customize streams.
6.1 Session Descriptions for Native Streams
The session description below is the minimal session description for a
native RTP MIDI stream sent over unicast UDP transport.
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 rtp-midi/44100
The rtpmap attribute line uses the rtp-midi MIME type to specify a
native stream.
We consider the session description to be "minimal" because the session
description does not customize the stream. Without such customization,
a native RTP MIDI stream has these characteristics:
1. If the stream uses unreliable transport (unicast UDP, multicast
UDP, ...) the recovery journal system is in use, and the RTP
payload contains both the MIDI command section and the journal
section. If the stream uses reliable transport (such as TCP),
the stream does not use journalling, and the payload contains
only the MIDI command section (Section 2.2).
2. If the stream uses the recovery journal system, the recovery
journal system uses the default sending policy and the default
journal semantics (Section 4).
3. In the MIDI command section of the payload, command timestamps
use the default semantics (Section 3).
4. The media time encoded by an RTP packet may range from 0 to
200 ms, and the RTP timestamp difference between sequential
packets in the stream may be arbitrarily large (Section 2.1).
5. If more than one minimal rtp-midi stream appears in a session,
the MIDI name spaces for these streams are independent: channel
1 in the first stream does not reference the same MIDI channel
as channel 1 in the second stream.
6. The rendering method for the stream is not specified.
Lazzaro/Wawrzynek [Page 24]
INTERNET-DRAFT 22 June 2004
By default, an RTP session, and thus a minimal RTP MIDI session, is
bidirectional ("sendrecv", as defined in [6]). Each participant acts as
sender and receiver, as in traditional telephony.
[6] and [13] describe how sendrecv session descriptions specify
transport addresses. [6] and [13] also show how to configure RTP
sessions to be unidirectional, so that parties act only as senders
("sendonly") or only as receivers ("recvonly").
The nature of the bidirectional data flows in a minimal RTP MIDI session
depends on the application. For applications that are analogous to
telephony, such as network musical performance [14], the parties are
peers, and the information flows between the parties are symmetric
(party A sends note commands to party B, party B sends note commands to
party A).
In other applications, such as control of a rack-mounted music
synthesizer by a personal computer, one party acts as a server (the
personal computer) and one party act as a client (the synthesizer). In
this case, the bidirectional RTP MIDI session mimics a MIDI 1.0 DIN
network, which we now briefly describe (see [1] for details).
In a MIDI 1.0 DIN network, a pair of MIDI cables cross-connect the
computer and the rack-mount music synthesizer. The traffic flow on the
MIDI cables is asymmetric. The server sends a heavy forward flow to the
client. The forward flow may include MIDI System commands that request
that feedback commands be sent from the client to the server.
Lazzaro/Wawrzynek [Page 25]
INTERNET-DRAFT 22 June 2004
6.2 Session Descriptions for mpeg4-generic Streams
An mpeg4-generic [4] RTP MIDI stream uses an MPEG 4 Audio Object Type to
render MIDI into audio. Three Audio Object Types accept MIDI input:
o General MIDI (Audio Object Type ID 15), based on the General
MIDI rendering standard [1].
o Wavetable Synthesis (Audio Object Type ID 14), based on the
Downloadable Sounds Level 2 (DLS 2) rendering standard [9].
o Main Synthetic (Audio Object Type ID 13), based on Structured
Audio and the programming language SAOL [5].
The primary service of an mpeg4-generic stream is to code Access Units
(AUs). We define the mpeg4-generic RTP MIDI AU as the MIDI payload
shown in Figure 1 of Section 2.1 of this memo: a MIDI command section
optionally followed by a journal section.
Exactly one RTP MIDI AU MUST be mapped to one mpeg4-generic RTP MIDI
packet. The mpeg4-generic options for placing several AUs in an RTP
packet MUST NOT be used with RTP MIDI. The mpeg4-generic options for
fragmenting and interleaving AUs MUST NOT be used with RTP MIDI. The
mpeg4-generic RTP packet payload (Figure 1 in [4]) MUST contain empty AU
Header and Auxiliary sections. These rules yield mpeg4-generic packets
that are structurally identical to native RTP MIDI packets, an essential
property for the correct operation of the payload format.
The session description below is the minimal session description for an
mpeg4-generic RTP MIDI stream sent over unicast UDP transport. This
example uses the General MIDI Audio Object Type under Synthesis Profile
@ Level 2.
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP6 FF1E:03AD::7F2E:172A:1E24
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; profile-level-id=12;
config=7A124D546864000000060000000100604D54726B0000000400FF2F000
(The a=fmtp line has been wrapped to fit the page to accommodate
memo formatting restrictions; it comprises a single line in SDP)
The fmtp attribute line codes the four parameters (streamtype, mode,
profile-level-id, and config) that are required in all mpeg4-generic
Lazzaro/Wawrzynek [Page 26]
INTERNET-DRAFT 22 June 2004
session descriptions [4]. For RTP MIDI streams, the streamtype
parameter MUST be set to 5, the "mode" parameter MUST be set to "rtp-
midi", and the "profile-level-id" parameter MUST be set to the MPEG-4
Profile Level for the stream.
In a minimal RTP MIDI session description, the config value MUST be a
hexadecimal encoding [4] of the AudioSpecificConfig data block [7] for
the stream. AudioSpecificConfig encodes the Audio Object Type for the
stream, and also encodes initialization data (SAOL programs, DLS 2 wave
tables, etc). Standard MIDI Files encoded in AudioSpecificConfig in a
minimal session description MUST be ignored by the receiver.
We consider the session description to be "minimal" because the session
description does not customize the stream. In Section 6.1, we describe
the behavior of a minimal native stream, as a numbered list of
characteristics. Items 1-4 on that list also describe the minimal
mpeg4-generic stream, but items 5 and 6 require restatements, as listed
below:
5. If more than one minimal mpeg4-generic stream appears in
a session, each stream uses an independent instance of the
Audio Object Type coded in the config parameter value.
6. A minimal mpeg4-generic stream encodes the AudioSpecificConfig
as an inline hexadecimal constant. If session description
is sent over UDP, it may be impossible to transport large
AudioSpecificConfig blocks, as the Maximum Transmission Size
(MTU) of the underlying network limits the UDP packet size
(for Ethernet, the MTU is 1500 octets).
By default, a minimal mpeg4-generic RTP MIDI session is bidirectional.
Each party acts as sender and receiver. The session follows the
telephony model: parties act as peers and the information flows between
parties are symmetrical in nature (party A sends note commands to party
B, party B sends note commands to party A).
Each party's session description MUST use identical values for the
mpeg4-generic MIME parameters (including the required streamtype, mode,
profile-level-id, and config parameters). As a consequence, each party
uses an identically-configured MPEG 4 Audio Object Type to render MIDI
commands into audio.
RTP sessions MAY be configured to be unidirectional, so that parties act
only as senders or only as receivers [6] [13]. In this way,
mpeg4-generic RTP MIDI may be used for content-delivery applications.
Lazzaro/Wawrzynek [Page 27]
INTERNET-DRAFT 22 June 2004
6.3 Session Configuration Tools
This section introduces the session configuration tools for RTP MIDI
sessions. The tools add features to the minimal streams described in
Sections 6.1-2, and support several types of services:
o Journal customization. The j_sec and j_update parameters
configure the use of the journal section. The ch_default,
ch_unused, ch_never, ch_anchor, and ch_active parameters
configure the semantics of the recovery journal chapters.
These MIME parameters are described in Appendix C.1, and
override the default stream behaviors 1 and 2 listed in
Section 6.1 and referenced in Section 6.2.
o MIDI command timestamp semantics. The tsmode, octpos,
mperiod, and linerate parameters customize the semantics
of timestamps in the MIDI command section. These parameters
let RTP MIDI accurately encode the implicit time coding of
MIDI 1.0 DIN cables. These MIME parameters are described in
Appendix C.2, and override default stream behavior 3 listed in
Section 6.1 and referenced in Section 6.2
o Media time. The standard SDP attributes ptime and
maxptime define the media time encoded by a packet. The
guardtime MIME parameter sets the minimum sending rate of
stream packets. These tools are described in Appendix C.3,
and override default stream behavior 4 listed in Section
6.1 and referenced in Section 6.2.
o Multiple streams. The musicport parameter labels the
MIDI name space of multi-stream sessions. The zerosync
parameter supports synchronization in multi-stream sessions.
These MIME parameters are described in Appendix C.4, and
override default stream behavior 5 in Sections 6.1 and 6.2.
o MIDI rendering. Several MIME parameters specify the MIDI
rendering method of a stream. These parameters are described
in Appendix C.5, and override default stream behavior 6 in
Sections 6.1 and 6.2.
In Appendix C.6, we specify the feature set that an RTP MIDI client is
expected to support, and we discuss how to use the Offer/Answer protocol
[13] to negotiate the feature set for a media session.
Lazzaro/Wawrzynek [Page 28]
INTERNET-DRAFT 22 June 2004
7. Extensibility
The payload format defined in this memo exclusively encodes all commands
that may legally appear on a MIDI 1.0 DIN cable.
Many worthy uses of MIDI over RTP do not fall within the narrow scope of
the payload format. For example, the payload format does not support
the direct transport of Standard MIDI File (SMF) meta-event and metric
timing data. As a second example, the payload format does not define
transport tools for user-defined commands (apart from tools to support
System Exclusive commands [1]).
The payload format does not provide an extension mechanism to support
new features of this nature, by design. Instead, we encourage the
development of new payload formats for specialized musical applications.
The IETF session management tools [13] [17] support codec negotiation,
to facilitate the use of new payload formats in a backward-compatible
way.
However, the payload format does provide several extensibility tools,
which we list below:
o Journalling. As described in Appendix C.1, new token
values for the j_sec and j_update MIME parameters may
be defined in IETF standards-track documents. This
mechanism supports the design of new journal formats
and the definition of new journal sending policies.
o Rendering. The payload format may be extended to support
new MIDI renderers (Appendix C.5.3). The extension
mechanism uses the standard MIME registration process [21].
Certain general aspects of the RTP MIDI rendering process may
also be extended, via IETF standards-track documents that
define new token values for the render (Appendix C.5) and
smf_info (Appendix C.5.5.1) MIME parameters.
o Undefined opcodes. [1] reserves 4 MIDI System opcodes
for future use (0xF4, 0xF5, 0xF9, 0xFD). If updates
to [1] define the reserved opcodes, IETF standards-track
documents may be defined to provide resiliency support for
the commands. Opaque LEGAL fields appear in System Chapter
D for this purpose (Appendix B.1.1).
Lazzaro/Wawrzynek [Page 29]
INTERNET-DRAFT 22 June 2004
A final form of extensibility involves the inclusion of the payload
format in framework documents. Framework documents describe how to
combine protocols to form a platform for interoperable applications.
For example, a network musical performance [14] framework might define
how to use SIP [16], SDP [6] and RTP/AVP [2] [3] to support real-time
performances between geographically-distributed players. We discuss
frameworks from an interoperability perspective in Appendix C.6.
8. Congestion Control
The RTP congestion control requirements defined in [2] apply to RTP MIDI
sessions, and implementors should carefully read the congestion control
section in [2]. As noted in [2], all transport protocols used on the
Internet need to address congestion control in some way, and RTP is not
an exception.
In addition, as RTP MIDI runs under the Audio/Video Profile [3], the
congestion control requirements defined in [3] apply to RTP MIDI
sessions. The basic congestion control requirement defined in [3] is
that RTP sessions that use UDP transport should monitor packet loss (via
RTCP, or via other means) to ensure that the RTP stream competes fairly
with TCP flows that share the network.
Finally, RTP MIDI has congestion control issues that are unique for an
audio RTP/AVP payload format. In applications such as network musical
performance [14], the packet rate is linked to the gestural rate of a
human performer. Senders MUST monitor the MIDI command source for
patterns that result in excessive packet rates, and take actions during
RTP transcoding to reduce the RTP packet rate. [19] offers
implementation guidance on this issue.
Lazzaro/Wawrzynek [Page 30]
INTERNET-DRAFT 22 June 2004
A. The Recovery Journal Channel Chapters
A.1 Recovery Journal Definitions
This Appendix defines the terminology and the coding idioms that are
used in the recovery journal bitfield descriptions in Section 5 (journal
header structure), Appendices A.2-9 (channel journal chapters) and
Appendices B.1-5 (system journal chapters).
We assume that the recovery journal resides in the journal section of an
RTP packet with sequence number I ("packet I") and that the Checkpoint
Packet Seqnum field in the top-level recovery journal header refers to a
previous packet with sequence number C (an exception is the self-
referential C = I case). Unless stated otherwise, algorithms are
assumed to use modulo 2^16 arithmetic for calculations on 16-bit
sequence numbers and modulo 2^32 arithmetic for calculations on 32-bit
extended sequence numbers.
Several bitfield coding idioms appear throughout the recovery journal
system, with consistent semantics. Most recovery journal elements begin
with an "S" (Single-packet loss) bit. S bits are designed to help
receivers efficiently parse through the recovery journal hierarchy in
the common case of the loss of a single packet.
As a rule, S bits MUST be set to 1. However, an exception applies if a
recovery journal element in packet I encodes data about a command stored
in the MIDI command section of packet I - 1. In this case, the S bit of
the recovery journal element MUST be set to 0. If a recovery journal
element has its S bit set to 0, all higher-level recovery journal
elements that contain it MUST also have S bits that are set to 0,
including the top-level recovery journal header.
Other consistent bitfield coding idioms are described below:
o R flag bit. R flag bits are reserved for future use. Senders
MUST set R bits to 0. Receivers MUST ignore R bit values.
o LENGTH field. All fields named LENGTH (as distinct from LEN)
code the number of octets in the structure that contains it,
including the header it resides in and all hierarchical levels
below it. If a structure contains a LENGTH field, a receiver
MUST use the LENGTH field value to advance past the structure
during parsing, rather than use knowledge about the internal
format of the structure.
Lazzaro/Wawrzynek [Page 31]
INTERNET-DRAFT 22 June 2004
We now define normative terms used to describe recovery journal
semantics.
o Checkpoint history. The checkpoint history of a recovery journal
is the concatenation of the MIDI command sections of packets C
through I - 1. The final command in the MIDI command section for
packet I - 1 is considered the most recent command; the first
command in the MIDI command section for packet C is the oldest
command. If command X is less recent than command Y, X is
considered to be "before Y". A checkpoint history with no
commands is considered to be empty. The checkpoint history
never contains the MIDI command section of the packet I (the
packet containing the recovery journal), so if C == I, the
checkpoint history is empty by definition.
o Session history. The session history of a recovery journal is
the concatenation of MIDI command sections from the first
packet of the session up to packet I - 1. The definitions of
command recency and history emptiness follow those in the
checkpoint history. The session history never contains the
MIDI command section of packet I, and so the session history of
the first packet in the session is empty by definition.
o Finished/unfinished commands. If all octets of a MIDI command
appear in the session history, the command is defined to be
finished. If some but not all octets of a command appear
in the session history, the command is defined to be unfinished.
Unfinished commands occur if segments of a SysEx command appear
in several RTP packets. For example, if a SysEx command is coded
as 3 segments, with segment 1 in packet K, segment 2 in packet
K + 1, and segment 3 in packet K + 2, the session histories for
packets K + 1 and K + 2 contain unfinished versions of the command.
A session history contains a finished version of a cancelled SysEx
command if the history contains the cancel sublist for the command.
o Reset State commands. Reset State (RS) commands reset
renderers to an initialized "powerup" condition. The
RS commands are: System Reset (0xFF), General MIDI System Enable
(0xF0 0x7E 0xcc 0x09 0x01 0xF7), General MIDI 2 System Enable
(0xF0 0x7E 0xcc 0x09 0x03 0xF7), General MIDI System Disable
(0xF0 0x7E 0xcc 0x09 0x00 0xF7), Turn DLS On (0xF0 0x7E 0xcc 0x0A
0x01 0xF7) and Turn DLS Off (0xF0 0x7E 0xcc 0x0A 0x02 0xF7).
MIME registrations for renderers (Appendix C.5.3) and IETF
standards-track documents MAY specify additional RS commands.
o Active commands. Active command are MIDI commands that do not
appear before a Reset State command in the session history.
Lazzaro/Wawrzynek [Page 32]
INTERNET-DRAFT 22 June 2004
o N-active commands. N-active commands are MIDI commands that do
not appear before one of the following commands in the session
history: MIDI Control Change numbers 123-127 (numbers with All
Notes Off semantics) or 120 (All Sound Off), and any Reset
State command.
o C-active commands. C-active commands are MIDI commands that do
not appear before one of the following commands in the session
history: MIDI Control Change number 121 (Reset All Controllers)
and any Reset State command.
o Parameter system. A MIDI feature that provides two sets of
16,384 parameters to expand the 0-127 controller number space.
The Registered Parameter Names (RPN) system and the Non-Registered
Parameter Names (NRPN) system each provides 16,384 parameters.
o Parameter system transaction. The value of RPNs and NRPNs are
changed by a series of Control Change commands that form a
parameter system transaction. A canonical transaction begins
with two Control Change commands to set the parameter number
(controller numbers 99 and 98 for NRPNs, controller numbers 101
and 100 for RPNs). The transaction continues with an arbitrary
number of Data Entry (controller numbers 6 and 38), Data Increment
(controller number 96), and Data Decrement (controller number
97) Control Change commands to set the parameter value. The
transaction ends with a second pair of (99, 98) or (101, 100)
Control Change commands that specify the null parameter (MSB
value 0x7F, LSB value 0x7F)
Several variants of the canonical transaction sequence are
possible. Most commonly, the terminal pair of (99, 98) or
(101, 100) Control Change commands may specify a parameter
other than the null parameter. In this case, the command
pair terminates the first transaction and starts a second
transaction. The command pair is considered to be a part
both transactions. This variant is legal and recommended
in [1]. We refer to this variant as a "type 1 variant".
Less commonly, the MSB (99 or 101) or LSB (98 or 100) command
of a (99, 98) or (101, 100) Control Change pair may be omitted.
If the MSB command is omitted, the transaction uses the MSB value
of the most recent C-active Control Change command for controller
number 99 or 101 that appears in the session history. We refer to
this variant as a "type 2 variant".
If the LSB command is omitted, the LSB value 0x00 is assumed. We
refer to this variant as a "type 3 variant". The type 2 and type 3
Lazzaro/Wawrzynek [Page 33]
INTERNET-DRAFT 22 June 2004
variants are defined as legal, but are not recommended, in [1].
o Initiated parameter system transaction. A canonical parameter
system transaction whose (99, 98) or (101, 100) initial Control
Change command pair appears in the session history is considered
to be an initiated parameter system transaction. This definition
also holds for type 1 variants. For type 2 variants (dropped MSB),
a transaction whose initial LSB Control Change command appears in
the session history is an initiated transaction. For type 3
variants (dropped LSB), a transaction is considered to be
initiated if at least one transaction command follows the initial
MSB (99 or 101) Control Change command in the session history.
The completion of a transaction does not nullify its "initiated"
status.
The chapter definitions in Appendices A.2-9 and B.1-5 reflect the
default recovery journal behavior. The ch_default, ch_unused, ch_never,
ch_anchor, and ch_active parameters modify these definitions, as
described in Appendix C.1.3.
The chapter definitions specify if data MUST be present in the journal.
Senders MAY also include non-required data in the journal. This
optional data MUST comply with the normative chapter definition. For
example, if a chapter definition states that a field codes data from the
most recent active command in the session history, the sender MUST NOT
code inactive commands or older commands in the field.
Finally, we note that a channel journal only encodes information about
MIDI commands appearing on the MIDI channel the journal protects. All
references to MIDI commands in Appendices A.2-9 should be read as "MIDI
commands appearing on this channel."
Lazzaro/Wawrzynek [Page 34]
INTERNET-DRAFT 22 June 2004
A.2 Chapter P: MIDI Program Change
A channel journal MUST contain Chapter P if an active Program Change
(0xC) command appears in the checkpoint history. Figure A.2.1 shows the
format for Chapter P.
0 1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| PROGRAM |B| BANK-MSB |X| BANK-LSB |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.2.1 -- Chapter P format
The chapter has a fixed size of 24 bits. The PROGRAM field indicates
the data value of the most recent active Program Change command in the
session history. By default, the B, BANK-MSB, X, and BANK-LSB fields
MUST be set to 0. Below, we define exceptions to this default
condition.
If an active Control Change (0xB) command for controller number 0 (Bank
Select MSB) appears before the Program Change command in the session
history, the B bit MUST be set to 1, and the BANK-MSB field MUST code
the data value of the Control Change command.
If B is set to 1, the BANK-LSB field MUST code the data value of the
most recent Control Change command for controller number 32 (Bank Select
LSB) that preceded the Program Change command coded in the PROGRAM field
and followed the Control Change command coded in the BANK-MSB field. If
no such Control Change command exists, the BANK-LSB field MUST be set to
0.
If B is set to 1, and if a Control Change command for controller number
121 (Reset All Controllers) appears in the MIDI stream between the
Control Change command coded by the BANK-MSB field and the Program
Change command coded by the PROGRAM field, the X bit MUST be set to 1.
Lazzaro/Wawrzynek [Page 35]
INTERNET-DRAFT 22 June 2004
A.3 Chapter C: MIDI Control Change
Readers may wish to review the Appendix A.1 definition of "C-active
commands" before reading this Appendix.
Figure A.3.1 shows the format for Chapter C.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 8 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| LEN |S| NUMBER |A| VALUE/ALT |S| NUMBER |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|A| VALUE/ALT | .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.3.1 -- Chapter C format
The chapter consists of a 1-octet header, followed by a variable length
list of 2-octet controller logs. The list MUST contain at least one
controller log. The 7-bit LEN field codes the number of controller logs
in the list, minus one. We define the semantics of the controller log
fields in Appendix A.3.2.
A channel journal MUST contain Chapter C if the rules defined in this
Appendix require that one or more controller logs appear in the list.
A.3.1 Log Inclusion Rules
If a C-active Control Change command for a controller number in the
range 0-119 appears in the checkpoint history, the list MUST contain a
controller log for the number, with possible exceptions for numbers 0,
6, 32-63 and 96-101.
If an active Control Change command for a controller number in the range
120-127 appears in the checkpoint history, the list MUST contain a
controller log for the number, with possible exceptions for numbers
124-127.
We now define the rules for the exceptions.
o MIDI streams may transmit 14-bit controller values using paired
Most Significant Byte (MSB, controller numbers 0-31, 99, 101) and
Least Significant Byte (LSB, controller numbers 32-63, 98, 100)
Control Change commands [1].
If the most recent C-active Control Change command in the session
Lazzaro/Wawrzynek [Page 36]
INTERNET-DRAFT 22 June 2004
history for a 14-bit controller pair uses the MSB number, Chapter
C MAY omit the controller log for the associated LSB number, as
the command ordering makes this LSB value irrelevant. However,
this exception MUST NOT be applied if the sender is not certain
that the MIDI source uses 14-bit semantics for the controller
number pair. Note that some MIDI sources ignore 14-bit controller
semantics, and use the LSB controller numbers as independent 7-bit
controllers.
o If C-active Control Change commands for controller numbers 0 (Bank
Select MSB) or 32 (Bank Select LSB) appear in the checkpoint
history, and if the command instances are also coded in the
BANK-MSB and BANK-LSB fields of the Chapter P (Appendix A.2),
Chapter C MAY omit the controller logs for the commands.
o Several controller numbers pairs are defined to be mutually
exclusive. Controller numbers 124 (Omni Off) and 125 (Omni On)
form a mutually exclusive pair, as do controller numbers 126
(Mono) and 127 (Poly).
If active Control Change commands for one or both members of
a mutually exclusive pair appear in the checkpoint history, a
log for the controller number of the most recent command for the
pair in the checkpoint history MUST appear in the controller list.
However, the list MAY omit the controller log for the other number
in the pair.
If active Control Change commands for one or both members of a
mutually exclusive pair appear in the session history, and a log
for the controller number of the most recent command for the pair
does not appear in the controller list, a log for the other number
of the pair MUST NOT appear in the controller list.
o Appendix A.3.4 defines exception rules for the MIDI Parameter
System controller numbers 6, 38, and 96-101.
The ch_active MIME parameter (Appendix C.1.3) may be used to change
Chapter C semantics to support MIDI renderers (such as [9] in certain
configurations) that exclude particular controller numbers from the
semantics of Control Change commands for controller 121 (Reset All
Controllers). Appendix C.1.3 defines how ch_active modifies Chapter C
semantics.
Lazzaro/Wawrzynek [Page 37]
INTERNET-DRAFT 22 June 2004
A.3.2 Controller Log Format
Figure A.3.2 shows the controller log structure of Chapter C.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NUMBER |A| VALUE/ALT |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.3.2 -- Chapter C controller log
The 7-bit NUMBER field identifies the controller number. The 7-bit
VALUE/ALT field codes recovery information for the controller number.
The A bit defines the coding format of the VALUE/ALT field.
Chapter C provides three tools for coding recovery information in the
VALUE/ALT field: the value tool, the toggle tool, and the count tool.
Implementations may choose among the tools to best code recovery
information for a particular controller number.
In the value tool, the 7-bit VALUE/ALT field codes the control value of
the most recent C-active (controller numbers 0-119) or active
(controller numbers 120-127) Control Change command in the session
history. This tool works best for controllers that code a continuous
quantity, such as number 1 (Modulation Wheel). If the value tool is
chosen, the A bit is set to 0.
The A bit is set to 1 to code the toggle or count tool. These tools
work best for controllers that code discrete actions. Figure A.3.3
shows the controller log for these tools.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NUMBER |1|T| ALT |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.3.3 -- Controller log for ALT tools
The T flag is set to 1 to code the toggle tool; T is set to 0 to code
the count tool. Both methods use the 6-bit ALT field as an unsigned
integer.
Lazzaro/Wawrzynek [Page 38]
INTERNET-DRAFT 22 June 2004
The toggle tool works best for controllers that act as on/off switches,
such as 64 (Damper Pedal (Sustain)). These controllers code the "off"
state with control values 0-63 and the "on" state with 64-127. The ALT
field codes the total number of toggles (off->on and on->off) due to
Control Change commands in the session history, including toggle events
caused by Control Change commands for controller number 121 (Reset All
Controllers).
Toggle counting is performed modulo 64. The toggle count is reset at
the start of a session, and whenever a Reset State command (Appendix
A.1) appears in the session history. When these reset events occur, the
toggle count for a controller is set to 0 (for controllers whose default
value is 0-63) or 1 (for controllers whose default value is 64-127).
The Damper Pedal (Sustain) controller illustrates the benefits of the
toggle tool over the value tool for switch controllers. As often used
in piano applications, the "on" state of the controller lets notes
resonate, while the "off" state immediately damps notes to silence. The
loss of the "off" command in an "on->off->on" sequence results in
ringing notes that should have been damped silent. The toggle tool lets
receivers detect this lost "off" command but the value tool does not.
The count tool is similar to the toggle tool, but is optimized for
controllers whose controller value is ignored, such as number 123 (All
Notes Off). For the count tool, the ALT field codes the total number of
Control Change commands in the session history. Command counting is
performed modulo 64.
The command count is set to 0 at the start of the session, and is reset
to 0 whenever a Reset State command (Appendix A.1) appears in the
session history.
A.3.3 Log List Coding Rules
In this section, we describe the organization of controller logs in the
Chapter C log list.
In most situations, a controller number SHOULD be coded by a single tool
(and thus, a single controller log). If a number is coded with a single
tool, and this tool is the count tool, recovery Control Change commands
generated by a receiver SHOULD use the default control value for the
controller.
A controller number MAY be coded by several tool types (and thus,
several controller logs, each using a different tool). This technique
may improve recovery performance for controllers with complex semantics,
such as controller number 84 (Portamento Control), or controller number
121 (Reset All Controllers) when used with a non-zero data octet (with
Lazzaro/Wawrzynek [Page 39]
INTERNET-DRAFT 22 June 2004
the semantics described in [9]).
However, multiple logs for the same controller number that use the SAME
tool type MUST NOT appear in the controller list.
We now define the ordering rules for the logs in the controller list.
Logs associated with the most recent Control Change command in the
session history coded in Chapter C MUST appear at the end of the list.
Logs associated with the oldest Control Change command in the session
history coded in Chapter C MUST appear at the start of the list.
All other logs MUST be arranged with respect to these boundary logs, to
produce a list ordering that strictly reflects the relative session
history recency of the Control Change commands coded by the logs.
This ordering codes the information necessary for the recovery of 14-bit
controller values, without precluding the use of MSB and LSB controller
pairs as independent 7-bit controllers.
A.3.4 The Parameter System
Readers may wish to review the Appendix A.1 definitions of "parameter
system", "parameter system transaction", and "initiated parameter system
transaction" before reading this section.
Parameter system transactions update a MIDI Registered Parameter Number
(RPN) or Non-Registered Parameter Number (NRPN) value. A parameter
system transaction is a sequence of Control Change commands that may use
the following controllers numbers:
o Data Entry MSB (6)
o Data Entry LSB (38)
o Data Increment (96)
o Data Decrement (97)
o Non-Registered Parameter Number (NPRN) LSB (98)
o Non-Registered Parameter Number (NRPN) MSB (99)
o Registered Parameter Number (RPN) LSB (100)
o Registered Parameter Number (RPN) MSB (101)
Control Change commands that are a part of a parameter system
transaction MUST NOT be coded in Chapter C controller logs. Instead,
these commands are coded in Chapter M, the MIDI Parameter chapter
defined in Appendix A.4.
Lazzaro/Wawrzynek [Page 40]
INTERNET-DRAFT 22 June 2004
However, Control Change commands that use the listed controllers as
general-purpose controllers (i.e. outside of a parameter system
transaction) MUST NOT be coded in Chapter M. Instead, the controllers
are coded as controller logs in Chapter C, under the following rules:
o If C-active Control Change commands for controller numbers 6,
38, or 96-101 appear in the checkpoint history, and these
commands are used as general-purpose controllers, the most
recent general-purpose command instance for these controller
numbers MUST appear as entries in the controller list.
o If C-active Control Change commands for controller numbers
6, 38, or 96-101 appear in the session history, and these
commands are used as general-purpose controllers, the most
recent general-purpose command instance for these controller
numbers MAY appear as entries in the controller list.
Also note that the rules for coding paired LSB and MSB controllers, as
defined in Appendix A.3.1, apply to the pairs (6, 38), (99, 98), and
(101, 100) when coded in Chapter C.
MIDI syntax permits a source to use controllers 6, 38, 96, and 97 as
parameter-system controllers AND general-purpose controllers in the same
stream. An RTP MIDI sender MUST deduce the role of each Control Change
command for these controller numbers by noting the placement of the
command in the stream, and MUST use this information to code the command
in Chapter C or Chapter M as appropriate.
A MIDI source that follows the recommendations of [1] exclusively uses
numbers 98-101 as parameter system controllers. Alternatively, a MIDI
source may exclusively use 98-101 as general-purpose controllers, and
lose the ability perform parameter system transactions in a stream.
In the language of [1], the general-purpose use of controllers 98-101
constitutes a non-standard controller assignment. As most real-world
MIDI sources use the standard controller assignment for controller
numbers 98-101, an RTP MIDI sender SHOULD assume these controllers act
as parameter system controllers unless it knows that a MIDI source uses
controller numbers 98-101 in a general-purpose way.
Lazzaro/Wawrzynek [Page 41]
INTERNET-DRAFT 22 June 2004
A.4 Chapter M: MIDI Parameter System
Readers may wish to review the Appendix A.1 definitions for "parameter
system", "parameter system transaction", and "initiated parameter system
transaction" before reading this Appendix.
Chapter M protects parameter system transactions for Registered
Parameter Number (RPN) and Non-Registered Parameter Number (NRPN)
values. Figure A.4.1 shows the format for Chapter M.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|P|E|U|W|Z| LENGTH |Q| PENDING | Log list ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.4.1 -- Top-level Chapter M format
Chapter M begins with a 2-octet header. If the P header bit is set to
1, a 1-octet field follows the header, coding the 7-bit PENDING value
and its associated Q bit.
The 10-bit LENGTH field codes the size of Chapter M, and conforms to
semantics described in Appendix A.1.
Chapter M ends with a list of zero or more variable-length parameter
logs. Appendix A.4.2 defines the bitfield format of a parameter log.
Appendix A.4.1 defines the inclusion semantics of the log list.
A channel journal MUST contain Chapter M if the rules defined in
Appendix A.4.1 require that one or more parameter logs appear in the
list. A channel journal also MUST contain Chapter M if the most recent
C-active Control Change command involved in a parameter system
transaction in the checkpoint history is:
o An RPN MSB (101) or NPRN MSB (99) controller.
o An RPN LSB (100) or NPRN LSB (98) controller that completes the
coding of the null parameter (MSB value 0x7F, LSB value 0x7F).
If the most recent C-active Control Change command involved in a
parameter system transaction in the session history is for the RPN MSB
or NPRN MSB controller, the P header bit MUST be set to 1, and the
PENDING field (and its associated Q bit) MUST follow the Chapter M
header. Otherwise, the P header bit MUST be set to 0, and the PENDING
field and Q bit MUST NOT appear in Chapter M.
Lazzaro/Wawrzynek [Page 42]
INTERNET-DRAFT 22 June 2004
If PENDING codes an NRPN MSB, the Q bit MUST be set to 1. If PENDING
codes an RPN MSB, the Q bit MUST be set to 0.
The E header bit codes the current transaction state of the MIDI stream,
as defined by the following rules:
o If no C-active parameter system transaction Control Change
commands appear in the session history, the E bit MUST be
set to 0.
o If the P header bit is set to 1, the E bit MUST be set to 0.
o If the most recent C-active parameter system transaction
Control Change command in the session history is for the
NPRN LSB or RPN LSB controller number, and this command
acts to complete the coding of the null parameter (MSB
value 0x7F, LSB value 0x7F), the E bit MUST be set to 0.
o Otherwise, an initiated transaction is in progress, and the
E bit MUST be set to 1.
The U, W, and Z header bits code properties that are shared by all
parameter logs in the list. By default, the U, W, and Z bits MUST be
set to 0.
However, if all parameter logs in the list code RPN parameters, the U
bit MAY be set to 1. If all parameter logs in the list code NRPN
parameters, the W bit MAY be set to 1. If the parameter numbers of all
RPN and NRPN logs in the list lie in the range 0-127 (and thus have an
MSB value of 0), the Z bit MAY be set to 1.
A.4.1 Log Inclusion Rules
Parameter logs code recovery information for a specific RPN or NRPN
parameter. In most cases, a parameter log MUST appear in the list if a
C-active command that forms a part of an initiated transaction for the
parameter appears in the checkpoint history. However, if the checkpoint
history only contains transaction Control Change commands for controller
numbers 98-101 that terminate the transaction, a log for the parameter
MAY be omitted from the list.
A log MAY appear in the list if an active Control Change command that
forms a part of an initiated transaction for the parameter appears in
the session history.
Multiple logs for the same RPN or NRPN parameter MUST NOT appear in the
log list.
Lazzaro/Wawrzynek [Page 43]
INTERNET-DRAFT 22 June 2004
Parameter logs MUST be ordered with respect to the relative recency of
transactions for the parameter. The log associated with the most recent
parameter transaction coded in Chapter M MUST appear at the end of the
list. The log associated with the oldest parameter transaction coded in
Chapter M MUST appear at the start of the list.
All other logs MUST be arranged with respect to these boundary logs, to
produce a list ordering that strictly reflects the relative session
history recency of the most recent transactions for the parameters coded
by Chapter M.
Parameter logs associated with the RPN or NRPN null parameter (LSB =
0x7F, MSB = 0x7F) MUST NOT appear in the log list. Chapter M uses the E
header bit (Figure A.4.1) and the log list ordering rules to code null
parameter semantics.
The ch_active MIME parameter (Appendix C.1.3) may be used to change
parameter log inclusion semantics, to support renderers that exclude
certain RPN parameters from the semantics of Control Change commands for
controller 121 (Reset All Controllers). Appendix C.1.3 defines how
ch_active modifies Chapter M semantics.
Parameter logs for RPN and NRPN parameters that are assigned to the
ch_never MIME parameter (Appendix C.1.3) MUST appear in the list if a C-
active command that forms a part of an initiated transaction for the
parameter appears in the checkpoint history, and if the loss of this
command would lead to the corruption of the parameter transaction
system. Otherwise, logs for RPN and NRPN parameters assigned to
ch_never MAY be omitted from the list.
A.4.2 Log Coding Rules
Figure A.4.2 shows the parameter log structure of Chapter M.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 8 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| PNUM-LSB |Q| PNUM-MSB |J|K|L|M|N|T|V|A| Fields ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.4.2 -- Parameter log format
The log begins with a header, whose default size (as shown in Figure
A.4.2) is 3 octets. If the Q header bit is set to 0, the log encodes an
RPN parameter. If Q = 1, the log encodes an NRPN parameter. The 7-bit
PNUM-MSB and PNUM-LSB fields code the parameter number, and reflect the
Lazzaro/Wawrzynek [Page 44]
INTERNET-DRAFT 22 June 2004
Control Change command data values for controllers 99 and 98 (for NRPNs)
or 101 and 100 (for RPNs).
The J, K, L, M, and N header bits form a Table of Contents (TOC) for the
log, and signal the presence of fixed-sized fields that follow the
header. A header bit that is set to 1 codes the presence of a field in
the log. The ordering of fields in the log follows the ordering of the
header bits in the TOC. Appendices A.4.2.1-2 define the fields
associated with each TOC header bit.
The T, V, and A header bits code information about the parameter log,
but are NOT part of the TOC. A set T, V, or A bit does NOT signal the
presence of any parameter log field.
If the rules in Appendix A.4.1 state that a log for a given parameter
MUST appear in Chapter M, the log MUST code sufficient information to
protect the parameter from the loss of C-active parameter transaction
Control Change commands in the checkpoint history.
An exception to the rule applies for logs associated with parameters
that are assigned to the ch_never MIME parameter (Appendix C.1.3). For
these parameters, the log MUST code sufficient information to protect
the transaction system, not the coded parameter itself.
Parameter logs provide two tools for parameter protection: the value
tool and the count tool. Depending on the semantics of the parameter,
senders may use either tool, both tools, or neither tool to protect a
given parameter.
The value tool codes information a receiver may use to determine the
current value of an RPN or NRPN parameter. If a parameter log uses the
value tool, the V header bit MUST be set to 1, and the semantics defined
in Appendices A.4.2.1 for setting the J, K, L, and M TOC bits MUST be
followed. If a parameter log does not use the value tool, the V bit
MUST be set to 0, and the J, K, L, and M TOC bits MUST also be set to 0.
The count tool codes the number of transactions for an RPN or NRPN
parameter. If a parameter log uses the count tool, the T header bit
MUST be set to 1, and the semantics defined in Appendices A.4.2.2 for
setting the N TOC bit MUST be followed. If a parameter log does not use
the count tool, the T bit and the N TOC bit MUST be set to 0.
The A header bit codes the level of protection provided by the value and
count tools. If the log parameter is assigned to the ch_active MIME
parameter (Appendix C.1.3), the A header bit MUST be set to 1, coding
the elevated protection level of the parameter. Otherwise, the A header
bit MUST be set to 0, coding the standard protection level.
Lazzaro/Wawrzynek [Page 45]
INTERNET-DRAFT 22 June 2004
In many cases, all parameters coded in the log list are of one type (RPN
and NRPN), and all parameter numbers lie in the range 0-127. As
described in Appendix A.4.1, senders MAY signal this condition by
setting the top-level Chapter M header bit Z to 1 (to code the
restricted range) AND by setting the U or W bit to 1 (to code the
parameter type).
If the top-level Chapter M header codes Z = 1 and either U = 1 or W = 1,
all logs in the parameter log list MUST use a modified header format.
This modification deletes bits 8-15 of the bitfield shown in Figure
A.4.2, to yield a 2-octet header. The values of the deleted PNUM-MSB
and Q fields may be inferred from the U, W, and Z bit values.
A.4.2.1 The Value Tool
The value tool uses several fields to track the value of an RPN or NRPN
parameter.
The J TOC bit codes the presence of the octet shown in Figure A.4.3 in
the field list.
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|X| ENTRY-MSB |
+-+-+-+-+-+-+-+-+
Figure A.4.3 -- ENTRY-MSB field
The 7-bit ENTRY-MSB field codes the data value of the most recent active
Control Change command for controller number 6 (Data Entry MSB) in the
session history that appears in a transaction for the log parameter. If
the command coded by ENTRY-MSB precedes the most recent Control Change
command for controller 121 (Reset All Controllers) in the session
history, the X bit associated with the ENTRY-MSB field MUST be set to 1.
Otherwise, the X bit MUST be set to 0.
Lazzaro/Wawrzynek [Page 46]
INTERNET-DRAFT 22 June 2004
The K TOC bit codes the presence of the octet shown in Figure A.4.4 in
the field list.
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|X| ENTRY-LSB |
+-+-+-+-+-+-+-+-+
Figure A.4.4 -- ENTRY-LSB field
The 7-bit ENTRY-LSB field codes the data value of the most recent active
Control Change command for controller number 38 (Data Entry LSB) in the
session history that appears in a transaction for the log parameter. If
the command coded by ENTRY-LSB precedes the most recent Control Change
command for controller 121 (Reset All Controllers) in the session
history, the X bit associated with the ENTRY-LSB field MUST be set to 1.
Otherwise, the X bit MUST be set to 0.
A parameter log that uses the value tool MUST include the ENTRY-MSB
field (and its associated X bit) if an active Control Change command for
controller number 6 appears in the checkpoint history. A parameter log
that uses the value tool MUST include the ENTRY-LSB field (and its
associated X bit) if an active Control Change command for controller
number 38 appears in the checkpoint history.
However, the ENTRY-LSB field (and its associated X bit) MUST NOT appear
in a parameter log if the Control Change command associated with the
ENTRY-LSB precedes a Control Change command for controller number 6
(Data Entry MSB) in the session history that appears in a transaction
for the log parameter.
If the A header bit is set to 0, the ENTRY-LSB and ENTRY-MSB fields (and
associated X bits) MAY be omitted from the parameter log if the commands
associated with the fields precede the most recent active Control Change
command for controller 121 (Reset All Controllers) in the session
history.
Lazzaro/Wawrzynek [Page 47]
INTERNET-DRAFT 22 June 2004
The L TOC bit codes the presence of the octets shown in Figure A.4.5 in
the field list.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|X|G| A-BUTTON |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.4.5 -- A-BUTTON field
The 14-bit A-BUTTON field codes a count of the number of active Control
Change commands for controller numbers 96 and 97 (Data Increment and
Data Decrement) in the session history that appear in a transaction for
the log parameter.
The M TOC bit codes the presence of the octets shown in Figure A.4.6 in
the field list.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|G| C-BUTTON |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.4.6 -- C-BUTTON field
The 14-bit C-BUTTON field has semantics identical to A-BUTTON, except
that Data Increment and Data Decrement Control Change commands that
precede the most recent Control Change command for controller 121 (Reset
All Controllers) in the session history are not counted.
For both A-BUTTON and C-BUTTON, Data Increment and Data Decrement
Control Change commands are not counted if they precede Control Changes
commands for controller numbers 6 (Data Entry MSB) or 38 (Data Entry
LSB) that appear in a transaction for the log parameter in the session
history.
The A-BUTTON and C-BUTTON fields are interpreted as unsigned integers,
and the G bit associated the field codes the sign of the integer (G = 1
for positive or zero, G = 0 for negative).
To compute and code the count value, initialize the count value to 0,
add 1 for each qualifying Data Increment command, subtract 1 for each
Lazzaro/Wawrzynek [Page 48]
INTERNET-DRAFT 22 June 2004
qualifying Data Decrement command, and limit the magnitude of the final
count to 16383. The G bit codes the sign of the count, and the A-BUTTON
or C-BUTTON field codes the magnitude of the count.
For the A-BUTTON field, if the most recent qualified Data Increment or
Data Decrement command precedes the most recent Control Change command
for controller 121 (Reset All Controllers) in the session history, the X
bit associated with A-BUTTON field MUST be set to 1. Otherwise, the X
bit MUST be set to 0.
A parameter log that uses the value tool MUST include the A-BUTTON and
C-BUTTON fields if an active Control Change command for controller
numbers 96 or 97 appears in the checkpoint history. However, to improve
coding efficiency, this rule has several exceptions:
o If the A header bit is set to 0, the A-BUTTON field (and
associated X and G bits) MAY be omitted from the log.
o If the log includes the A-BUTTON field, and if the X bit of
the A-BUTTON field is set to 1, the C-BUTTON field (and its
associated R and G bits) MAY be omitted from the log.
o If the log includes the A-BUTTON field, and if the A-BUTTON
and C-BUTTON fields (and their associated G bits) code identical
values, the C-BUTTON field (and its associated R and G bits)
MAY be omitted from the log.
A.4.2.2 The Count Tool
The count tool tracks the number of transactions for an RPN or NRPN
parameter. The N TOC bit codes the presence of the octet shown in
Figure A.4.7 in the field list.
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|X| COUNT |
+-+-+-+-+-+-+-+-+
Figure A.4.7 -- COUNT field
The 7-bit COUNT codes the number of initiated transactions for the log
parameter that appear in the session history. Initiated transactions
are counted if they contain one or more active Control Change commands,
including commands for controllers 98-101 that initiate the parameter
transaction.
Lazzaro/Wawrzynek [Page 49]
INTERNET-DRAFT 22 June 2004
If the most recent counted transaction precedes the most recent Control
Change command for controller 121 (Reset All Controllers) in the session
history, the X bit associated with the COUNT field MUST be set to 1.
Otherwise, the X bit MUST be set to 0.
Transaction counting is performed modulo 128. The transaction count is
set to 0 at the start of a session, and is reset to 0 whenever a Reset
State command (Appendix A.1) appears in the session history.
A parameter log that uses the count tool MUST include the COUNT field
(and its associated X bit) if a command that increments the transaction
count (modulo 128) appears in the checkpoint history. However, if the A
header bit is set to 0, the COUNT field (and its associated X bit) MAY
be omitted from the log if the command precedes the most recent active
Control Change command for controller 121 (Reset All Controllers) in the
session history.
A.5 Chapter W: MIDI Pitch Wheel
A channel journal MUST contain Chapter W if an active MIDI Pitch Wheel
(0xE) command appears in the checkpoint history. Figure A.5.1 shows the
format for Chapter W.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| FIRST |R| SECOND |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.5.1 -- Chapter W format
The chapter has a fixed size of 16 bits. The FIRST and SECOND fields
are the 7-bit values of the first and second data octets of the most
recent active Pitch Wheel command in the session history.
Lazzaro/Wawrzynek [Page 50]
INTERNET-DRAFT 22 June 2004
A.6 Chapter N: MIDI NoteOff and NoteOn
In this Appendix, we consider NoteOn commands with zero velocity to be
NoteOff commands. Readers may wish to review the Appendix A.1
definition of "N-active commands" before reading this Appendix.
A channel journal MUST contain Chapter N if an N-active MIDI NoteOn
(0x9) or NoteOff (0x8) command appears in the checkpoint history.
Figure A.6.1 shows the format for Chapter N.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 8 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|B| LEN | LOW | HIGH |S| NOTENUM |Y| VELOCITY |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NOTENUM |Y| VELOCITY | .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OFFBITS | OFFBITS | .... | OFFBITS |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.6.1 -- Chapter N format
Chapter N consists of a 2-octet header, followed by least one of the
following data structures:
o A list of note logs to code NoteOn commands.
o A NoteOff bitfield structure to code NoteOff commands.
We define the header bitfield semantics in Appendix A.6.1. We define
the note log semantics and the NoteOff bitfield semantics in Appendix
A.6.2.
If one or more N-active NoteOn or NoteOff commands in the checkpoint
history reference a note number, the note number MUST be coded in either
the note log list or the NoteOff bitfield structure.
The note log list MUST contain an entry for all note numbers whose most
recent checkpoint history appearance is in an N-active NoteOn command.
The NoteOff bitfield structure MUST contain a set bit for all note
numbers whose most recent checkpoint history appearance is in an N-
active NoteOff command.
A note number MUST NOT be coded in both structures.
All note logs and NoteOff bitfield set bits MUST code the most recent N-
active NoteOn or NoteOff reference to a note number in the session
Lazzaro/Wawrzynek [Page 51]
INTERNET-DRAFT 22 June 2004
history.
A.6.1 Header Structure
The header for Chapter N, shown in Figure A.6.2, codes the size of the
note list and bitfield structures.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|B| LEN | LOW | HIGH |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.6.2 -- Chapter N header
The LEN field, a 7-bit integer value, codes the number of 2-octet note
logs in the note list. Zero is a valid value for LEN, and codes an
empty note list.
The 4-bit LOW and HIGH fields code the number of OFFBITS octets that
follow the note log list. LOW and HIGH are unsigned integer values. If
LOW <= HIGH, there are (HIGH - LOW + 1) OFFBITS octets in the chapter.
The value pairs (LOW = 15, HIGH = 0) and (LOW = 15, HIGH = 1) code an
empty NoteOff bitfield structure (i.e. no OFFBITS octets). Other (LOW >
HIGH) value pairs MUST NOT appear in the header.
The B bit provides S-bit functionality (Appendix A.1) for the NoteOff
bitfield structure. By default, the B bit MUST be set to 1. However,
if the MIDI command section of the previous packet (packet I - 1, with I
as defined in Appendix A.1) includes a NoteOff command for the channel,
the B bit MUST be set to 0. If the B bit is set to 0, the higher-level
recovery journal elements that contain Chapter N MUST have S bits that
are set to 0, including the top-level journal header.
The LEN value of 127 codes a note list length of 127 or 128 note logs,
depending on the values of LOW and HIGH. If LEN = 127, LOW = 15, and
HIGH = 0, the note list holds 128 note logs, and the NoteOff bitfield
structure is empty. For other values of LOW and HIGH, LEN = 127 codes
that the note list contains 127 note logs. In this case, the chapter
has (HIGH - LOW + 1) NoteOff OFFBITS octets if LOW <= HIGH, and has no
OFFBITS octets if LOW = 15 and HIGH = 1.
Lazzaro/Wawrzynek [Page 52]
INTERNET-DRAFT 22 June 2004
A.6.2 Note Structures
Figure A.6.3 shows the 2-octet note log structure.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NOTENUM |Y| VELOCITY |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.6.3 -- Chapter N note log
The 7-bit NOTENUM field codes the note number for the log. A note
number MUST NOT be represented by multiple note logs in the note list.
The 7-bit VELOCITY field codes the velocity value for the most recent N-
active NoteOn command for the note number in the session history.
VELOCITY is never zero; NoteOn commands with zero velocity are coded as
NoteOff commands in the NoteOff bitfield structure.
The note log does not code the execution time of the NoteOn command.
However, the Y bit codes a hint from the sender about the NoteOn
execution time. The Y bit codes a recommendation to play (Y = 1) or
skip (Y = 0) the NoteOn command recovered from the note log. In a
normative sense, the Y bit is set to 1 if the sender considers the
command coded by the log to be simultaneous with the RTP timestamp of
the packet that contains the log. In all other cases, the Y bit is set
to 0.
Figure A.6.1 shows the NoteOff bitfield structure, as the list of
OFFBITS octets at the end of the chapter. A NoteOff OFFBITS octet codes
NoteOff information for eight consecutive MIDI note numbers, with the
most-significant bit representing the lowest note number. The most-
significant bit of the first OFFBITS octet codes the note number 8*LOW;
the most-significant bit of the last OFFBITS octet codes the note number
8*HIGH.
A set bit codes a NoteOff command for the note number. In the most
efficient coding for the NoteOff bitfield structure, the first and last
octets of the structure contain at least one set bit. Note that Chapter
N does not code NoteOff velocity data.
Lazzaro/Wawrzynek [Page 53]
INTERNET-DRAFT 22 June 2004
A.7 Chapter E: MIDI Note Command Extras
Readers may wish to review the Appendix A.1 definition of "N-active
commands" before reading this Appendix. In this Appendix, a NoteOn
command with a velocity of 0 is considered to be a NoteOff command with
a release velocity value of 64.
Chapter E encodes recovery information about MIDI NoteOn (0x9) and
NoteOff (0x8) command features that rarely appear in MIDI streams.
Receivers use Chapter E to reduce transient artifacts for streams where
several NoteOn commands appear for a note number without an intervening
NoteOff. Receivers also use Chapter E to reduce transient artifacts for
streams that use NoteOff release velocity. Chapter E supplements the
note information coded in Chapter N (Appendix A.6).
Figure A.7.1 shows the format for Chapter E.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 8 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| LEN |S| NOTENUM |V| COUNT/VEL |S| NOTENUM |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V| COUNT/VEL | .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.7.1 -- Chapter E format
The chapter consists of a 1-octet header, followed by a variable length
list of 2-octet note logs. Appendix A.7.1 defines the bitfield format
for a note log.
The log list MUST contain at least one note log. The 7-bit LEN header
field codes the number of note logs in the list, minus one. A channel
journal MUST contain Chapter E if the rules defined in this Appendix
require that one or more note logs appear in the list.
Lazzaro/Wawrzynek [Page 54]
INTERNET-DRAFT 22 June 2004
A.7.1 Note Log Format
Figure A.7.2 reproduces the note log structure of Chapter E.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NOTENUM |V| COUNT/VEL |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.7.2 -- Chapter E note log
A note log codes information about the MIDI note number coded by the
7-bit NOTENUM field. The nature of the information depends on the value
of the V flag bit.
If the V bit is set to 1, the COUNT/VEL field codes the release velocity
value for the most recent N-active NoteOff command for the note number
that appears in the session history.
If the V bit is set to 0, the COUNT/VEL field codes a reference count of
the number of NoteOn and NoteOff commands for the note number that
appear in the session history.
The reference count is set to 0 at the start of the session. NoteOn
commands increment the count by 1. NoteOff commands decrement the count
by 1. However, a decrement that generates a negative count value is not
performed.
If the reference count is in the range 0-126, the 7-bit COUNT/VEL field
codes an unsigned integer representation of the count. If the count is
greater or equal to 127, COUNT/VEL is set to 127.
By default, the count is reset to 0 whenever a Reset State command
(Appendix A.1) appears in the session history, and whenever MIDI Control
Change commands for controller numbers 123-127 (numbers with All Notes
Off semantics) or 120 (All Sound Off) appear in the session history.
The ch_active MIME parameter may be used to change count reset behavior,
as described in Appendix C.1.3.
A.7.2 Log Inclusion Rules
If the most recent N-active NoteOn or NoteOff command for a note number
in the checkpoint history is a NoteOff command with a release velocity
value other than 64, a note log whose V bit is set to 1 MUST appear in
Chapter E for the note number.
Lazzaro/Wawrzynek [Page 55]
INTERNET-DRAFT 22 June 2004
If the most recent N-active NoteOn or NoteOff command for a note number
in the checkpoint history is a NoteOff command, and if the reference
count for the note number is greater than 0, a note log whose V bit is
set to 0 MUST appear in Chapter E for the note number.
If the most recent N-active NoteOn or NoteOff command for a note number
in the checkpoint history is a NoteOn command, and if the reference
count for the note number is greater than 1, a note log whose V bit is
set to 0 MUST appear in Chapter E for the note number.
At most two note logs MAY appear in Chapter E for a note number: one log
whose V bit is set to 0, and one log whose V bit is set to 1.
Chapter E codes a maximum of 128 note logs. If the log inclusion rules
yield more than 128 REQUIRED logs, note logs whose V bit is set to 1
MUST be dropped from Chapter E in order to reach the 128-log limit.
Note logs whose V bit is set to 0 MUST NOT be dropped.
Most MIDI streams do not use NoteOn and NoteOff commands in ways that
would trigger the log inclusion rules. For these streams, Chapter E
would never be REQUIRED to appear in a channel journal.
The ch_never MIME parameter (Appendix C.1.3) may be used to configure
the log inclusion rules for Chapter E.
A.8 Chapter T: MIDI Channel Aftertouch
A channel journal MUST contain Chapter T if an N-active MIDI Channel
Aftertouch (0xD) command appears in the checkpoint history. Figure
A.8.1 shows the format for Chapter T.
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|S| PRESSURE |
+-+-+-+-+-+-+-+-+
Figure A.8.1 -- Chapter T format
The chapter has a fixed size of 8 bits. The 7-bit PRESSURE field holds
the pressure value of the most recent N-active Channel Aftertouch
command in the session history.
Lazzaro/Wawrzynek [Page 56]
INTERNET-DRAFT 22 June 2004
A.9 Chapter A: MIDI Poly Aftertouch
A channel journal MUST contain Chapter A if an N-active Poly Aftertouch
(0xA) command appears in the checkpoint history. Figure A.9.1 shows the
format for Chapter A.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 8 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| LEN |S| NOTENUM |X| PRESSURE |S| NOTENUM |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|X| PRESSURE | .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.9.1 -- Chapter A format
The chapter consists of a 1-octet header, followed by a variable length
list of 2-octet note logs. A note log MUST appear for a note number if
an N-active Poly Aftertouch command for the note number appears in the
checkpoint history. A note number MUST NOT be represented by multiple
note logs in the note list.
The 7-bit LEN field codes the number of note logs in the list, minus
one. Figure A.9.2 reproduces the note log structure of Chapter A.
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NOTENUM |X| PRESSURE |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.9.2 -- Chapter A note log
The 7-bit PRESSURE field codes the pressure value of the most recent N-
active Poly Aftertouch command in the session history for the MIDI note
number coded in the 7-bit NOTENUM field.
As a rule, the X bit MUST be set to 0. However, the X bit MUST be set
to 1 if the command coded by the log appears before one of the following
commands in the session history: MIDI Control Change numbers 123-127
(numbers with All Notes Off semantics) or 120 (All Sound Off). This
condition occurs only if the session description uses the ch_active MIME
parameter (Appendix C.1.3) to alter the semantics of Chapter A.
Lazzaro/Wawrzynek [Page 57]
INTERNET-DRAFT 22 June 2004
B. The Recovery Journal System Chapters
B.1 System Chapter D: Simple System Commands
The system journal MUST contain Chapter D if an active MIDI Reset
(0xFF), MIDI Tune Request (0xF6), MIDI Song Select (0xF3), undefined
MIDI System Common (0xF4 and 0xF5), or undefined MIDI System Real-time
(0xF9 and 0xFD) command appears in the checkpoint history.
Figure B.1.1 shows the variable-length format for Chapter D.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|B|G|H|J|K|Y|Z| Command logs ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.1.1 -- System Chapter D format
The chapter consists of a 1-octet header, followed by one or more
command logs. Header flag bits indicate the presence of command logs
for the Reset (B = 1), Tune Request (G = 1), Song Select (H = 1),
undefined System Common 0xF4 (J = 1), undefined System Common 0xF5 (K =
1), undefined System Real-time 0xF9 (Y = 1), or undefined System Real-
time 0xFD (Z = 1) commands.
Command logs appear in a list following the header, in the order that
the flag bits appear in the header.
Figure B.1.2 shows the 1-octet command log format for the Reset and Tune
Request commands.
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|S| COUNT |
+-+-+-+-+-+-+-+-+
Figure B.1.2 -- Command log for Reset and Tune Request
Chapter D MUST contain the Reset command log if an active Reset command
appears in the checkpoint history. The 7-bit COUNT field codes the
total number of Reset commands (modulo 128) present in the session
history.
Lazzaro/Wawrzynek [Page 58]
INTERNET-DRAFT 22 June 2004
Chapter D MUST contain the Tune Request command log if an active Tune
Request command appears in the checkpoint history. The 7-bit COUNT
field codes the total number of Tune Request commands (modulo 128)
present in the session history.
Figure B.1.3 shows the 1-octet command log format for the Song Select
command.
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|S| VALUE |
+-+-+-+-+-+-+-+-+
Figure B.1.3 -- Song Select command log format
Chapter D MUST contain the Song Select command log if an active Song
Select command appears in the checkpoint history. The 7-bit VALUE field
codes the song number of the most recent active Song Select command in
the session history.
B.1.1 Undefined System Commands
In this section, we define the Chapter D command logs for the undefined
System opcodes. [1] reserves the undefined System opcodes 0xF4, 0xF5,
0xF9, and 0xFD for future use. At the time of this writing, any MIDI
command stream that uses these opcodes is non-compliant with [1].
However, future versions of [1] may define these opcodes, and a few
products do use these opcodes in a non-compliant manner.
Figure B.1.4 shows the variable length command log format for the
undefined System Common commands (0xF4 and 0xF5).
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|C|V|L|R|R|DSZ| LENGTH | COUNT | VALUE ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LEGAL ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.1.4 -- Undefined System Common command log format
The command log codes a single opcode type (0xF4 or 0xF5, not both).
Lazzaro/Wawrzynek [Page 59]
INTERNET-DRAFT 22 June 2004
Chapter D MUST contain a command log if an active 0xF4 command appears
in the checkpoint history, and MUST contain an independent command log
if an active 0xF5 command appears in the checkpoint history.
Chapter D consists of a two-octet header followed by a variable number
of data fields. Header flag bits indicate the presence of the COUNT
field (C = 1), the VALUE field (V = 1), and the LEGAL field (L = 1).
The 8-bit LENGTH field codes the size of the command log, and conforms
to semantics described in Appendix A.1.
The 2-bit DSZ field codes the number of data octets in the command
instance that appears most recently in the session history. If DSZ =
0-2, the command has 0-2 data octets. If DSZ = 3, the command has 3 or
more command data octets.
We now define the default rules for the use of the COUNT, VALUE, and
LEGAL fields. The session configuration tools defined in Appendix C.1.3
may be used to override this behavior.
If the DSZ field is set to 0, the command log MUST include the COUNT
field. The 8-bit COUNT field codes the total number of opcode commands
present in the session history, modulo 256.
If the DSZ field is set to 1-3, the command log MUST include the VALUE
field. The variable-length VALUE field codes a verbatim copy the data
octets for the most recent use of the opcode in the session history.
The most-significant bit of the final data octet MUST be set to 1, and
the most-significant bit of all other data octets MUST be set to 0.
The LEGAL field is reserved for future use. If an update to [1] defines
the 0xF4 or 0xF5 opcode, an IETF standards-track document MAY define the
LEGAL field to protect the opcode. Until such a document appears,
senders MUST NOT use the LEGAL field, and receivers MUST use the LENGTH
field to skip over the LEGAL field.
Lazzaro/Wawrzynek [Page 60]
INTERNET-DRAFT 22 June 2004
Figure B.1.5 shows the variable length command log format for the
undefined System Real-time commands (0xF9 and 0xFD).
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|C|L| LENGTH | COUNT | LEGAL ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.1.5 -- Undefined System Real-time command log format
The command log codes a single opcode type (0xF9 or 0xFD, not both).
Chapter D MUST contain a command log if an active 0xF9 command appears
in the checkpoint history, and MUST contain an independent command log
if an active 0xFD command appears in the checkpoint history.
Chapter D consists of a one-octet header followed by a variable number
of data fields. Header flag bits indicate the presence of the COUNT
field (C = 1) and the LEGAL field (L = 1). The 5-bit LENGTH field codes
the size of the command log, and conforms to semantics described in
Appendix A.1.
We now define the default rules for the use of the COUNT and LEGAL
fields. The session configuration tools defined in Appendix C.1.3 may
be used to override this behavior.
The 8-bit COUNT field codes the total number of opcode commands present
in the session history, modulo 256. By default, the COUNT field MUST be
present in the command log.
The LEGAL field is reserved for future use. If an update to [1] defines
the 0xF9 or 0xFD opcode, an IETF standards-track document MAY define the
LEGAL field to protect the opcode. Until such a document appears,
senders MUST NOT use the LEGAL field, and receivers MUST use the LENGTH
field to skip over the LEGAL field.
Finally, we note that some non-standard uses of the undefined System
Real-time opcodes act to implement non-compliant variants of the MIDI
sequencer system. In Appendix B.3.1, we describe resiliency tools for
the MIDI sequencer system that provide some protection in this case.
Lazzaro/Wawrzynek [Page 61]
INTERNET-DRAFT 22 June 2004
B.2 System Chapter V: Active Sense Command
The system journal MUST contain Chapter V if an active MIDI Active Sense
(0xFE) command appears in the checkpoint history. Figure B.2.1 shows
the format for Chapter V.
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|S| COUNT |
+-+-+-+-+-+-+-+-+
Figure B.2.1 -- System Chapter V format
The 7-bit COUNT field codes the total number of Active Sense commands
(modulo 128) present in the session history.
B.3 System Chapter Q: Sequencer State Commands
This Appendix describes Chapter Q, the system chapter for the MIDI
sequencer commands.
The system journal MUST contain Chapter Q if an active MIDI Song
Position Pointer (0xF2), MIDI Clock (0xF8), MIDI Start (0xFA), MIDI
Continue (0xFB) or MIDI Stop (0xFC) command appears in the checkpoint
history, and if the rules defined in this Appendix require a change in
the Chapter Q bitfield contents because of the command appearance.
Figure B.3.1 shows the variable-length format for Chapter Q.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|N|D|C|T| TOP | CLOCK | TIMETOOLS ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.3.1 -- System Chapter Q format
Chapter Q consists of a 1-octet header followed by several optional
fields, in the order shown in Figure B.3.1.
Lazzaro/Wawrzynek [Page 62]
INTERNET-DRAFT 22 June 2004
Header flag bits signal the presence of the 16-bit CLOCK field (C = 1)
and the 24-bit TIMETOOLS field (T = 1). The 3-bit TOP header field is
interpreted as an unsigned integer, as are CLOCK and TIMETOOLS. We
describe the TIMETOOLS field in Appendix B.3.1.
Chapter Q encodes the most recent state of the sequencer system.
Receivers use the chapter to re-synchronize the sequencer after a packet
loss episode. Chapter fields encode the on/off state of the sequencer,
the current position in the song, and the downbeat.
The N header bit encodes the relative occurrence of the Start, Stop, and
Continue commands in the session history. If an active Start or
Continue command appears most recently, the N bit MUST be set to 1. If
an active Stop appears most recently, or if no active Start, Stop, or
Continue commands appear in the session history, the N bit MUST be set
to 0.
The C header flag, the TOP header field, and the CLOCK field act to code
the current position in the sequence:
o If C = 1, the 3-bit TOP header field and the 16-bit
CLOCK field are combined to form the 19-bit unsigned quantity
65536*TOP + CLOCK. This value encodes the song position
in units of MIDI Clocks (24 clocks per quarter note),
modulo 524288.
o If C = 0, the song position is the start of the song.
The C = 0 position is identical to the position coded
by C = 1, TOP = 0, and CLOCK = 0, for the case where
the song position is less than 524288 MIDI clocks.
The C, TOP, and CLOCK fields MUST be set to code the current song
position, for both N = 0 and N = 1 conditions. If C = 0, the TOP field
MUST be set to 0. See [1] for a precise definition of a song position.
The D header bit encodes information about the downbeat, and acts to
qualify the song position coded by the C, TOP, and CLOCK fields.
If the D bit is set to 1, the song position represents the most recent
position in the sequence that has played. If D = 1, the next Clock
command (if N = 1) or the next (Continue, Clock) pair (if N = 0) acts to
increment the song position by one clock, and to play the updated
position.
If the D bit is set to 0, the song position represents a position in the
sequence that has not yet been played. If D = 0, the next Clock command
(if N = 1) or the next (Continue, Clock) pair (if N = 0) acts to play
the point in the song coded by the song position. The song position is
Lazzaro/Wawrzynek [Page 63]
INTERNET-DRAFT 22 June 2004
not incremented.
An example stream that uses D = 0 coding is one whose most recent
sequence command is a Start or Song Position Pointer command (both N = 1
conditions). However, it is also possible to construct examples where D
= 0 and N = 0. A Start command immediately followed by a Stop command
is coded in Chapter Q by setting C = 0, D = 0, N = 0, TOP = 0.
B.3.1 Non-compliant Sequencers
The Chapter Q description in this Appendix assumes that the sequencer
system counts off time with Clock commands, as mandated in [1].
However, a few non-compliant products do not use Clock commands to count
off time, but instead use non-standard methods.
Chapter Q uses the TIMETOOLS field to provide resiliency support for
these non-standard products. By default, the TIMETOOLS field MUST NOT
appear in Chapter Q, and the T header bit MUST be set to 0. The session
configuration tools described in Appendix C.1.3 may be used to select
TIMETOOLS coding.
Figure B.3.2 shows the format of the 24-bit TIMETOOLS field.
0 1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TIME |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.3.2 -- TIMETOOLS format
The TIME field is a 24-bit unsigned integer quantity, with units of
milliseconds. TIME codes an additive correction term for the song
position coded by the TOP, CLOCK, C fields. TIME is coded in network
byte order (big-endian).
A receiver computes the correct song position by converting TIME into
units of MIDI clocks and adding it to 65536*TOP + CLOCK (assuming C =
1). Alternatively, a receiver may convert 65536*TOP + CLOCK into
milliseconds (assuming C = 1) and add it to TIME. The downbeat (D
header bit) semantics defined in Appendix B.3 apply to the corrected
song position.
Lazzaro/Wawrzynek [Page 64]
INTERNET-DRAFT 22 June 2004
B.4 System Chapter F: MIDI Time Code Tape Position
This Appendix describes Chapter F, the system chapter for the MIDI Time
Code (MTC) commands. Readers may wish to review the Appendix A.1
definition of "finished/unfinished commands" before reading this
Appendix.
The system journal MUST contain Chapter F if an active System Common
Quarter Frame command (0xF1) or an active finished System Exclusive
(Universal Real Time) MTC Full Frame command (F0 7F cc 01 01 hr mn sc fr
F7) appears in the checkpoint history. Otherwise, the system journal
MUST NOT contain Chapter F.
Figure B.4.1 shows the variable-length format for Chapter F.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|C|P|Q|D|POINT| COMPLETE ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | PARTIAL ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+
Figure B.4.1 -- System Chapter F format
Chapter F holds information about recent MTC tape positions coded in the
session history. Receivers use Chapter F to re-synchronize the MTC
system after a packet loss episode.
Chapter F consists of a 1-octet header followed by several optional
fields, in the order shown in Figure B.4.1. The C and P header bits
form a Table of Contents (TOC), and signal the presence of the 32-bit
COMPLETE field (C = 1) and the 32-bit PARTIAL field (P = 1).
The Q header bit codes information about the COMPLETE field format. If
Chapter F does not contain a COMPLETE field, Q MUST be set to 0.
The D header bit codes the tape movement direction. If the tape is
moving forward, or if the tape direction is indeterminate, the D bit
MUST be set to 0. If the tape is moving in the reverse direction, the D
bit MUST be set to 1. In most cases, the ordering of commands in the
session history clearly defines the tape direction. However, a few
command sequences have an indeterminate direction (such as a session
history consisting of one Full Frame command).
Lazzaro/Wawrzynek [Page 65]
INTERNET-DRAFT 22 June 2004
The 3-bit POINT header field is interpreted as an unsigned integer.
Appendix B.4.1 defines how the POINT field codes information about the
contents of the PARTIAL field. If Chapter F does not contain a PARTIAL
field, POINT MUST be set to 7 (if D = 0) or 0 (if D = 1).
Chapter F MUST include the COMPLETE field if an active finished Full
Frame command appears in the checkpoint history, or if an active Quarter
Frame command that completes the encoding of a frame value appears in
the checkpoint history.
The COMPLETE field encodes the most recent active complete MTC frame
value that appears in the session history. This frame value may take
the form of a series of 8 active Quarter Frame commands (0xF1 0x0n
through 0xF1 0x7n for forward tape movement, 0xF1 0x7n through 0xF1 0x0n
for reverse tape movement), or may take the form of an active finished
Full Frame command.
If the COMPLETE field encodes a Quarter Frame command series, the Q
header bit MUST be set to 1, and the COMPLETE field MUST have the format
shown in Figure B.4.2. The 4-bit fields MT0 through MT7 code the data
(lower) nibble for the Quarter Frame commands for Message Type 0 through
Message Type 7 [1]. These nibbles encode a complete frame value, in
addition to fields reserved for future use by [1].
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MT0 | MT1 | MT2 | MT3 | MT4 | MT5 | MT6 | MT7 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.4.2 -- COMPLETE field format, Q = 1
In this usage, the frame value encoded in the COMPLETE field MUST be
offset by 2 frames (relative to the frame value encoded in the Quarter
Frame commands) if the frame value codes a 0xF1 0x0n through 0xF1 0x7n
command sequence. This offset compensates for the two-frame latency of
the Quarter Frame encoding for forward tape movement. No offset is
applied if the frame value codes a 0xF1 0x7n through 0xF1 0x0n Quarter
Frame command sequence.
The most recent active complete MTC frame value may alternatively be
encoded by an active finished Full Frame command. In this case, the Q
header bit MUST be set to 0, and the COMPLETE field MUST have format
shown in Figure B.4.3. The HR, MN, SC, and FR fields correspond to the
hr, mn, sc, and fr data octets of the Full Frame command.
Lazzaro/Wawrzynek [Page 66]
INTERNET-DRAFT 22 June 2004
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| HR | MN | SC | FR |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.4.3 -- COMPLETE field format, Q = 0
B.4.1 Partial Frames
The most recent active session history command that encodes MTC frame
value data may be a Quarter Frame command other than a forward-moving
0xF1 0x7n command (which completes a frame value for forward tape
movement) or a reverse-moving 0xF1 0x1n command (which completes a frame
value for reverse tape movement).
We consider this type of Quarter Frame command to be associated with a
partial frame value. The Quarter Frame sequence that defines a partial
frame value MUST either start at Message Type 0 and increment
contiguously to an intermediate Message Type less than 7, or start at
Message Type 7 and decrement contiguously to an intermediate Message
type greater than 0. A Quarter Frame command sequence that does not
follow this pattern is not associated with a partial frame value.
Chapter F MUST include a PARTIAL field if the most recent active command
in the checkpoint history that encodes MTC frame value data is a Quarter
Frame command that is associated with a partial frame value. Otherwise,
Chapter F MUST NOT include a PARTIAL field.
The partial frame value consists of the data (lower) nibbles of the
Quarter Frame command sequence. The PARTIAL field codes the partial
frame value, using the format shown in Figure B.4.2. Message Type
fields that are not associated with a Quarter Frame command MUST be set
to 0.
The POINT header field indicates the Message Type fields in the PARTIAL
field code valid data. If P = 1, the POINT field MUST encode the
unsigned integer value formed by the lower 3 bits of the upper nibble of
the data value of the most recent active Quarter Frame command in the
session history. If D = 0 and P = 1, POINT MUST take on a value in the
range 0-6. If D = 1 and P = 1, POINT MUST take on a value in the range
1-7.
If D = 0, MT fields (Figure B.4.2) in the inclusive range 0 up to and
including the POINT value encode the partial frame value. If D = 1, MT
fields in the inclusive range 7 down to and including the POINT value
encode the partial frame value. Note that unlike the COMPLETE field
Lazzaro/Wawrzynek [Page 67]
INTERNET-DRAFT 22 June 2004
encoding, senders MUST NOT add a 2-frame offset to the partial frame
value encoded in PARTIAL.
For the default semantics, if a recovery journal contains Chapter F, and
if the session history codes a legal [1] series of Quarter Frame and
Full Frame commands, the chapter always contains a COMPLETE or a PARTIAL
field (and may contain both fields). Thus, a one-octet Chapter F (C = P
= 0) always codes the presence of an illegal command sequence in the
session history (under some conditions, the C = 1, P = 0 condition may
also code the presence of an illegal command sequence). The illegal
command sequence conditions are transient in nature, and usually
indicate that a Quarter Frame command sequence began with an
intermediate Message Type.
B.5 System Chapter X: System Exclusive
This Appendix describes Chapter X, the system chapter for MIDI System
Exclusive (SysEx) commands (0xF0). Readers may wish to review the
Appendix A.1 definition of "finished/unfinished commands" before reading
this Appendix.
Chapter X consists of a list of one or more command logs. Each log in
the list codes information about a specific finished or unfinished SysEx
command that appears in the session history. The system journal MUST
contain Chapter X if the rules defined in Appendix B.5.2 require that
one or more logs appear in the list.
The log list is NOT preceded by a header. Instead, each log implicitly
encodes its own length. Given the length of the N'th list log, the
presence of the (N+1)'th list log may be inferred from the LENGTH field
of the system journal header (Figure 10 in Section 5 of the main text).
The ordering of logs in the list MUST reflect the recency of the SysEx
commands coded in the list. The log that codes the most recent listed
command in the session history appears at the end of the list, preceded
by a log that codes an older command, preceded by a log that codes an
even older command, etc.
Lazzaro/Wawrzynek [Page 68]
INTERNET-DRAFT 22 June 2004
B.5.1 Chapter Format
Figure B.5.1 shows the bitfield format for the Chapter X command log.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|T|C|F|D|L|STA| TCOUNT | COUNT | FIRST ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DATA ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.5.1 -- Chapter X command log format
A Chapter X command log consists of a 1-octet header, followed by the
optional TCOUNT, COUNT, FIRST, and DATA fields.
The T, C, F, and D header bits act as a Table of Contents (TOC) for the
log. If T is set to 1, the 1-octet TCOUNT field appears in the log. If
C is set to 1, the 1-octet COUNT field appears in the log. If F is set
to 1, the variable-length FIRST field appears in the log. If D is set
to 1, the variable-length DATA field appears in the log.
The L header bit sets the coding tool for the log. We define the log
coding tools in Appendix B.5.2.
The STA field codes the status of the command coded by the log. The
2-bit STA value is interpreted as an unsigned integer. If STA is 0, the
log codes an unfinished command. Non-zero STA values code different
classes of finished commands. An STA value of 1 codes a cancelled
command, an STA value of 2 codes a command that uses the "dropped F7"
construction, and an STA value of 3 codes all other finished commands.
Section 3.2 in the main text describes cancelled and "dropped F7"
commands.
The S bit (Appendix A.1) of the first log in the list acts as the S bit
for Chapter X. For the other logs in the list, the S bit refers to the
log itself. The value of the "phantom" S bit associated with the first
log is defined by the following rules:
o If the list codes one log, the phantom S-bit value is
the same as the Chapter X S-bit value.
o If the list codes multiple logs, the phantom S-bit value is
the logical OR of the S-bit value of the first and second
Lazzaro/Wawrzynek [Page 69]
INTERNET-DRAFT 22 June 2004
command logs in the list.
In all other respects, the S bit follows the semantics defined in
Appendix A.1.
The FIRST field (present if F = 1) encodes a variable-length unsigned
integer value that sets the coverage of the DATA field. The FIRST field
consists of an octet whose most-significant bit is set to 0, optionally
preceded by one or more octets whose most-significant bit is set to 1.
The algorithm shown in Figure B.5.2 decodes this format into an unsigned
integer, to yield the value dec(FIRST). FIRST uses a variable-length
encoding because dec(FIRST) references a data octet in a SysEx command,
and a SysEx command may contain an arbitrary number of data octets.
One-Octet FIRST value:
Encoded form: 0ddddddd
Decoded form: 00000000 00000000 00000000 0ddddddd
Two-Octet FIRST value:
Encoded form: 1ccccccc 0ddddddd
Decoded form: 00000000 00000000 00cccccc cddddddd
Three-Octet FIRST value:
Encoded form: 1bbbbbbb 1ccccccc 0ddddddd
Decoded form: 00000000 000bbbbb bbcccccc cddddddd
Four-Octet FIRST value:
Encoded form: 1aaaaaaa 1bbbbbbb 1ccccccc 0ddddddd
Decoded form: 0000aaaa aaabbbbb bbcccccc cddddddd
Figure B.5.2 -- Decoding FIRST field formats
The DATA field (present if D = 1) encodes a modified version of the data
octets of the SysEx command coded by the log. Status octets MUST NOT be
coded in the DATA field.
Lazzaro/Wawrzynek [Page 70]
INTERNET-DRAFT 22 June 2004
If F = 0, the DATA field begins with the first data octet of the SysEx
command, and includes all subsequent data octets for the command that
appear in the session history. If F = 1, the DATA field begins with the
(dec(FIRST) + 1)'th data octet of the SysEx command, and includes all
subsequent data octets for the command that appear in the session
history. Note that the word "command" in the descriptions above refers
to the original SysEx command as it appears in the source MIDI data
stream, not to a particular MIDI list SysEx command segment.
The length of the DATA field is coded implicitly, using the most-
significant bit of each octet. The most-significant bit of the final
octet of the DATA field MUST be set to 1. The most significant bit of
all other DATA octets MUST be set to 0. This coding method relies on
the fact that the most-significant bit of a MIDI data octet is 0 by
definition. Apart from this length-coding modification, the DATA field
encodes a verbatim copy of all data octets it encodes.
B.5.2 Log Inclusion Semantics
By default, the Chapter X log list MUST code sufficient information to
protect the rendered MIDI performance from indefinite artifacts caused
by the loss of all finished or unfinished active SysEx commands that
appear in the checkpoint history (excluding finished MTC Full Frame
commands, which are coded in Chapter F (Appendix B.4)).
Chapter X offers two tools to protect SysEx commands: the "recency" tool
and the "list" tool.
To protect a specific SysEx command type with the recency tool, senders
MUST code a log in the log list for the most recent finished active
instance of the command type that appears in the checkpoint history.
Additionally, if an unfinished active instance of the command type
appears in the checkpoint history, senders MUST code a log in the log
list for the unfinished command instance. The L header bit of both
command logs MUST be set to 0.
To protect a specific SysEx command type with the list tool, senders
MUST code a log in the Chapter X log list for each finished or
unfinished active instance of the command type that appears in the
checkpoint history. The L header bit of list tool command logs MUST be
set to 1.
As a rule, a log REQUIRED by the list or recency tool MUST include a
DATA field that codes all data octets that appear in the checkpoint
history for the SysEx command instance associated with the log. The
FIRST field MAY be used to configure a DATA field that minimally meets
this requirement.
Lazzaro/Wawrzynek [Page 71]
INTERNET-DRAFT 22 June 2004
An exception to this rule applies to cancelled commands (defined in
Section 3.2). REQUIRED command logs associated with cancelled commands
MAY be coded with no DATA field. However, if DATA appears in the log,
DATA MUST code all data octets that appear in the checkpoint history for
the command associated with the log.
The recency and list tools are specified in terms of a protected
"command type" covered by the log. Two commands with identical data
octet fields (same number of data octets, same value for each data
octet) always have the same type. In addition, two instances of the
same command that set or report the value of the same parameter have the
same type. Otherwise, the two commands have different types.
As defined by the preceding text in this section, by default all
finished or unfinished active SysEx commands that appear in the
checkpoint history (excluding finished MTC Full Frame commands) MUST be
protected by the list tool or the recency tool.
For some MIDI source streams, this default yields a Chapter X whose size
is too large. For example, imagine that a sender begins to transcode a
SysEx command with 10,000 data octets onto a UDP RTP stream "on the
fly", by sending SysEx command segments as soon as data octets are
delivered by the MIDI source. After 1000 octets have been sent, the
expansion of Chapter X yields an RTP packet that is too large to fit in
the Maximum Transmission Unit (MTU) for the stream.
In this situation, if a sender uses the closed-loop sending policy for
SysEx commands, the RTP packet size may always be capped by stalling the
stream. In a stream stall, once the packet reaches a maximum size, the
sender refrains from sending new packets with non-empty MIDI Command
Sections until receiver feedback permits the trimming of Chapter X. If
the stream permits arbitrary commands to appear between SysEx segments
(selectable during configuration using the tools defined in Appendix
C.1.3), the sender may stall the SysEx segment stream but continue to
code other commands in the MIDI list.
Stalls are a workable but sub-optimal solution to Chapter X size issues.
As an alternative to stalls, senders SHOULD take preemptive action
during session configuration to reduce the anticipated size of Chapter
X, using the methods described below:
o Partitioned transport. Appendix C.4 provides tools
for partitioning a MIDI source into several streams.
Senders may use these tools to map a MIDI source
into a low-latency UDP RTP stream (for channel commands
and short SysEx commands) and a reliable [22] TCP stream
(for bulk-data SysEx commands). The ch_unused and
ch_active parameters (Appendix C.1) may be used to
Lazzaro/Wawrzynek [Page 72]
INTERNET-DRAFT 22 June 2004
communicate the nature of the SysEx command partition.
As TCP is reliable, the RTP MIDI TCP stream would not
use the recovery journal. To minimize transmission
latency for short SysEx commands, senders may begin
segmental transmission for all SysEx commands over the
UDP stream, and then cancel the UDP transmission of long
commands (using tools described in Section 3.2) and
resend the commands over the TCP stream.
o Selective protection. Journal protection may not be
necessary for all SysEx commands in a stream. The
ch_never parameter (Appendix C.1) may be used to
communicate which SysEx commands are excluded from
Chapter X. Alternatively, senders may communicate
the deletion of certain SysEx commands from the
stream (and thus Chapter X), via the ch_unused parameter.
B.5.3 TCOUNT and COUNT fields
If the T header bit is set to 1, the 8-bit TCOUNT field appears in the
command log. If the C header bit is set to 1, the 8-bit COUNT field
appears in the command log. TCOUNT and COUNT are interpreted as
unsigned integers.
The TCOUNT field codes the total number of SysEx commands of the command
type coded by the log that appear in the session history, at the moment
after the (finished or unfinished) command coded by the log enters the
session history.
The COUNT field codes the total number of SysEx commands that appear in
the session history, excluding commands that are excluded from Chapter X
via the ch_never parameter (Appendix C.1), at the moment after the
(finished or unfinished) command coded by the log enters the session
history.
Command counting for TCOUNT and COUNT uses modulo-256 arithmetic. MTC
Full Frame command instances (Appendix B.4) are included in command
counting if the TCOUNT and COUNT definitions warrant their inclusion, as
are cancelled commands (Section 3.2).
Senders use the TCOUNT and COUNT fields to track the identity and (for
TCOUNT) the sequence position of a command instance. Senders MUST use
the TCOUNT or COUNT fields if identity or sequence information is
necessary to protect the command type coded by the log.
If a sender uses the COUNT field in a session, the final command log in
every Chapter X in the stream MUST code the COUNT field. This rule lets
receivers resynchronize the COUNT value after a packet loss.
Lazzaro/Wawrzynek [Page 73]
INTERNET-DRAFT 22 June 2004
C. SDP Session Configuration Tools
In the main text, we show minimal session descriptions for native
(Section 6.1) and mpeg4-generic (Section 6.2) streams. In this
Appendix, we describe how to customize (and perhaps negotiate [13])
stream behavior, through the use of the payload format MIME parameters
and standard SDP attributes.
The Appendix begins with 5 sections, each devoted to parameters that
affect a particular aspect of stream behavior:
o Appendix C.1 describes the journalling system (ch_active,
ch_anchor, ch_default, ch_never, ch_unused, j_sec, j_update).
o Appendix C.2 describes MIDI command timestamp semantics
(linerate, mperiod, octpos, tsmode).
o Appendix C.3 describes media time (guardtime, maxptime, ptime).
o Appendix C.4 describes multi-stream sessions (musicport,
zerosync).
o Appendix C.5 describes MIDI rendering (chanmask, cid,
dirsense, inline, multimode, render, rinit, smf_cid,
smf_info, smf_inline, smf_url, url).
The parameters listed above may optionally appear in RTP MIDI session
descriptions; none are REQUIRED to appear.
Session descriptions use the parameters listed above to add extra
functionality ("features") to minimal RTP MIDI streams. Appendices
C.1-5 define these features. In Appendix C.6, we define the minimum
feature set that RTP MIDI clients SHOULD support. Appendix C.6 also
addresses interoperability issues in Offer/Answer [13] negotiations.
The participants in an RTP MIDI session MUST share a common view of the
session. Participants MUST use the same value for all RTP MIDI MIME
parameters that appear in the session description.
Participants MUST also use the same value for ptime, maxptime, and all
other SDP attribute parameters that act to configure the stream (as
discussed in Section 6.1 of [13]). Parameters that define data flow
direction (sendonly [6], recvonly [6], and dirsense (Appendix C.5.2))
are exceptions to this rule -- for these parameters, each party MUST use
the parameter in a way that denotes the role of the party in the
session.
In some applications, it is too restrictive for all session parties to
Lazzaro/Wawrzynek [Page 74]
INTERNET-DRAFT 22 June 2004
share a common view of the session. For example, a two-party network
musical performance application may wish to customize the renderer on
each host to match the CPU performance of the host [14]. This problem
is easily solved by using two unidirectional RTP sessions in lieu of a
single bidirectional RTP session. The data flows in the two sessions
travel in opposite directions, to control receivers configured to use
different renderers.
Appendix D defines the Augmented Backus-Naur Form (ABNF, [10]) syntax
for the payload MIME parameters. Appendix H provides information to the
Internet Assigned Numbers Authority (IANA) on the MIME types and
parameters defined in this document.
Appendix C.5.6 defines the MIME type "audio/asc", a stored object for
initializing mpeg4-generic renderers. RTP stream semantics are not
defined for "audio/asc". Therefore, "asc" MUST NOT appear on the rtpmap
line of a session description.
C.1 SDP Definitions: The Journalling System
In this Appendix, we define the payload format MIME parameters that
configure stream journalling and the recovery journal system.
The j_sec parameter (Appendix C.1.1) sets the journalling method for the
stream. The j_update parameter (Appendix C.1.2) sets the recovery
journal sending policy for the stream. Appendix C.1.2 also defines the
sending policies of the recovery journal system.
Appendix C.1.3 defines several parameters that modify the recovery
journal semantics. These parameters change the default recovery journal
semantics as defined in Section 5 and Appendices A-B.
C.1.1 The j_sec Parameter
Section 2.2 defines the default journalling method for a stream.
Streams that use unreliable transport (such as UDP) default to using the
recovery journal. Streams that use reliable transport (such as TCP)
default to not using a journal.
The MIME parameter j_sec may be used to override this default. This
memo defines two symbolic values for j_sec: "none", to indicate that all
stream payloads MUST NOT contain a journal section, and "recj", to
indicate that all stream payloads MUST contain a journal section that
uses the recovery journal format.
For example, the j_sec parameter might be set to "none" for a UDP stream
that travels between two hosts on a local network that is known to
Lazzaro/Wawrzynek [Page 75]
INTERNET-DRAFT 22 June 2004
provide reliable datagram delivery.
The session description below configures a UDP stream that does not use
the recovery journal:
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 rtp-midi/44100
a=fmtp:96 j_sec=none
Other IETF standards-track documents may define alternative journal
formats. These documents MUST define new symbolic values for the j_sec
parameter to signal the use of the format.
Parties MUST NOT accept a j_sec value that violates the recovery journal
mandate (see Section 4 for details). If a session description uses a
j_sec value unknown to the recipient, the recipient MUST NOT accept the
description.
Special j_sec issues arise when sessions are managed by the Real Time
Streaming Protocol (RTSP, [17]). In many streaming applications, the
session description in the response to the DESCRIBE method does not code
the transport details (such as UDP or TCP) for the session. Instead,
server and client negotiate transport details using the SETUP method.
In this scenario, the use of the j_sec parameter may be ill-advised, as
the server does not yet know the transport type for the session. In
this case, the session description SHOULD configure the journalling
system using the parameters defined in the remainder of Appendix C.1,
but SHOULD NOT use j_sec to set the journalling status. Recall that if
j_sec does not appear in the session description, the default method for
choosing the journalling method is in effect (no journal for reliable
transport, recovery journal for unreliable transport).
However, in situations where the server knows journalling is always
required or never required, the session description returned by the
DESCRIBE method SHOULD use the j_sec parameter.
C.1.2 The j_update Parameter
In Section 4, we use the term "sending policy" to describe the method a
sender uses to choose the checkpoint packet identity for each recovery
journal in a stream. In the sub-sections that follow, we normatively
define three sending policies: anchor, closed-loop, and open-loop.
Lazzaro/Wawrzynek [Page 76]
INTERNET-DRAFT 22 June 2004
As stated in Section 4, the default sending policy for a stream is the
closed-loop policy. The j_update MIME parameter may be used to override
this default.
We define three symbolic values for j_update: "anchor", to indicate that
the stream uses the anchor sending policy, "open-loop", to indicate that
the stream uses the open-loop sending policy, and "closed-loop", to
indicate that the stream uses the closed-loop sending policy. See
Appendix C.1.3 for examples session descriptions that use the j_update
parameter.
Parties MUST NOT accept a j_update value that violates the recovery
journal mandate (Section 4).
Other IETF standards-track documents may define additional sending
policies for the recovery journal system. These documents MUST define
new symbolic values for the j_update parameter to signal the use of the
new policy. If a session description uses a j_update value unknown to
the recipient, the recipient MUST NOT accept the description.
C.1.2.1 The anchor Sending Policy
In the anchor policy, the sender uses the first packet in the stream as
the checkpoint packet for all packets in the stream. The anchor policy
satisfies the recovery journal mandate (Section 4), as the checkpoint
history always covers the entire stream.
The anchor policy does not require the use of the Real Time Control
Protocol (RTCP, [2]) or other feedback from receiver to sender. Senders
do not need to take special actions to ensure that received streams
start up free of artifacts, as the recovery journal always covers the
entire history of the stream. Receivers are relieved of the
responsibility of tracking the changing identity of the checkpoint
packet, because the checkpoint packet never changes.
The main drawback of the anchor policy is bandwidth efficiency. Because
the checkpoint history covers the entire stream, the size of the
recovery journals produced by this policy usually exceeds the journal
size of alternative policies. For single-channel MIDI data streams, the
bandwidth overhead of the anchor policy is often acceptable (see
Appendix A.4 of [14]). For dense streams, the closed-loop or open-loop
policies may be more appropriate.
C.1.2.2 The closed-loop Sending Policy
The closed-loop policy is the default policy of the recovery journal
system. For each packet in the stream, the policy lets senders choose
the smallest possible checkpoint history that satisfies the recovery
Lazzaro/Wawrzynek [Page 77]
INTERNET-DRAFT 22 June 2004
journal mandate. As smaller checkpoint histories generally yield
smaller recovery journals, the closed-loop policy reduces the bandwidth
of a stream, relative to the anchor policy.
The closed-loop policy relies on feedback from receiver to sender. The
policy assumes that a receiver periodically informs the sender of the
highest sequence number it has seen so far in the stream, coded in the
32-bit extension format defined in [2]. In sessions that use RTCP,
receivers transmit this information in the Extended Highest Sequence
Number Received (EHSNR) field of Receiver Report (RR) packets. However,
applications MAY use any method of feedback to implement the closed-loop
policy.
The sender may safely use receiver sequence number feedback to guide
checkpoint history management, because Section 4 requires receivers to
repair indefinite artifacts whenever a packet loss event occur.
We now normatively define the closed-loop policy. At the moment a
sender prepares an RTP packet for transmission, the sender is aware of R
>= 0 receivers for the stream. Senders may become aware of a receiver
via RTCP traffic from the receiver, via RTP packets from a paired stream
sent by the receiver to the sender, via messages from a session
management tool, or by other means. As receivers join and leave a
session, the value of R changes.
Each known receiver k (1 <= k <= R) is associated with a 32-bit extended
packet sequence number M(k), where the extension reflects the sequence
number rollover count of the sender.
If the sender has received at least one feedback report from receiver k,
M(k) is the most recent report of the highest RTP packet sequence number
seen by the receiver, normalized to reflect the rollover count of the
sender.
If the sender has not received a feedback report from the receiver, M(k)
is the extended sequence number of the last packet the sender
transmitted before it became aware of the receiver. If the sender
became aware of this receiver before it sent the first packet in the
stream, M(k) is the extended sequence number of the first packet in the
stream.
Given this definition of M(), we now state the closed-loop policy. When
preparing a new packet for transmission, a sender MUST choose a
checkpoint packet with extended sequence number N, such that M(k) >= (N
- 1) for all k, 1 <= k <= R, where R >= 1. The policy does not restrict
sender behavior in the R == 0 (no known receivers) case.
Under the closed-loop policy as defined above, a sender may transmit
Lazzaro/Wawrzynek [Page 78]
INTERNET-DRAFT 22 June 2004
packets whose checkpoint history is shorter than the session history (as
defined in Appendix A.1). In this event, a new receiver that joins the
stream may experience indefinite artifacts.
For example, if a Control Change (0xB) command for Channel Volume
(controller number 7) was sent early in a stream, and later a new
receiver joins the session, the closed-loop policy may permit all
packets sent to the new receiver to use a checkpoint history that does
not include the Channel Volume Control Change command. As a result, the
new receiver experiences an indefinite artifact, and play all notes on a
channel too loudly or too softly.
To address this issue, the closed-loop policy states that whenever a
sender becomes aware of a new receiver, the sender MUST determine if the
receiver would be subject to indefinite artifacts under the closed-loop
policy. If so, the sender MUST ensure that the receiver starts the
session free of indefinite artifacts. In satisfying this requirement,
senders MAY infer the initial MIDI state of the receiver from the
session description. For example, the stream example in Section 6.2 has
the initial state defined in [1] for General MIDI.
In some types of sessions, a receiver may have access to stream packets
before the sender is aware of the receiver. In this case, the
restrictions the closed-loop policy places on the sender may not protect
the receiver from indefinite artifacts.
To address this issue, the closed-loop policy states that if a receiver
participates in a session where it may have access to a stream before
the sender is aware of the receiver, the receiver MUST take actions to
ensure that its rendered MIDI performance does not contain indefinite
artifacts. The receiver MUST NOT discontinue these protective actions
until it is certain that the sender is aware of its presence.
The final set of normative closed-loop policy requirements concern how
senders drop receivers from a stream. As defined earlier in this
section, the closed-loop policy states that a sender MUST choose a
checkpoint packet with extended sequence number N, such that M(k) >= (N
- 1) for all k, 1 <= k <= R, where R >= 1. If the sender has received
at least one feedback report from receiver k, M(k) is the most recent
report of the highest RTP packet sequence number seen by the receiver,
normalized to reflect the rollover count of the sender.
If this receiver k stops sending feedback to the sender, the M(k) value
used by the sender reflects the last feedback report from the receiver.
As time progresses without feedback from receiver k, this fixed M(k)
value forces the sender to increase the size of the checkpoint history,
and thus increases the bandwidth of the stream.
Lazzaro/Wawrzynek [Page 79]
INTERNET-DRAFT 22 June 2004
At some point, the sender may need to take action in order to limit the
bandwidth of the stream. The closed-loop policy states that if this
situation occurs, and if the nature of the session permits a sender to
stop transmitting packets to the offending receiver, the sender MUST
stop transmitting packets to this receiver. In other words, it is not
permissible for a sender to no longer use M(k) in computing the
checkpoint packet identity but still send the stream to receiver k, if
it is possible for the sender to actively cut off receiver k from the
stream.
In certain types of sessions, it may not be possible for a sender to
actively stop sending packets to a particular receiver. The closed-loop
policy states that if receivers participate in a session where senders
are unable to stop sending packets to a particular receiver of the
stream, the receiver MUST monitor the RTP stream, and any other sources
of information, to determine if the sender is no longer using the M(k)
feedback from the receiver to choose each checkpoint packet. If the
receiver detects this condition, it MUST leave the session, and close
down the rendered MIDI performance in a manner that is free of
indefinite artifacts.
Finally, we note that the closed-loop policy is suitable for use in
RTP/RTCP sessions that use multicast transport. However, aspects of the
closed-loop policy do not scale well to sessions with large numbers of
participants. The sender state scales linearly with the number of
receivers, as the sender needs to track the identity and M(k) value for
each receiver k. The average recovery journal size is not independent
of the number of receivers, as the RTCP reporting interval backoff slows
down the rate of a full update of M(k) values. The backoff algorithm
may also increase the amount of ancillary state used by implementations
of the normative sender and receiver behaviors defined in Section 4.
C.1.2.3 The open-loop Sending Policy
The open-loop policy is suitable for sessions that are not able to
implement the receiver-to-sender feedback required by the closed-loop
policy, and are also not able to use the anchor policy because of
bandwidth constraints.
The open-loop policy does not place constraints on how a sender chooses
the checkpoint packet for each packet in the stream. In the absence of
such constraints, a receiver may find that the recovery journal in the
packet that ends a loss event has a checkpoint history that does not
cover the entire loss event. We refer to loss events of this type as
uncovered loss events.
To ensure that uncovered loss events do not compromise the recovery
journal mandate, the open-loop policy assigns specific recovery tasks to
Lazzaro/Wawrzynek [Page 80]
INTERNET-DRAFT 22 June 2004
senders, receivers, and the creators of session descriptions. The
underlying premise of the open-loop policy is that the indefinite
artifacts produces during uncovered loss events fall into two classes.
One class of artifacts are recoverable indefinite artifacts. Receivers
are able to repair recoverable artifacts that occur during an uncovered
loss event without intervention from the sender, at the potential cost
of unpleasant transient artifacts.
For example, after an uncovered loss event, receivers are able to repair
indefinite artifacts due to NoteOff (0x8) commands that may have
occurred during the loss event, by executing NoteOff commands for all
active NoteOns commands. This action causes a transient artifacts (a
sudden silent period in the performance), but ensures that no stuck
notes sound indefinitely. We refer to MIDI commands that are amenable
to repair in this fashion as recoverable MIDI commands.
A second class of artifacts are unrecoverable indefinite artifacts. If
this class of artifact occurs during an uncovered loss event, the
receiver is not able to repair the stream.
For example, after an uncovered loss event, receivers are not able to
repair indefinite artifacts due to Control Change (0xB) Channel Volume
(controller number 7) commands that have occurred during the loss event.
A repair is impossible because the receiver has no way of determining
the data value of a lost Channel Volume command. We refer to MIDI
commands that are fragile in this way as unrecoverable MIDI commands.
The open-loop policy does not specify how to partition the MIDI command
set into recoverable and unrecoverable commands. Instead, it assumes
that the creators of the session descriptions are able to come to
agreement on a suitable recoverable/unrecoverable MIDI command partition
for an application.
Given these definitions, we now state the normative requirements for the
open-loop policy.
In the open-loop policy, the creators of the session description MUST
use the ch_unused or ch_anchor MIME parameters (defined in Appendix
C.1.3) to protect all unrecoverable MIDI command types from indefinite
artifacts.
In a general sense, the ch_anchor parameter changes the recovery journal
semantics to use the anchor checkpoint policy (Appendix C.1.2.1) for a
command, and the ch_unused parameter acts to exclude a command type from
the stream. These options act to shield command types from artifacts
during an uncovered loss event.
Lazzaro/Wawrzynek [Page 81]
INTERNET-DRAFT 22 June 2004
In the open-loop policy, receivers MUST examine the Checkpoint Packet
Seqnum field of the recovery journal header after every loss event, to
check if the loss event is an uncovered loss event. Section 5 shows how
to perform this check. If an uncovered loss event has occurred, a
receiver MUST perform indefinite artifact recovery for all MIDI command
types that are not shielded by ch_anchor and ch_unused parameter
assignments in the session description.
The open-loop policy does not place specific constraints on the sender.
However, the open-loop policy works best if the sender manages the size
of the checkpoint history to ensure that uncovered losses occur
infrequently, by taking into account the delay and loss characteristics
of the network. Also, as each checkpoint packet change incurs the risk
of an uncovered loss, senders should only move the checkpoint if it
reduces the size of the journal.
C.1.3 Recovery Journal Chapter Inclusion Parameters
The recovery journal chapter definitions (Appendices A-B) specify under
what conditions a chapter MUST appear in the recovery journal. In most
cases, the definition states that if a certain command appears in the
checkpoint history, a certain chapter type MUST appear in the recovery
journal to protect the command.
In this section, we describe the chapter inclusion MIME parameters.
These parameters modify the conditions under which a chapter appears the
journal. These parameters are essential to the use of the open-loop
policy (Appendix C.1.2.3), and may also be used to simplify
implementations of the closed-loop (Appendix C.1.2.2) and anchor
(Appendix C.1.2.1) policies.
The parameters may also be used to indicate the MIDI command types that
do not appear in a session. In this role, the parameters may be used
with streams that do not use journalling. For example, if a session
partitions a MIDI source into two RTP streams (Appendix C.4), the
parameters may be used to define which commands appear in each stream.
A party MUST NOT accept chapter inclusion parameter values that violate
the recovery journal mandate (Section 4). In addition, a party MUST NOT
accept chapter inclusion parameter values that yield a MIDI command
stream incompatible with its renderer (Appendix C.5).
Each parameter represents a type of chapter inclusion semantics. An
assignment to a parameter declares which chapters (or chapter subsets)
obey the inclusion semantics. We describe the assignment syntax for
these parameters later in this section.
Lazzaro/Wawrzynek [Page 82]
INTERNET-DRAFT 22 June 2004
Below, we normatively define the semantics of the chapter inclusion
parameters. For clarity, we define the action of parameters on complete
chapters. If a parameter is assigned a subset of a chapter, the
definition applies only to the chapter subset.
o ch_unused. If a chapter is assigned to the ch_unused parameter,
the command types encoded by the chapter MUST NOT appear in the
MIDI command sections of stream packets. As a consequence,
the chapter MUST NOT appear in the recovery journal.
In contrast with ch_unused, if a chapter is assigned to the parameters
we define below, the command types encoded by the chapter MAY appear in
the MIDI command section of stream packets.
o ch_never. A chapter assigned to the ch_never parameter MUST
NOT appear in the recovery journal (Appendix A.4.1-2 defines
exceptions to this rule for Chapter M).
o ch_default. A chapter assigned to the ch_default parameter
MUST follow the default semantics for the chapter, as defined
in Appendices A-B.
o ch_anchor. A chapter assigned to the ch_anchor MUST obey a
modified version of the default chapter semantics. In the
modified semantics, all references to the checkpoint history
are replaced with references to the session history, and all
references to the checkpoint packet are replaced with
references to the first packet sent in the stream.
o ch_active. A chapter assigned to the ch_anchor MUST obey a
modified version of the default chapter semantics. In the
modified semantics, certain uses of the terms "C-active"
and "N-active" in the chapter definitions are replaced by
the term "active" (quoted terms are defined in Appendix A.1).
The exact semantics for ch_active appear later in this section.
Parameter assignments obey the following syntax (see Appendix D for
ABNF):
<parameter> = [channel list]<chapter list>[field list]
The chapter list is mandatory; the channel and field lists are optional.
Multiple assignments to parameters have a cumulative effect, and are
applied in the order of parameter appearance in a media description.
A later assignment of a chapter to the same parameter expands the scope
of the earlier assignment. In most cases, a later assignment of a
chapter to a different parameter cancels (partially or completely) the
Lazzaro/Wawrzynek [Page 83]
INTERNET-DRAFT 22 June 2004
effect of an earlier assignment. An exception is the parameter pair
ch_active and ch_anchor: if a chapter is applied to both parameters, the
semantic modifications of both parameters apply to the chapter.
The chapter list specifies the channel or system chapters for which the
parameter applies. The chapter list is a concatenated sequence of one
or more of the letters corresponding to the chapter types
(ACDEFMNPQTVWX). In addition, the list may contain one or more of the
letters for the sub-chapter types (BGHJKYZ) of System Chapter D.
The letters in a chapter list MUST be upper case, and MUST appear in
alphabetical order. Letters other than (ABCDEFGHJKMNPQTVWXYZ) that
appear in the chapter list MUST be ignored.
The channel list specifies the channel journals for which this parameter
applies; if no channel list is provided, the parameter applies to all
channel journals. The channel list takes the form of a list of channel
numbers (0 through 15) and dash-separated channel number ranges (i.e.
0-5, 8-12, etc). Dots (i.e. "." characters) separate elements in the
channel list.
A few system channels use special semantics for the channel list, which
we now define.
For the J and K Chapter D sub-chapters (undefined System Common), the
digit 0 codes that the parameter applies to the LEGAL field of the
associated command log (Figure B.1.4 of Appendix B.1), the digit 1 codes
that the parameter applies to the VALUE field of the command log, and
the digit 2 codes that the parameter applies to the COUNT field of the
command log.
For the Y and Z Chapter D sub-chapters (undefined System Real-time), the
digit 0 codes that the parameter applies to the LEGAL field of the
associated command log (Figure B.1.5 of Appendix B.1) and the digit 1
codes that the parameter applies to the COUNT field of the command log.
For Chapter Q (Sequencer State Commands), the digit 0 codes that the
parameter applies to the default Chapter Q definition, which forbids the
TIME field. The digit 1 codes that the parameter applies to the
optional Chapter Q definition, which supports the TIME field.
For Chapter X, the digit 0 codes that SysEx "cancel" sublists (Section
3.2 in the main text) MUST NOT appear in the MIDI list. The digit 1
codes that cancel sublists MAY appear in the MIDI list (the default
condition). The channel list MUST NOT contain both digits 0 and 1.
Also for Chapter X, the digit 2 codes that commands other than System
Real-time MIDI commands MUST NOT appear between SysEx command segments
Lazzaro/Wawrzynek [Page 84]
INTERNET-DRAFT 22 June 2004
in the MIDI list (the default condition, as defined in Section 3.2 in
the main text). The digit 3 codes that any MIDI command type may appear
between SysEx command segments in the MIDI list, with the exception of
the segmented encoding of a second SysEx command (verbatim SysEx
commands are OK). The channel list MUST NOT contain both digits 2 and
3.
The syntax for field lists follows the syntax for channel lists. If no
field list is provided, the parameter applies to all controller or note
numbers.
For Chapter C, the field list codes the controller numbers for which the
parameter applies.
For Chapter M, the field list codes the Registered Parameter Numbers
(RPNs) and Non-Registered Parameter Numbers (NRPNs) for which the
parameter applies. The number range 0-16383 specifies RPNs, the number
range 16384-32767 specifies NRPNs (16384 corresponds to NRPN 0, 32767
corresponds to NRPN 16383).
For Chapters N and A, the field list codes the note numbers for which
the parameter applies. The note number range specified for Chapter N
also applies to Chapter E.
For Chapter E, the digit 0 codes that the parameter applies to Chapter E
note logs whose V bit is set to 0, the digit 1 codes that the parameter
applies to note logs whose V bit is set to 1.
For sub-chapters J and K of Chapter D, the field list consists of a
single digit, which specifies the number of data octets that follow the
command octet.
For Chapter X, the field list codes the number of data octets that may
appear in a SysEx command. Thus, the field list 0-255 specifies SysEx
commands with 255 or fewer data octets, the field list 256-429496729
specifies SysEx commands with more than 255 data octets but excludes
commands with 255 or fewer data octets, and the field list 0 excludes
all commands.
The chapter list assigned to the ch_active parameter MUST NOT code
chapters other than Chapters C, E, M, N, T, and A. The semantics of
ch_active assignment for these chapters are defined below:
o Chapter C. Logs for controllers assigned to ch_active obey
the semantics defined in Appendices A.3.1-4, except that all
uses of the term "C-active" are replaced with the term "active".
o Chapter E. If any note number is assigned to ch_active,
Lazzaro/Wawrzynek [Page 85]
INTERNET-DRAFT 22 June 2004
all uses of the term N-active in Appendix A.7 are replaced
with the term "active. In addition, MIDI Control Change
commands no longer act to reset note reference counts.
o Chapter M. Logs for RPN and NRPN parameters assigned to
ch_active obey the semantics defined in Appendices A.4.1-2,
except that all uses of the term "C-active" in these
Appendices are replaced with the term "active". The ch_active
parameter does not affect uses of the term "C-active" in the
Appendix A.4 text that precedes Appendix A.4.1.
o Chapter N. If any note number is assigned to ch_active,
all uses of the term N-active in Appendix A.6 are replaced
with the term "active.
o Chapter T. If Chapter T is assigned to ch_active, all uses
of the term N-active in Appendix A.8 are replaced with the
term "active.
o Chapter A. If any note number is assigned to ch_active,
all uses of the term "N-active" in Appendix A.9 are replaced
with the term "active".
Chapter E parameter assignments to ch_unused indicate that NoteOff and
NoteOn commands do not use the semantics protected by Chapter E. In
contrast, Chapter N ch_unused assignments refer to all NoteOff and
NoteOn commands.
A secondary parameter assignment syntax customizes Chapter X (see
Appendix D for complete ABNF):
<parameter> = "__" <h-list> ["_" <h-list>] "__"
The assignment defines a class of SysEx commands whose Chapter X coding
obeys the semantics of the assigned parameter. The command class is
specified by listing the permitted values of the first N data octets
that follow the SysEx 0xF0 command octet. Any SysEx command whose first
N data octets match the list is a member of the class.
Each <h-list> defines a data octet of the command, as a dot-separated
(".") list of one or more hexadecimal constants (such as "7F") or dash-
separated hexadecimal ranges (such as "01-1F"). Underscores ("_")
separate each <h-list>. Double-underscores ("__") delineate the data
octet list.
Using this syntax, each assignment specifies a single SysEx command
class. Session descriptions may use several assignments to the same (or
different) parameters to specify complex Chapter X behaviors. The
Lazzaro/Wawrzynek [Page 86]
INTERNET-DRAFT 22 June 2004
ordering behavior of multiple assignments follows the guidelines for
chapter parameter assignments described earlier in this section.
The example session description below illustrates the use of the chapter
inclusion parameters:
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP6 FF1E:03AD::7F2E:172A:1E24
a=rtpmap:96 rtp-midi/44100
a=fmtp:96 j_update=open-loop; ch_unused=ABDEFGHJKMQTVWXYZ; ch_anchor=P;
ch_anchor=C7.64; ch_never=4.11-13N; ch_anchor=__7E_00-7F_09_01.02.03__;
ch_anchor=__7F_00-7F_04_01.02__
(The a=fmtp line has been wrapped to fit the page to accommodate
memo formatting restrictions; it comprises a single line in SDP)
The j_update parameter codes that the stream uses the open-loop policy.
Most chapters are assigned to ch_unused, a typical MIDI usage pattern of
a low-bandwidth stream.
To guard against indefinite artifacts, the MIDI Program Change command
and several MIDI Control Change controller numbers are assigned to
ch_anchor. Note that the ordering of the ch_anchor chapter C assignment
after the ch_unused command acts to override the ch_unused assignment
for the listed controller numbers (7 and 64).
Chapter N for several MIDI channels is assigned to ch_never; in
practice, this assignment pattern would reflect knowledge about a
resilient rendering method in use for certain channels. In this
example, Chapter N for MIDI channels other than 4, 11, 12, and 13 may
appear in the recovery journal, per the default behavior.
The assignment of Chapter X to ch_unused excludes most System Exclusive
commands from the stream. Exceptions are made for General MIDI System
On/Off commands and for the Master Volume and Balance commands, via the
use of the secondary assignment syntax.
Lazzaro/Wawrzynek [Page 87]
INTERNET-DRAFT 22 June 2004
C.2 SDP Definitions: Command Execution Semantics
The MIDI command section of the payload format consists of a list of
commands, each with an associated timestamp. Section 3.1 defines the
default semantics for command timestamps. These semantics work well for
transcoding Standard MIDI Files (SMFs), but are problematic for
transcoding MIDI sources (such as MIDI 1.0 DIN cables [1]) that use
implicit "time-of-arrival" coding.
In this Appendix, we define session configuration tools for customizing
the timestamp semantics of the MIDI command section.
The MIME parameter "tsmode" specifies the timestamp semantics for a
stream. The parameter takes on one of three token values: "comex",
"async", or "buffer".
The "comex" value specifies the default timestamp semantics. The
"async" value specifies an asynchronous timestamp sampling algorithm for
time-of-arrival sources (Appendix C.2.1). The "buffer" value specifies
a synchronous timestamp sampling algorithm (Appendix C.2.2).
Ancillary MIME parameters may follow tsmode in a media description. One
such parameter is "linerate". This parameter codes the timespan of one
MIDI octet on the transmission medium of the MIDI source to be sampled
(such as a MIDI 1.0 DIN cable). The parameter has units of nanoseconds,
and takes on integral values. For MIDI 1.0 DIN cables, the correct
linerate value is 320000 (this value is also the default value for the
parameter). Other ancillary MIME parameters are defined in Appendices
C.2.1-2 below.
C.2.1 The async Algorithm
The "async" tsmode value specifies the asynchronous sampling of a MIDI
time-of-arrival source. In asynchronous sampling, the moment an octet
is received from a source it is labelled with a wall-clock time value.
The time value has RTP timestamp units.
The "octpos" ancillary MIME parameter defines how RTP command timestamps
are derived from octet time values. If octpos has the token value
"first", a timestamp codes the time value of the first octet of the
command. If octpos has the token value "last", a timestamp codes the
time value of the last octet of the command. If the octpos parameter
does not appear in the media description, a timestamp MAY reflect the
time value of any octet of the command.
The octpos semantics refer to the first or last octet of a command as it
appears on a time-of-arrival MIDI source, not as it appears in an RTP
MIDI packet. This distinction is significant because the RTP coding may
Lazzaro/Wawrzynek [Page 88]
INTERNET-DRAFT 22 June 2004
contain octets that are not present in the source. For example, the
status octet of the first MIDI command in a packet may have been added
to the MIDI stream during transcoding, to comply with the RTP MIDI
running status requirements (Section 3.2).
We now show a session description example for the async algorithm.
Consider a sender that is transcoding a MIDI 1.0 DIN cable source into
RTP. The sender runs on a computing platform that assigns time values
to every incoming octet of the source, and the sender uses the time
values to label the first octet of each command in the RTP packet. This
session description describes the transcoding:
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 rtp-midi/44100
a=fmtp:96 tsmode=async; linerate=320000; octpos=first
a=sendonly
C.2.2 The buffer Algorithm
The "buffer" tsmode value specifies the synchronous sampling of a MIDI
time-of-arrival source.
In synchronous sampling, octets received from a source are placed in a
holding buffer upon arrival. At periodic intervals, the RTP sender
examines the buffer. The sender removes complete commands from the
buffer, and codes those commands in an RTP packet. The command
timestamp codes the moment of buffer examination, expressed in RTP
timestamp units. Note that several commands may have the same timestamp
value.
The "mperiod" ancillary MIME parameter defines the nominal periodic
sampling interval. The parameter takes on positive integral values, and
has RTP timestamp units.
The "octpos" ancillary MIME parameter, defined in Appendix C.2.1 for
asynchronous sampling, plays a different role in synchronous sampling.
In synchronous sampling, the parameter specifies the timestamp semantics
of a command whose octets span several sampling periods.
If octpos has the token value "first", the timestamp reflects the
arrival period of the first octet of the command. If octpos has the
token value "last", the timestamp reflects the arrival period of the
last octet of the command. If the octpos parameter does not appear in
Lazzaro/Wawrzynek [Page 89]
INTERNET-DRAFT 22 June 2004
the media description, the timestamp MAY reflect the arrival period of
any octet of the command. The octpos semantics refer to the first or
last octet of the command as it appears on a time-of-arrival source, not
as it appears in the RTP packet.
We now show a session description example for the buffer algorithm.
Consider a sender that is transcoding a MIDI 1.0 DIN cable source into
RTP. The sender runs on a computing platform that places source data
into a buffer upon receipt. The sender polls the buffer 1000 times a
second, extracts all complete commands from the buffer, and places the
commands in an RTP packet. This session description describes the
transcoding:
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP6 FF1E:03AD::7F2E:172A:1E24
a=rtpmap:96 rtp-midi/44100
a=fmtp:96 tsmode=buffer; linerate=320000; octpos=last; mperiod=44
a=sendonly
The mperiod value of 44 is derived by dividing the clock rate specified
by the rtpmap attribute (44100 Hz) by the 1000 Hz buffer sampling rate,
and rounding to the nearest integer. Command timestamps might not
increment by exact multiples of 44, as the actual sampling period might
not precisely match the nominal mperiod value.
Lazzaro/Wawrzynek [Page 90]
INTERNET-DRAFT 22 June 2004
C.3 SDP Definitions: Packet Timing Tools
In this Appendix, we describe session configuration tools for
customizing the temporal behavior of MIDI stream packets.
C.3.1 ptime and maxptime
Senders code the temporal nature of a stream by choosing the amount of
media time encoded in each packet. Short media times (20 ms or less)
often imply an interactive session. Longer media times (100 ms or more)
usually indicate a content streaming session. The AVP profile permits
audio packet media times to range from 0 to 200 ms.
An RTP receiver dynamically senses the media time of packets in a
stream, and chooses the length of its playout buffer to match the
stream. A receiver typically sizes its playout buffer to fit several
audio packets, and adjusts the buffer length to reflect the network
jitter and the sender timing fidelity.
Alternatively, the packet media time may be statically set during
session configuration. The standard "ptime" attribute sets the typical
packet media time for a session. The standard "maxptime" attribute sets
the maximum packet media time for a session [6].
0 ms is a reasonable media time value for MIDI packets. In a packet
with a 0 ms media time, all commands execute at the instant coded by the
packet timestamp. Prohibitions in [13] against 0 ms ptime values are
not relevant for MIDI streams, and may be ignored.
As stated in the Appendix C preamble, the senders and receivers of a
stream MUST agree on common values for ptime and maxptime if the
parameters appear in the media description for the stream. This
requirement is atypical for RTP payload formats, and is a consequence of
the critical role of latency in interactive MIDI applications. The
session description example below defines a stream suitable for use in
low-latency interactive applications:
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 rtp-midi/44100
a=ptime:0
a=maxptime:0
Lazzaro/Wawrzynek [Page 91]
INTERNET-DRAFT 22 June 2004
C.3.2 The guardtime Parameter
RTP/AVP permits a sender to stop sending audio packets for an arbitrary
period of time during a session. When sending resumes, the RTP sequence
number series continues unbroken, and the RTP timestamp value reflects
the media time silence gap.
This RTP/AVP feature has its roots in telephony, but is also well
matched to interactive MIDI sessions, as players may fall silent for
several seconds during (or between) songs.
Certain MIDI applications benefit from a slight enhancement to this
RTP/AVP feature. In interactive applications, receivers may use on-line
network models to guide heuristics for handling lost and late RTP
packets. These models may work poorly if a sender ceases packet
transmission for long periods of time.
Session descriptions may use the MIME parameter "guardtime" to set a
minimum sending rate for a media session. The value assigned to
guardtime codes the maximum separation time between two sequential
packets, as expressed in RTP timestamp units. Typical guardtime values
are 500-2000 ms.
Below, we show a session description that uses the guardtime parameter.
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP6 FF1E:03AD::7F2E:172A:1E24
a=rtpmap:96 rtp-midi/44100
a=fmtp:96 guardtime=44100
a=ptime:0
a=maxptime:0
Lazzaro/Wawrzynek [Page 92]
INTERNET-DRAFT 22 June 2004
C.3.3 MIDI Time Code Issues
RTP defines tools to synchronize the playout of multiple RTP media
streams. Appendix C.4 shows how to use these tools in MIDI streams.
In content-creation applications, it may be necessary to synchronize
stream playout with media that are not sent over RTP. For example,
analog video may be marked with SMPTE 12M timecode, and an application
may need to synchronize MIDI playout the video using timecode.
The MIDI standard includes the MIDI Time Code (MTC) commands for SMPTE
12M timecode [1]. An application MAY use MTC to send timecode data
(including offsets and user data) in the MIDI command stream for
heterogeneous synchronization purposes.
Lazzaro/Wawrzynek [Page 93]
INTERNET-DRAFT 22 June 2004
C.4 SDP Definitions: Multiple Streams
Several MIDI streams may appear in a session description. By default,
the MIDI name space (16 voice channels + systems) for each stream is
unique, and the rendering for each stream proceeds independently. The
audio outputs of the streams are presented in a synchronized fashion.
In this Appendix, we define two MIME parameters for use in sessions with
several streams. These parameters ("musicport" and "zerosync") add
these features to RTP MIDI:
1. A sendonly RTP session and a recvonly RTP session [6] may
be associated to form a "virtual" bidirectional RTP session
(identical to the "built-in" bidirectionality of sendrecv
RTP sessions).
2. Several streams may target the same MIDI name space.
3. Several streams may be bundled to form a larger MIDI
name space, that a single rendering system may treat as
an ordered entity.
4. Streams may specify relative timebase offsets, to support
synchronization with zero sync-lock delay.
In Appendices C.4.1-2, we define the musicport and zerosync parameters.
In Appendix C.4.3, we show session description examples.
Parties MUST NOT accept multi-stream session descriptions that violate
the recovery journal mandate (defined in Section 4 of the main text).
Other payload formats MAY define musicport and zerosync MIME parameters.
Formats would define these parameters so that their streams could be
bundled into RTP MIDI name spaces. The parameter definitions MUST be
compatible with the musicport and zerosync semantics defined in this
Appendix.
C.4.1 The musicport Parameter
The musicport parameter codes an arbitrary identification number for the
MIDI name space (16 voice channels + systems) of an RTP stream. The
musicport parameter may take on integer values between 0 and 429496729.
If several MIDI streams in a session have contiguous musicport values
(i.e. i, i+1, ... i+k), the MIDI name spaces of the streams form an
ordered entity. In this case, the streams in the entity are said to
share an ordered relationship.
Lazzaro/Wawrzynek [Page 94]
INTERNET-DRAFT 22 June 2004
If several MIDI streams in a session share the same musicport value, the
streams form an identity relationship.
In an identity relationship, streams that travel in same direction
target the same MIDI name space. Streams that do not travel in the same
direction inherit an association that is identical to the association
shared by the bidirectional streams in a single sendrecv RTP session [6]
[13]. We refer to such sessions as "virtual" sendrecv sessions.
Note that a stream may participate in both an identity and an ordered
relationship. For example, a stream in an identity relationship may
have a musicport value that forms part of an ordered relationship. If
the musicport values of two streams are not part of an ordered or
identity relationship, the two streams are independent, and have
independent MIDI name spaces.
RTP MIDI streams in an ordered or identity relationship MUST be all
native streams or all mpeg4-generic streams. Thus, we refer to
relationships as being native relationships or mpeg4-generic
relationships.
An exception to this "all relationship streams share the same type" rule
applies to relationships that exclusively contain sendonly and recvonly
streams. In this case, the sendonly streams MUST be all native streams
or all mpeg4-generic streams, and the recvonly streams MUST be all
native streams or all mpeg4-generic streams, but the sendonly and
recvonly streams may be of differing types.
For native relationships, at most one stream may specify MIDI renderers
(using the tools described in Appendix C.5). Each MIDI rendering type
may define its own semantics with regard to identity and ordered
relationships.
For mpeg4-generic relationships, at most one stream in an identity or
ordered relationship may have a config parameter value other than the
empty string. In this case, the config value configures the stream.
Alternatively, all config parameters may be set to the empty string. In
this case, exactly one stream in the relationship MUST define the
configuration using the tools described in Appendix C.5.
For both native and mpeg4-generic relationships, an exception to the
"one stream defines the rendering" rule applies to relationships that
exclusively contain sendonly and recvonly streams. In this case, a
stream in each direction may define a renderer.
In an identity relationship, a sender MAY partition the MIDI name space
(16 voice channels + systems) into several RTP streams. Senders MAY use
the chapter inclusion parameters (Appendix C.1.3) to predefine a
Lazzaro/Wawrzynek [Page 95]
INTERNET-DRAFT 22 June 2004
partition, or MAY use a dynamic partitioning strategy. We now specify
receiver and sender responsibilities to ensure the robust transmission
of identity relationships.
Receivers that merge identity relationship streams into a single MIDI
command stream MUST maintain the structural integrity of the MIDI
commands coded in each stream during the merging process, in the same
way that software that merges traditional MIDI 1.0 DIN cable flows is
responsible for creating a merged command flow compatible with [1].
Senders MUST partition the name space so that the rendered MIDI
performance does not contain indefinite artifacts (as defined in Section
4). This responsibility holds even if all streams are sent over
reliable transport, as imperfect synchronization of reliable streams may
yield indefinite artifacts. For example, stuck notes may occur in a
performance split over two TCP streams, if NoteOn commands are sent on
one stream and NoteOff commands are sent on the other.
Senders MUST NOT split a Registered Parameter Name (RPN) or Non-
Registered Parameter Name (NRPN) transaction appearing on a MIDI channel
across multiple identity relationship streams. Receivers MUST assume
that the RPN/NRPN transactions that appear on different identity
relationship streams are independent, and MUST preserve transactional
integrity during the MIDI merge.
A simple way to safely partition voice channel commands is to place all
MIDI commands for a particular voice channel into the same stream. Safe
partitioning of MIDI Systems commands may be more complicated for
streams that extensively use System Exclusive commands.
C.4.2 The zerosync Parameter
The RTP timestamp of the first packet in a stream is not set to zero.
Instead, [2] mandates that the RTP timestamp is initialized to a
randomly chosen value, to guard against plaintext attacks on encrypted
streams. As a consequence, a receiver cannot directly use RTP
timestamps to play back two RTP streams in sync.
The Real Time Control Protocol (RTCP), a low-bandwidth feedback channel
that is paired with each RTP stream, provides synchronization services.
Certain types of RTCP packets code the current time in two forms: the
format of the RTP timestamp, and the 64-bit Network Time Protocol (NTP)
format. A receiver may examine the NTP timestamps of several RTCP
streams, and use this information to deduce the temporal relationship
between the RTP streams associated with the RTCP streams. This method
assumes that the NTP timestamps coded by all streams derive from a
common clock source.
Lazzaro/Wawrzynek [Page 96]
INTERNET-DRAFT 22 June 2004
For many applications, this RTCP-based method is a good way to
synchronize streams. In some applications, however, this method is not
optimal, because of the synchronization time delay at the start of the
session.
The zerosync parameter provides an alternative mechanism for stream
synchronization. The zerosync parameter codes the RTP timestamp offsets
for each stream, so that streams generated in a synchronized fashion may
be played back in sync without using RTCP feedback.
The use of the zerosync parameter weakens the security of RTP, as
discussed in Appendix G of this memo.
The zerosync parameter supports two synchronization mechanisms. One
mechanism potentially synchronizes all streams within a given
relationship in each direction. Media descriptions code this mechanism
with a zerosync parameter whose value is in the range 1-429496729. We
refer to this mechanism as the non-zero behavior.
A second mechanism potentially synchronizes all RTP MIDI streams in a
session flowing in one direction. Media descriptions code this
mechanism with a zerosync parameter whose value is set to 0. We refer
to this mechanism as the zero behavior.
A media description may contain, at most, one zerosync parameter
assignment. Thus, a stream may participate in a non-zero behavior or a
zero behavior, but not both. In both zero and non-zero behaviors, all
media descriptions synchronized by the behavior flowing in one direction
MUST have identical clock rates (as specified by the rtpmap attribute).
In a non-zero behavior, all streams within a relationship flowing in one
direction share an underlying timebase, but the randomly chosen initial
timestamp value for each stream obscures this commonality. To unmask
the similarity, each media description in the relationship MAY include a
zerosync parameter whose non-zero value codes its initial timestamp
value. In this scheme, the underlying timestamp for a packet is
computed by subtracting (modulo 2^32) the zerosync value from the packet
timestamp.
In a zero behavior, all affected streams flowing in one direction share
an underlying timebase AND the same initial timestamp value (in direct
violation of [2]). Thus, the packet timestamps code the "true"
timestamp directly.
Lazzaro/Wawrzynek [Page 97]
INTERNET-DRAFT 22 June 2004
C.4.3 Multi-stream examples using musicport and zerosync.
This section shows several session description examples that use the
musicport and zerosync parameters.
Our first session description example shows two mpeg4-generic streams
that drive the same General MIDI decoder.
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; profile-level-id=12;
config=7A124D546864000000060000000100604D54726B0000000400FF2F000;
musicport=12; zerosync=1726
m=audio 5006 RTP/AVP 97
c=IN IP4 192.0.2.94
a=rtpmap:97 mpeg4-generic/44100
a=fmtp:97 streamtype=5; mode=rtp-midi; config=""; profile-level-id=12;
musicport=12; zerosync=726
(The a=fmtp lines have been wrapped to fit the page to accommodate
memo formatting restrictions; they comprise single lines in SDP)
The musicport values indicate the streams share an identity
relationship, and the zerosync values code the non-zero behavior.
A variant on this example, whose session description is not shown, would
use two streams in an identity relationship driving the same MIDI
renderer, each with a different transport type. One stream would use
UDP, and would be dedicated to real-time messages. A second stream
would use TCP [22] and would be used for SysEx bulk data messages.
Lazzaro/Wawrzynek [Page 98]
INTERNET-DRAFT 22 June 2004
In the next example, two mpeg4-generic streams form an ordered
relationship to drive a Structured Audio decoder with 32 MIDI voice
channels.
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP6 FF1E:03AD::7F2E:172A:1E24
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; config=""; profile-level-id=13;
musicport=5; zerosync=0
m=audio 5006 RTP/AVP 97
c=IN IP6 FF1E:03AD::7F2E:172A:1E24
a=rtpmap:97 mpeg4-generic/44100
a=fmtp:97 streamtype=5; mode=rtp-midi; config=""; profile-level-id=13;
profile-level-id=13; musicport=6; zerosync=0; render=synthetic;
rinit="audio/asc"; url="http://example.com/cardinal.asc";
cid="azsldkaslkdjqpwojdkmsldkfpe"
(The a=fmtp lines have been wrapped to fit the page to accommodate
memo formatting restrictions; they comprise single lines in SDP)
The sequential musicport values for the two streams establishes the
ordered relationship. The musicport=5 stream maps to Structured Audio
extended channels range 0-15, the musicport=6 stream maps to Structured
Audio extended channels range 16-31. The zerosync values code the zero
behavior.
Both config strings are empty. The configuration data is specified by
MIME parameters that appear in the fmtp line of the second media
description. We define this configuration method in Appendix C.5.
Lazzaro/Wawrzynek [Page 99]
INTERNET-DRAFT 22 June 2004
C.5 SDP Definitions: MIDI Rendering
This Appendix defines the session configuration tools for rendering.
The "render" MIME parameter specifies a rendering method for a stream.
The parameter is assigned a token value that signals the top-level
rendering class. This memo defines four token values for render:
"unknown", "synthetic", "api", and "null":
o An "unknown" renderer is a renderer whose nature is unspecified.
It is the default renderer for native RTP MIDI streams.
o A "synthetic" renderer transforms the MIDI stream into audio
output (or sometimes, into stage lighting changes or other
actions). It is the default renderer for mpeg4-generic
RTP MIDI streams.
o An "api" renderer presents the command stream to applications
via an Application Programmer Interface (API).
o The "null" renderer discards the MIDI stream.
The "null" and "unknown" render values play special roles during
Offer/Answer negotiations (Appendix C.6.5). A party uses the "null"
value in an answer to reject an offered renderer. We describe the
Offer/Answer roles of "unknown" and "null" in detail in Appendix C.6.5.
Other IETF standards-track documents MAY define additional token values
for the render parameter. If a party is offered a session description
that uses a token value that is not known to the party, and if the party
needs to know about the renderer for correct operation, the party MUST
NOT accept the renderer. Options include rejecting the renderer (using
the "null" value) or rejecting the entire session description.
Other MIME parameters MAY follow a render parameter in the media
description, and define the exact nature of a renderer. The "rinit"
MIME parameter (defined in Appendix C.5.3) specifies the MIME type and
subtype for the renderer, and the "inline", "url", and "cid" MIME
parameters (defined in Appendix C.5.4) specify renderer initialization
data. We describe these parameters (and related renderer MIME
parameters) in the sections below.
Special rules apply to using the render parameter in an mpeg4-generic
stream. We define these rules in Appendix C.5.6.
Lazzaro/Wawrzynek [Page 100]
INTERNET-DRAFT 22 June 2004
C.5.1 The multimode Parameter
A media description MAY contain several render parameters. By default,
if a media description lists several render parameters, a receiver MUST
choose exactly one renderer from the list to render the stream. The
MIME parameter "multimode" may be used to override this default. We
define two token values for multimode: "one" and "all":
o The default "one" value requests rendering by exactly one of
the listed renderers.
o The "all" value requests the synchronized rendering of the RTP
MIDI stream by all listed renderers, if possible.
If the multimode parameter appears in a media description, it MUST
appear before the first render parameter assignment.
If the multimode parameter is assigned the "all" value, receivers MUST
be aware of the Reset State commands (Appendix C.1) for all stream
renderers, and MUST insure that no renderer experiences indefinite
artifacts due to the presence (or the loss) of a Reset State command.
Render parameters appear in the media description in order of decreasing
priority. A receiver MAY use the priority ordering to decide which
renderer(s) to retain in a session.
If the "offer" in an Offer/Answer-style negotiation [13] contains a
media description with one or more render parameters, the "answer" MUST
set the render parameters of all unchosen renderers to "null".
C.5.2 The dirsense Parameter
As we describe in Section 6.1 in the main text, the nature of the
bidirectional data flows in a minimal RTP MIDI session depends on the
application. Some applications are analogous to telephony, where the
flows in each direction carry a musical conversation between two peers
[13]. Other applications use MIDI 1.0 DIN network semantics [1], where
one party is a server (example: a personal computer running a sequencer
program) and one party is a client (example: a rack-mount synthesizer
that is driven by the sequencer program).
The "dirsense" MIME parameter defines the bidirectional nature of a
renderer. The dirsense parameter describes the "render" parameter that
most recently precedes it in the session description. The dirsense
parameter MUST NOT appear in sessions descriptions that do not use the
"render" parameter, and MUST NOT appear before the first use of "render"
in the session description.
Lazzaro/Wawrzynek [Page 101]
INTERNET-DRAFT 22 June 2004
The dirsense parameter is assigned one of four token values: "unknown",
"sym", "fasym", or "rasym". The parameter describes the flow received
at the port indicated on the media ("m=") line of the stream associated
with the renderer. This definition is useful even if the session is
sendonly: by knowing the dirsense value of the "phantom" stream received
by a host, we can infer the dirsense value of the stream sent by a host.
The "unknown" token value leaves the bidirectional nature of the
renderer unspecified. It is the default value for native RTP MIDI
streams.
The "sym" token value indicates that the renderer uses the symmetrical
bidirectionality of telephony. It is the default dirsense value for
mpeg4-generic streams. In a sendrecv "dirsense=sym" stream with two
parties, two instances of the renderer run, one on each host.
The "fasym" and "rasym" values indicate that the stream uses the client-
server semantics of MIDI 1.0 DIN network cable pairs. The "fasym" value
indicates the forward flow from server to client; the "rasym value codes
the reverse flow from client to server. In a session that uses client-
server semantics, the host that receives the "fasym" flow runs an
instance of the renderer, but the host that receives the "rasym" flow
does not.
When "fasym" and "rasym" are used in an unicast Offer/Answer
negotiation, the value of dirsense is different in the offer and the
answer, because the port number on the media line is different. This
convention applies even if the session description is marked sendonly.
When used in a unidirectional RTP session, a "fasym" dirsense value
indicates that the renderer does not prefer to participate in a
bidirectional session that uses client-server semantics at the present
time, but is capable of doing so. A "sym" dirsense value in a
unidirectional session indicates that the renderer does not support
client-server semantics in bidirectional sessions, but does support
telephony semantics. An "unknown" diresense value indicates no support
for bidirectional sessions.
C.5.3 The rinit Parameter
The "rinit" MIME parameter defines the exact nature of the renderer. At
most one rinit parameter MAY follow a render parameter in a media
description. The value assigned to the rinit parameter MUST be the MIME
type/subtype [8] for the renderer.
Authors of rendering systems and MIDI APIs SHOULD register [21] a MIME
subtype for use with RTP MIDI.
Lazzaro/Wawrzynek [Page 102]
INTERNET-DRAFT 22 June 2004
A renderer that directly produces audio output SHOULD use the "audio"
MIME type. API presentation renderers, and renderers that control
primarily non-audio devices, SHOULD use the "application" MIME type.
The MIME subtype registration for a renderer MAY define a data object.
Additional MIME parameters (defined in Appendix C.5.4) MAY follow the
rinit parameter to encode an instance of the object. Alternatively, an
instance of the object MAY be made available through other means.
Renderers that directly produce audio or control output usually define a
data object that codes initialization data for the rendering algorithm.
The data object MAY also encapsulate a Standard MIDI File (SMF), so that
the data object may be used as a format for stored performances. By
default, the SMFs that are encapsulated in a data object MUST be ignored
by an RTP MIDI receiver. We define MIME parameters to override this
default in Appendix C.5.5.
Renderers that present a MIDI stream to an API usually define a data
object that is compatible with the API. The data object may describe
the hardware device that generates the stream (manufacturer, model,
etc), information about MIDI command semantics, and audio rendering
algorithm preferences and initialization data.
MIME registrations for renderers SHOULD specify the appropriate use of
the dirsense payload MIME parameter (Appendix C.5.2) with the renderer.
MIME registrations for renderers MAY specify additional Reset State
commands (Appendix A.1).
The "audio/octet-stream" or "application/octet-stream" type/subtypes MAY
be used as an rinit MIME type/subtypes, but this practice is
discouraged, as it limits interoperability. We refer to renderers
specified in this way as "opaque renderers", because clients do not
discover the renderer identity through the MIME system, but instead rely
on other means.
If a party is offered a session description that uses a renderer whose
rinit MIME subtype value is not known to the party, and if the party
needs to know about the subtype for correct operation, the party MUST
NOT accept the renderer. Options include rejecting the renderer (by
setting the "render" parameter to "null", NOT by setting the "rinit"
value to "null") or rejecting the entire session description.
Special rules apply to using the rinit parameter in an mpeg4-generic
stream. We define these rules in Appendix C.5.6.
Lazzaro/Wawrzynek [Page 103]
INTERNET-DRAFT 22 June 2004
C.5.4 Encoding rinit Data Objects
The "inline", "url", and "cid" MIME parameters MAY follow the rinit
parameter in a media description. These parameters encode the
initialization data object for the renderer.
The "inline" parameter supports the inline encoding of the data object.
The parameter is assigned a double-quoted Base64 [8] encoding of the
binary data object, with no line breaks.
The "url" parameter is assigned a double-quoted string representation of
a Uniform Resource Locator (URL) for the data object. If the URL points
to a MIME object, the object MUST have the MIME type/subtype value coded
by the rinit parameter.
The "cid" parameter supports data object caching. The parameter is
assigned a double-quoted string value that encodes a globally unique
identifier for the data object.
The cid parameter MAY immediately follow the inline or url parameters,
in which case the identifier MUST be associated with the inline or url
data object. If the url string points to a MIME object, the cid string
MUST match the Content-ID header [8] value of the object.
Alternatively, the cid parameter MAY be used without the inline and url
parameters. In this case, the identifier references a local or
distributed catalog of data objects.
In most cases, only one data object is coded in the media description
for each renderer. The correct receiver interpretation of multiple data
objects SHOULD be defined in the renderer MIME registration.
C.5.5 MIDI Channel Mapping
In this Appendix, we specify how to map MIDI name spaces (16 voice
channels + systems) onto a renderer.
In the general case:
o A session may define an ordered relationship (Appendix C.4)
that presents more than one MIDI name space to a renderer.
o A renderer may accept an arbitrary number of MIDI name spaces,
or may expect a specific number of MIDI name spaces.
A session description SHOULD provide a compatible MIDI name space to
each renderer in the session. If a receiver detects that a session
description has too many or too few MIDI name spaces for a renderer,
Lazzaro/Wawrzynek [Page 104]
INTERNET-DRAFT 22 June 2004
MIDI data from extra stream name spaces MUST be discarded, and extra
renderer name spaces MUST NOT be driven with MIDI data (except as
described in Appendix C.5.5.1 below).
If a media description defines several renderers and assigns the "all"
token value to the multimode parameter, the same name space is presented
to each renderer. However, the "chanmask" MIME parameter may be used to
mask out selected voice channels to each renderer. We define "chanmask"
and other MIDI management MIME parameters in the sub-sections below.
C.5.5.1 The smf_info MIME Parameter
The smf_info parameter defines the use of all SMFs encapsulated in
renderer data objects (if any). The smf_info parameter also defines the
use of SMFs coded in the smf_inline, smf_url, and smf_cid MIME
parameters (defined in Appendix C.5.5.2).
The smf_info parameter describes the "render" parameter that most
recently precedes it in the session description. The smf_info parameter
MUST NOT appear in sessions descriptions that do not use the "render"
parameter, and MUST NOT appear before the first use of "render" in the
session description.
We define three token values for smf_info: "ignore", "sdp_start", and
"identity":
o The "ignore" value indicates that the SMFs MUST be discarded.
This behavior is the default SMF rendering behavior.
o The "sdp_start" value codes that SMFs MUST be rendered,
and that the rendering MUST begin upon the acceptance of
the session description. If a receiver is offered a session
description with a renderer that uses an smf_info parameter
set to sdp_start, and if the receiver does not support
rendering SMFs, the receiver MUST NOT accept the renderer
associated with the smf_info parameter. Options include
rejecting the renderer (by setting the "render" parameter
to "null") or rejecting the entire session description.
o The "identity" value indicates the SMFs code the identity
of the renderer. The value is meant for use in session
management negotiations [13]. The MIDI commands coded in
the SMF are informational in nature, and MUST NOT be
presented to a renderer for audio presentation. In
typical use, the SMF would use SysEx Identity Reply
commands (F0 7E nn 06 02, as defined in [1]) to identify
devices, and use device-specific SysEx commands to describe
current state of the devices (patch memory contents, etc).
Lazzaro/Wawrzynek [Page 105]
INTERNET-DRAFT 22 June 2004
Other IETF standards-track documents MAY define additional token values
for smf_info.
If a party is offered a session description that uses an smf_info
parameter value that is not known to the party, the party MUST NOT
accept the renderer associated with the smf_info parameter. Options
include rejecting the renderer (by setting the "render" parameter to
"null") or rejecting the entire session description.
We now define the rendering semantics for the "sdp_start" token value in
detail.
The SMFs and RTP MIDI streams in a session description share the same
MIDI name space(s). In the simple case of a single RTP MIDI stream and
a single SMF, the SMF MIDI commands and RTP MIDI commands are merged
into a single name space and presented to the renderer. The indefinite
artifact responsibilities for merged MIDI streams defined in Appendix
C.4.1 also apply to merging RTP and SMF MIDI data.
If a media description codes multiple SMFs, the SMF name spaces are
presented as an ordered entity to the renderer. The first SMF maps to
the first renderer name space, the second SMF maps to the second
renderer name space, etc. If the associated RTP MIDI streams also form
an ordered relationship, the first SMF is merged with the first name
space of the relationship, the second SMF is merged to the second name
space of the relationship, etc.
Unless the streams and the SMFs both use MIDI Time Code, the time offset
between SMF and stream data is unspecified. This restriction may limit
the use of SMFs to applications where synchronization is not critical,
such as the transport of System Exclusive commands for renderer
initialization, or human-SMF interactivity.
C.5.5.2 The smf_inline, smf_url, and smf_cid MIME Parameters
In some applications, the renderer data object may not encapsulate SMFs,
but an application may wish to use SMFs in the manner defined in
Appendix C.5.5.1.
The "smf_inline", "smf_url", and "smf_cid" MIME parameters address this
situation. These parameters use the syntax and semantics of the inline,
url, and cid parameters defined in C.5.4, except that the encoded data
object is an SMF.
The "smf_inline", "smf_url", and "smf_cid" parameters belong to the
"render" parameter that most recently precedes it in the session
description. The "smf_inline", "smf_url", and "smf_cid" parameters MUST
NOT appear in sessions descriptions that do not use the "render"
Lazzaro/Wawrzynek [Page 106]
INTERNET-DRAFT 22 June 2004
parameter, and MUST NOT appear before the first use of "render" in the
session description. If several "smf_inline", "smf_url", or "smf_cid"
parameters appear for a renderer, the order of the parameters defines
the SMF name space ordering.
If smf_url points to a MIME object, the "application/octet-stream"
type/subtype SHOULD be used for the object.
C.5.5.3 The chanmask MIME Parameter
The chanmask MIME parameter instructs the renderer to ignore all MIDI
voice commands for certain channel numbers. The parameter value is a
concatenated string of "1" and "0" digits. Each string position maps to
a MIDI voice channel number (system channels may not be masked). A "1"
instructs the renderer to process the voice channel; a "0" instructs the
renderer to ignore the voice channel.
The string length of the chanmask parameter value MUST be 16 (for a
single stream or an identity relationship) or a multiple of 16 (for an
ordered relationship).
The chanmask parameter describes the "render" parameter that most
recently precedes it in the session description; chanmask MUST NOT
appear in sessions descriptions that do not use the "render" parameter,
and MUST NOT appear before the first use of "render" in the session
description.
The chanmask parameter describes the final MIDI name spaces presented to
the renderer. The SMF and stream components of the MIDI name spaces may
not be independently masked.
If a receiver is offered a session description with a renderer that uses
the chanmask parameter, and if the receiver does not implement the
semantics of the chanmask parameter, the receiver MUST NOT accept the
renderer unless the chanmask parameter value contains only "1"'s.
Lazzaro/Wawrzynek [Page 107]
INTERNET-DRAFT 22 June 2004
C.5.6 The audio/asc MIME Type
In Appendix H.3, we register the audio/asc MIME type. The data object
for audio/asc is a binary encoding of the AudioSpecificConfig data block
used to configure mpeg4-generic streams (Section 6.2 and [7]).
An mpeg4-generic media description MAY use the render and rinit
parameters with the audio/asc MIME type for renderer configuration.
Several restrictions apply to the use of these parameters in
mpeg4-generic media descriptions:
o An mpeg4-generic media description that uses the render parameter
MUST assign the empty string ("") to the mpeg4-generic "config"
parameter. The use of the streamtype, mode, and profile-level-id
MIME parameters MUST follow the normative text in Section 6.2.
o Sessions that use identity or ordered relationships MUST follow
the mpeg4-generic configuration restrictions in Appendix C.4.1.
o The render parameter MUST be assigned the value "synthetic",
"unknown", or "null". The "null" and "unknown" values are
intended for use in session management negotiation [13].
Other render token values MUST NOT be used.
o The rinit parameter MUST be assigned the value "audio/asc".
Other rinit token values MUST NOT be used.
o If the render parameter is assigned the value "synthetic",
the AudioSpecificConfig data object MUST be encoded using the
mechanisms defined in C.5.3-4 (including the mechanism in
C.5.3 for coding the data object by other means). The
AudioSpecificConfig data MUST encode one of the MPEG 4 Audio
Object Types defined for use with mpeg4-generic in Section 6.2.
o If the render parameter is assigned the value "null" or
"unknown", the data object MAY be omitted.
o The permissible values of the dirsense MIME parameter
(Appendix C.5.2) are "sym" and "unknown".
Several general restrictions apply to the use of the audio/asc MIME type
in RTP MIDI:
o A native stream MUST NOT assign "audio/asc" to rinit.
o The audio/asc MIME type defines a stored object type; it does
not define semantics for RTP streams. Thus, audio/asc MUST NOT
appear on an rtpmap line of a session description.
Lazzaro/Wawrzynek [Page 108]
INTERNET-DRAFT 22 June 2004
Below, we show session description examples for audio/asc. The session
description below uses the inline parameter to code the
AudioSpecificConfig block for a mpeg4-generic General MIDI stream.
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; config=""; profile-level-id=12;
render=synthetic; rinit="audio/asc";
inline="ehJNVGhkAAAABgAAAAEAYE1UcmsAAAAEAP8vAAA="
(The a=fmtp line has been wrapped to fit the page to accommodate
memo formatting restrictions; it comprises a single line in SDP)
The session description below uses the url MIME parameter to code the
AudioSpecificConfig block for the same General MIDI stream:
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; config=""; profile-level-id=12;
render=synthetic; rinit="audio/asc"; url="http://example.net/oski.asc";
cid="xjflsoeiurvpa09itnvlduihgnvet98pa3w9utnuighbuk"
(The a=fmtp line has been wrapped to fit the page to accommodate
memo formatting restrictions; it comprises a single line in SDP)
Lazzaro/Wawrzynek [Page 109]
INTERNET-DRAFT 22 June 2004
C.6 Interoperability
The parameters in Appendices C.1-5 support the creation of RTP MIDI
media sessions with a wide range of features. We expect that most RTP
MIDI senders and receivers will not implement the entire feature set.
In this section, we address the interoperability questions this
situation raises:
o What feature set should a typical RTP MIDI client be
expected to support?
o How should RTP MIDI session descriptions be constructed to
maximize interoperability?
RTP MIDI interoperability comes in two flavors: vertical within a class
of applications, and horizontal across application classes.
For vertical interoperability, we expect manufacturers of common types
of MIDI devices to collectively define RTP MIDI framework documents.
These frameworks may specify a required feature set for RTP MIDI senders
and receivers, and a stylized way to using parameters to express these
features. An example of a framework would be a MIDI content-streaming
framework for mobile telephones. Framework efforts may also register
renderer MIME types (Appendix C.5.3).
In this Appendix, we focus on horizontal interoperability across RTP
MIDI application classes. Figure C.6.1 summarizes the feature set we
expect the typical RTP MIDI implementation to support. The first item
listed in the "Section/Appendix" column for each feature indicates the
section of this Appendix that defines the feature set expectation.
Lazzaro/Wawrzynek [Page 110]
INTERNET-DRAFT 22 June 2004
-------------------------------------------------------------------
| Feature/Parameter | Any RTP MIDI client SHOULD | Section/Appendix |
|-------------------------------------------------------------------|
| transport | support one of: unicast | C.6.1, 6.1 |
| | UDP, multicast UDP, TCP. | |
| | sendrecv clients SHOULD | |
| | support sendonly/recvonly. | |
|-------------------------------------------------------------------|
| journal method | support transport default | C.6.1, C.1.1 |
|-------------------------------------------------------------------|
| sending policy | use closed-loop or anchor | C.6.1, C.1.2.1-2 |
|-------------------------------------------------------------------|
| chapter inclusion | use them to describe what | C.6.1, C.1.3 |
| parameters | it wants to send/receive. | |
| | MUST use them to satisfy | |
| | recovery journal mandate. | |
|-------------------------------------------------------------------|
| command timestamp | receivers: all three types | C.6.1.2, C.2 |
| semantic types | senders: at least one type | |
|-------------------------------------------------------------------|
| packet media time | senders: support short | C.6.3, C.3.1 |
| (ptime, maxptime) | times AND longer times. | |
| | receivers: support short | |
| | times AND/OR longer times. | |
|-------------------------------------------------------------------|
| guardtime param | fully support | C.6.3, C.3.2 |
|-------------------------------------------------------------------|
| ordered | not be expected to support | C.6.4, C.4.1 |
| relationships | | |
|-------------------------------------------------------------------|
| partitioned | support if bulk-data aware | C.6.4, C.4.1 |
| transport | and if not supporting | |
| | B.5.2 bulk-data approach | |
|-------------------------------------------------------------------|
| synchronization | support zerosync methods | C.6.4, C.4.2 |
|-------------------------------------------------------------------|
| rendering | describe and recognize | C.6.5, C.5 |
| specification | its rendering wishes in a | |
| | session description | |
|-------------------------------------------------------------------|
| chanmask param | fully support | C.6.5, C.5.5.3 |
|-------------------------------------------------------------------|
| dirsense param | fully support | C.6.5, C.5.2 |
|-------------------------------------------------------------------|
| smf_info param | support "ignore" value | C.6.5, C.5.5.1 |
-------------------------------------------------------------------
Figure C.6.1 -- Expected Feature Set of RTP MIDI client
Lazzaro/Wawrzynek [Page 111]
INTERNET-DRAFT 22 June 2004
In the interoperability text below, we do not use the RFC 2119 key words
MUST, MUST NOT, REQUIRED, SHALL, or SHALL NOT, except to repeat
normative responsibilities defined elsewhere in this memo. We avoid
these keywords so that the authors of framework documents for vertical
interoperability will be free to define restricted profiles for
resource-constrained application classes.
Unless noted otherwise, the interoperability text below applies to both
native and mpeg4-generic RTP MIDI media descriptions. The text also
assumes that a client initiating a session does not have advance
knowledge of the RTP MIDI features and preferences of other clients. If
such knowledge is available, clients SHOULD use it to simplify
negotiations (see Section 9 of [13] for details).
In the text below, we use the term "client" as a synonym for "senders
and receivers", and we use "party" as a synonym for "client". We use
"sender" or "receiver" in lieu of "client" or "party" for text that
applies only to senders or only to receivers.
We use the terms "offer" and "answer" in the context of Offer/Answer
protocol defined in [13], although the advice also applies to other
session negotiation methods. We use the term "reject an offer" as in
Section 4 of [13], and we use the term "reject a stream" as in Section 6
of [13].
C.6.1 Transport and Recovery Journal
A client SHOULD support at least one of the following transport types
for RTP MIDI streams: unicast UDP, multicast UDP, TCP. We expect a
substantial fraction of clients to support only one transport type.
Thus, an implementation that aims to achieve wide interoperability
SHOULD support multiple transport types.
When preparing an initial session description to offer to a party, a
client SHOULD pick a transport it believes that the party supports, but
SHOULD also be prepared for a rejection of the initial offer, followed
by a new offer from the party using an alternative transport. If no
such offer follows the rejection, the client SHOULD offer sessions with
other supported transport types before abandoning attempts to
communicate.
Clients SHOULD assume that other clients use the default journalling
method defined at the end of Section 6.1 (unreliable transport uses the
recovery journal, reliable transport does not use journalling). To
maximize interoperability given this assumption, the j_sec MIME
parameter (Appendix C.1.1) SHOULD NOT be used in initial offers. RTSP
implementors should examine Appendix C.1.1 for RTSP-specific
interoperability advice.
Lazzaro/Wawrzynek [Page 112]
INTERNET-DRAFT 22 June 2004
Clients SHOULD support at least one of the following sending policies:
the default closed-loop sending policy algorithm (Appendix C.1.2.2), the
anchor sending policy algorithm (Appendix C.1.2.1). An implementation
that aims to achieve wide interoperability SHOULD support multiple
sending policies.
To maximize interoperability, a client supporting both closed-loop and
anchor policies SHOULD request the closed-loop policy in its initial
offer to a party. However, the client SHOULD be prepared for a
rejection of the initial offer, followed by a new offer from the party
using a different policy. If no such offer follows the rejection, the
client SHOULD offer sessions with other supported policies before
abandoning attempts to communicate.
When preparing an initial offer to a receiver, a sender SHOULD use the
chapter inclusion parameters (Appendix C.1.3) to code the types of MIDI
commands it is able to send. When preparing an initial offer to a
sender, a receiver SHOULD use the chapter inclusion parameters to code
the types of MIDI commands it is able to render.
Clients SHOULD be prepared for a rejection of the initial offer,
followed by a new offer that expands or contracts the set of acceptable
MIDI commands using a revised chapter inclusion parameter configuration.
To maximize interoperability, clients SHOULD accept the revised
configuration offer if (1) the configuration does not significantly
impact the functionality of the application, and (2) the configuration
does not violate the recovery journal mandate. If the configuration is
rejected, the client SHOULD attempt to offer compromise configurations
before abandoning attempts to communicate.
Depending on its nature, a client may be capable of unidirectional
behavior (sending, or receiving) bidirectional behavior (sending and
receiving, simultaneously). A clients SHOULD be able to express and
recognize its stream directionality using the sendrecv, sendonly, and
recvonly attribute parameters [6] [13]. A client that supports sendrecv
behavior SHOULD also support sendonly and recvonly behaviors.
If "Client B" receives an offer for a sendrecv stream from "Client A",
but Client B is unable to use the parameter configuration in the stream
for both sending and receiving, Client B SHOULD reject the initial
offer.
Client B SHOULD follow the rejection with a revised offer to client A,
with two RTP MIDI streams (one sendonly, one recvonly), whose parameter
configuration resolves the compatibility issue via minimal changes to
the original offer. The streams in the revised offer SHOULD use the
musicport parameter (Appendix C.4.1) to indicate that the sendonly and
recvonly streams share an identity relationship.
Lazzaro/Wawrzynek [Page 113]
INTERNET-DRAFT 22 June 2004
If Client A receives a rejection of the offer from Client B as described
above, but does not receive a revised sendonly/recvonly offer following
the rejection, Client A SHOULD make a new offer that uses the
sendonly/recvonly stream pair before abandoning attempts to communicate.
In the Appendix C.6.2 that follows, we describe an example of a
situation where "Client B" would need to split a sendrecv session into
sendonly and recvonly sessions. To avoid repetition, we do include such
examples in Appendices C.6.3-5. However, note that situations may arise
with many of the parameters discussed in Appendix C which would require
a sendrecv session to split into sendonly and recvonly streams.
C.6.2 Command Timestamp Semantics
The linerate, mperiod, octpos, tsmode parameters (Appendix C.2) define
the temporal semantics of MIDI command timestamps. The tsmode parameter
sets the general type of semantics (async, buffer, and the default
comex).
In practice, most senders and receivers have a native internal
representation of time that uses one type of temporal semantics, and
supporting other semantic types may degrade temporal fidelity.
Receivers SHOULD be able to accept all three types of temporal
semantics. Senders SHOULD support at least one type of temporal
semantics. We expect a substantial fraction of senders to only support
one type of temporal semantics.
When preparing an initial offer to a sender, a receiver MAY use the
temporal semantics parameters to express a preference, or MAY refrain
from using the parameters. The latter choice codes a request for the
default comex semantics and a willingness to accept other semantics. In
either case, receivers SHOULD be prepared for a rejection, followed by a
new offer from the sender using revised temporal semantics. Receivers
SHOULD accept these revised semantics.
When preparing an initial offer to a receiver, a sender SHOULD use the
temporal semantics parameters to describe its preferred internal time
representation. If a sender does not have such a preference, sender
SHOULD NOT use the parameters: this offer codes a request for the
default comex semantics and a willingness to accept other semantics.
Upon reception of this offer, a receiver SHOULD accept the offer if it
supports the requested temporal semantics. Otherwise, the receiver
SHOULD reject the offer and propose its desired temporal semantics in an
offer to the sender.
In sendrecv sessions, the protocol described above may not be
successful. For example, if Client A only supports sending "async"
Lazzaro/Wawrzynek [Page 114]
INTERNET-DRAFT 22 June 2004
semantics, and Client B only supports sending "buffer" semantics, it is
not possible for both session participants to share a common view of the
session. If a client detects this situation, it SHOULD reject the
offer, and follow the rejection with a session description that uses two
unidirectional RTP MIDI sessions (one sendonly, one recvonly). The end
of Appendix C.6.1 describes this configuration.
C.6.3 Packet Timing Tools
The ptime and maxptime attribute parameters play a special role in RTP
MIDI. Appendix C.3.1 describes these parameters in detail, including
the semantics of 0 ms parameter values. The interoperability text below
assumes that readers have reviewed Appendix C.3.1.
For maximum interoperability, senders SHOULD support the generation of
media streams with both short packet times (ideally down to 0 ms) and
the longer packet times that support efficient content streaming
(ideally up to 200 ms or more). Receivers SHOULD NOT be expected to
perform well for both short and long packet times, although it improves
interoperability if they do.
When preparing an initial offer, clients that are predominantly
interactive (example: MIDI piano keyboard) or are predominantly content-
streaming (example: media player application) SHOULD use the ptime and
maxptime attributes to specify their nature. Clients that are agnostic
to packet timing SHOULD NOT use the attributes.
Upon receipt of an initial offer that uses ptime and/or maxptime,
receivers SHOULD reject offers that propose packet timing it cannot
support, and SHOULD make a new offer with acceptable timing. Senders,
however, SHOULD NOT reject an offer because of packet timing issues.
Instead, senders SHOULD accept the offer, and send packets in accordance
with the accepted ptime and maxptime values.
Upon the receipt of an initial offer that does not use ptime or
maxptime, receivers SHOULD use ptime and/or maxptime in the reply to
encode packet timing preferences, and senders SHOULD use ptime and/or
maxptime in the reply to encode packet timing intentions.
The guardtime MIME parameter (Appendix C.3.2) sets the minimum packet
sending rate for an RTP MIDI sessions. All clients SHOULD support this
parameter.
During session negotiations, a client SHOULD accept an offer with a
smaller guardtime value than it prefers. A client SHOULD reject an
offer with a larger guardtime value that it prefers, and follow this
rejection with a new offer using its preferred guardtime value. Note
that if the guardtime parameter does not appear in a session
Lazzaro/Wawrzynek [Page 115]
INTERNET-DRAFT 22 June 2004
description, the effective guardtime value is infinite.
C.6.4 Multiple Streams
Several RTP MIDI streams may appear in a media session. As described in
Appendix C.4, the musicport and zerosync MIME parameters may be used to
configure multi-stream sessions. Multi-stream RTP MIDI sessions may be
organized in two ways: ordered relationships and identity relationships.
Ordered relationships concatenate several 16-voice MIDI name spaces to
form a name space with a larger number of voices. Identity
relationships split a single 16-voice MIDI name space across several
streams, or act to associate sendonly and recvonly sessions to form a
"virtual" sendrecv session.
We expect a substantial fraction of clients to only support a single
16-voice MIDI name space. Thus, during session negotiations, clients
SHOULD NOT be expected to support ordered relationships.
If a client that does not support ordered relationships receives an
initial offer on an ordered relationship session, it SHOULD reject the
offer, and make a new offer for a session that uses the streams(s)
associated with the first 16 channels of the original ordered
relationship.
Clients that support sendrecv RTP MIDI sessions SHOULD support identity
relationships that associate sendonly and recvonly RTP MIDI sessions to
form a virtual sendrecv session. We discuss Offer/Answer techniques for
virtual sendrecv sessions in Appendices C.6.1-2.
The second major use for identity relationships in RTP MIDI is
"partitioned transport" -- the separation of large MIDI System Exclusive
commands ("bulk data") onto a stream that uses reliable transport.
Appendix B.5.2 describes partitioned transport in detail.
We expect that a substantial fraction of senders will not generate bulk
data commands, and we expect a significant fraction of receivers to be
unable to process received bulk data commands. Clients that do not use
bulk data commands SHOULD NOT be expected to use identity relationships
for partitioned transport.
If a client that does not support partitioned transport receives an
initial offer on a session that uses partitioned transport, the client
SHOULD reject the offer, and make a new offer that uses a single stream
to carry the offered MIDI name space. The negotiation process SHOULD be
used to discover a mutually acceptable way to handle the bulk data:
filtering bulk data from the stream, using reliable transport for the
stream, or using unreliable transport with the bulk data transmission
methods discussed in Appendix B.5.2.
Lazzaro/Wawrzynek [Page 116]
INTERNET-DRAFT 22 June 2004
For maximum interoperability, clients SHOULD support a zerosync-based
synchronization method (zero behavior or non-zero behavior, as defined
in Appendix C.4.2).
If a client receives an initial offer for a multi-stream session that
uses the zerosync parameter (Appendix C.4.2) in one of its two variants
(zero behavior or non-zero behavior), it MAY reject the offer and make a
new offer that uses the alternative variant.
Offers made to senders that use the zerosync parameter in non-zero mode
SHOULD use arbitrary non-zero values for the parameter. In the reply,
the sender MAY replace these values with alternative values, or MAY keep
the proposed values. However, a receiver (or a sendrecv client) that is
proffered an offer that uses the zerosync parameter in non-zero mode
SHOULD NOT change the zerosync parameter values to other non-zero
values.
We expect that a significant fraction of multi-stream clients will not
support the traditional RTP synchronization methods used for lip-sync
[2]. These methods use the RTCP fields for NTP and RTP timestamps to
synchronize multiple streams. Thus, during session negotiations,
clients SHOULD NOT be expected to accept multi-stream RTP MIDI sessions
that do not use the zerosync parameter.
C.6.5 MIDI Rendering
Sections 6.1-2 in the main text describe the default rendering behavior
for RTP MIDI streams. Sessions may use the "render" MIME parameter and
associated parameters (Appendix C.5) to override the default rendering
behavior. Below, we discuss rendering interoperability issues.
For some clients, the identity of the renderer is irrelevant (example: a
receiver that is transcoding an incoming RTP MIDI stream onto a MIDI 1.0
DIN cable, with no knowledge of what device is connected to the other
end of the MIDI DIN cable).
For other clients, the identity of the renderer is essential (example: a
receiver that uses a General MIDI [1] synth would not want to receive a
performance composed for a DLS 2 synth [9], but instead would want to
receive the General MIDI version of the performance).
If the renderer identity is irrelevant to a client, the initial offer
made by the client SHOULD use a single "render" parameters set to the
"unknown" render value. An "rinit" parameter (Appendix C.5.3) SHOULD
NOT follow the render parameter, as this parameter combination has a
special meaning (described later in this Appendix).
If the client wishes to specify a renderer, the initial offer made by
Lazzaro/Wawrzynek [Page 117]
INTERNET-DRAFT 22 June 2004
the client SHOULD use the render parameter (and, if needed, other
parameters defined in Appendix C.5) to describe the desired renderer.
If the client wishes to offer multiple rendering options, it SHOULD do
so, via the default "one" semantics for the "multimode" parameter, as
described in Appendix C.5.1. As "one" is the default value for the
multimode parameter, the multimode parameter itself need not appear in
the session description. Receivers SHOULD NOT be expected to render a
stream using multiple renderers simultaneously (i.e. "all" multimode
semantics).
If a client receives an offer using the "unknown" renderer without an
"rinit" qualifier, the client SHOULD accept the offer. The answer
SHOULD describe the client's preferred renderer for the stream. An
answer that keeps the render value as "unknown" signals that the
answerer also has no rendering preferences.
Otherwise, if a client receives an offer that specifies a renderer other
than "unknown", and renderer identity is irrelevant to the client, the
client SHOULD accept the offer. If the accepted offer presents multiple
rendering options using the "one" semantics for the "multimode"
parameter, the reply SHOULD mark all but the first listed offer in the
answer as "null".
However, if a client receives an offer, and the identity of the renderer
IS important to the client, the client SHOULD examine the offer for the
presence of an acceptable renderer (including the acceptability of
supporting MIME parameters for the renderer).
If an acceptable renderer appears, the client SHOULD accept the offer.
Otherwise, the client SHOULD reject the offer, and make a revised offer
to the offering client that displays acceptable rendering options. Note
that if an offer is accepted, and the offer presents multiple rendering
options, in the answer the "render" parameter for all unchosen renderers
MUST be marked "null", as explained in Appendix C.5.1.
In some cases, a client may wish to specify the desired "rinit" MIME
type it wishes to use for rendering a session in an offer, but is
willing to let the other client specify the data object for the MIME
type (see Appendix C.5.3 for details). In this case, a client SHOULD
create an offer that includes a "render" parameter set to "unknown",
followed by an "rinit" parameter set to the desired MIME type, with no
data object specification parameters following rinit.
If a client wishes to specify an mpeg4-generic stream renderer in this
partial fashion, the render parameter in the initial offer is set to
"unknown", rinit is set to "audio/asc", and the "profile-level-id"
parameter encodes the profile and level of the desired MPEG 4 renderer
Lazzaro/Wawrzynek [Page 118]
INTERNET-DRAFT 22 June 2004
(see Section 6.2 for details).
If the recipient of an offer that includes a partially-specified
renderer wishes to use the renderer, the recipient SHOULD first reject
the offer. Then, the recipient SHOULD prepare a new offer that (1) sets
"render" to "synthetic" or "api" as appropriate (2) sets "rinit" to the
originally-offered MIME type, and (3) specifies a data object for the
renderer.
Receivers SHOULD implement the chanmask parameter (Appendix C.5.5.3).
Receivers SHOULD NOT be expected to implement smf_info parameter values
other than "ignore" (Appendix C.5.5.1).
Bidirectional sessions whose bidirectional nature (Appendix C.5.2)
differs from the default SHOULD use the "dirsense" parameter (Appendix
C.5.2) to declare the nature of the session. This recommendation
applies both true sendrecv sessions, and to the "virtual" sendrecv
sessions as described in Appendices C.4 and C.6.4.
Lazzaro/Wawrzynek [Page 119]
INTERNET-DRAFT 22 June 2004
D. Parameter Syntax Definitions
In this Appendix, we define the syntax for the RTP MIDI MIME parameters
in Augmented Backus-Naur Form (ABNF, [10]). All MIME parameters MUST
appear on a single fmtp attribute line of an RTP MIDI media description.
For mpeg4-generic RTP MIDI streams, this line MUST also include any
mpeg4-generic MIME parameters (usage described in Section 6.2). An fmtp
attribute line may be defined (after [4]) as:
;
; SDP fmtp line definition
;
fmtp = "a=fmtp:" token SP param-assign 0*(";" SP param-assign) CRLF
where <token> codes the RTP payload type. Note that white space MUST
NOT appear between the "a=fmtp:" and the RTP payload type.
We now define the syntax of the MIME parameters defined in Appendix C.
The definition takes the form of the incremental assembly of the <param-
assign> token. See [4] for the syntax of the mpeg4-generic MIME
parameters discussed in Section 6.2.
;
;
; top-level definition for all parameters
;
;
;
; Parameters defined in Appendix C.1
param-assign = "j_sec" "=" ("none" / "recj" / *ietf-extension)
param-assign /= "j_update" "=" ("anchor" / "closed-loop" / "open-loop"
/ *ietf-extension)
param-assign /= "ch_default" "=" ([channel-list] chapter-list [f-list])
/ sysex-data
param-assign /= "ch_unused" "=" ([channel-list] chapter-list [f-list])
/ sysex-data
param-assign /= "ch_never" "=" ([channel-list] chapter-list [f-list])
/ sysex-data
param-assign /= "ch_anchor" "=" ([channel-list] chapter-list [f-list])
/ sysex-data
Lazzaro/Wawrzynek [Page 120]
INTERNET-DRAFT 22 June 2004
param-assign /= "ch_active" "=" ([channel-list] chapter-list [f-list])
;
; sysex-data NOT an option for ch_active
;
; Parameters defined in Appendix C.2
param-assign /= "tsmode" "=" ("comex" / "async" / "buffer")
param-assign /= "linerate" "=" nonzero-four-octet
param-assign /= "octpos" "=" ("first" / "last")
param-assign /= "mperiod" "=" nonzero-four-octet
;
; Parameter defined in Appendix C.3
param-assign /= "guardtime" "=" nonzero-four-octet
;
; Parameters defined in Appendix C.4
param-assign /= "musicport" "=" four-octet
param-assign /= "zerosync" "=" four-octet
;
; Parameters defined in Appendix C.5
param-assign /= "chanmask" "=" 1*( 16( "0" / "1" ) )
param-assign /= "cid" "=" double-quote cid-block double-quote
param-assign /= "dirsense" "=" ("fasym" / "rasym" / "sym" / "unknown")
param-assign /= "inline" "=" double-quote base-64-block double-quote
param-assign /= "multimode" "=" ("all" / "one")
param-assign /= "render" "=" ("synthetic" / "api" / "null" /
"unknown" / *ietf-extension)
param-assign /= "rinit" "=" mime-type "/" mime-subtype
param-assign /= "smf_cid" "=" double-quote cid-block double-quote
param-assign /= "smf_info" "=" ("ignore" / "identity" / "sdp_start"
Lazzaro/Wawrzynek [Page 121]
INTERNET-DRAFT 22 June 2004
/ *ietf-extension)
param-assign /= "smf_inline" "=" double-quote base-64-block double-quote
param-assign /= "smf_url" "=" double-quote uri-element double-quote
param-assign /= "url" "=" double-quote uri-element double-quote
;
; list definitions for the ch_ chapter-list
;
chapter-list = chapter-part1 chapter-part2 chapter-part3
chapter-part1 = 0*1"A" 0*1"B" 0*1"C" 0*1"D" 0*1"E" 0*1"F" 0*1"G"
chapter-part2 = 0*1"H" 0*1"J" 0*1"K" 0*1"M" 0*1"N" 0*1"P" 0*1"Q"
chapter-part3 = 0*1"T" 0*1"V" 0*1"W" 0*1"X" 0*1"Y" 0*1"Z"
;
; list definitions for the ch_ channel-list
;
channel-list = midi-chan-element *("." midi-chan-element)
midi-chan-element = midi-chan / midi-chan-range
midi-chan-range = midi-chan "-" midi-chan
; decimal value of left midi-chan
; MUST be strictly less than decimal
; value of right midi-chan
midi-chan = %d0-15
;
; list definitions for the ch_ field list (f-list)
;
f-list = midi-field-element *("." midi-field-element)
midi-field-element = midi-field / midi-field-range
midi-field-range = midi-field "-" midi-field
;
; decimal value of left midi-field
; MUST be strictly less than decimal
Lazzaro/Wawrzynek [Page 122]
INTERNET-DRAFT 22 June 2004
; value of right midi-field
midi-field = four-octet
;
; large range accommodates Chapter M
; RPN (0-16383) and NRPN (16384-32767)
; parameters, and Chapter X octet sizes.
;
; definitions for ch_ sysex-data
;
sysex-data = "__" h-list *("_" h-list) "__"
h-list = hex-field-element *("." hex-field-element)
hex-field-element = hex-octet / hex-field-range
;
; hex-octet as defined on page 23 of [8]
; however, hex-octet values MUST NOT exceed 7F.
hex-field-range = hex-octet "-" hex-octet
;
; hexadecimal value of left hex-octet
; MUST be strictly less than hexadecimal
; value of right hex-octet
;
; definitions for rinit MIME parameter
;
mime-type = type
;
; as defined on page 12 in [8]
mime-subtype = subtype
;
; as defined on page 12 in [8]
;
; generic rules
;
ietf-extension = token
;
; token as defined in reference [6].
; ietf-extension may only be defined in
; standards-track RFCs (Section 7).
Lazzaro/Wawrzynek [Page 123]
INTERNET-DRAFT 22 June 2004
four-octet = %d0-429496729
; unsigned encoding of 32-bits
nonzero-four-octet = %d1-429496729
; unsigned encoding of 32-bits, ex-zero
uri-element = uri
; as defined in reference [6].
base-64-block = base64
; as defined in reference [6].
double-quote = %x22
; the double-quote (") character
cid-block = msg-id
; as discussed in Section 7 of
; reference [8]
;
; End of ABNF
The mpeg4-generic RTP payload [4] defines a "mode" parameter that
signals the type of MPEG stream in use. We add a new mode value, "rtp-
midi", using the ABNF rule below:
;
; mpeg4-generic mode parameter extension
;
mode /= "rtp-midi"
; as described in Section 6.2 of this memo
Lazzaro/Wawrzynek [Page 124]
INTERNET-DRAFT 22 June 2004
E. A MIDI Overview for Networking Specialists
This Appendix presents an overview of the MIDI standard, for the benefit
of networking specialists new to musical applications. Implementors
should consult [1] for a normative description of MIDI.
Musicians make music by performing a controlled sequence of physical
movements. For example, a pianist plays by coordinating a series of key
presses, key releases, and pedal actions. MIDI represents a musical
performance by encoding these physical gestures as a sequence of MIDI
commands. This high-level musical representation is compact but
fragile: one lost command may be catastrophic to the performance.
MIDI commands have much in common with the machine instructions of a
microprocessor. MIDI commands are defined as binary elements.
Bitfields within a MIDI command have a regular structure and a
specialized purpose. For example, the upper nibble of the first command
octet (the opcode field) codes the command type. MIDI commands may
consist of an arbitrary number of complete octets, but most MIDI
commands are 1, 2, or 3 octets in length.
-------------------------------------------------------------
| Name | Bitfield Pattern |
|-------------------------------------------------------------|
| NoteOff (end a note) | 1000cccc 0nnnnnnn 0vvvvvvv |
|-------------------------------------------------------------|
| NoteOn (start a note) | 1001cccc 0nnnnnnn 0vvvvvvv |
|-------------------------------------------------------------|
| PTouch (Polyphonic Aftertouch) | 1010cccc 0nnnnnnn 0aaaaaaa |
|-------------------------------------------------------------|
| CControl (Controller Change) | 1011cccc 0xxxxxxx 0yyyyyyy |
|-------------------------------------------------------------|
| PChange (Program Change) | 1100cccc 0ppppppp |
|-------------------------------------------------------------|
| CTouch (Channel Aftertouch) | 1101cccc 0aaaaaaa |
|-------------------------------------------------------------|
| PWheel (Pitch Wheel) | 1110cccc 0xxxxxxx 0yyyyyyy |
|-------------------------------------------------------------|
| System (sub-opcode is xxxx) | 1111xxxx ... |
-------------------------------------------------------------
Figure E.1 -- MIDI command chart
Figure E.1 shows the MIDI command family. There are two major classes
of commands: voice commands (opcode field values in the range 0x8
through 0xE) and system commands (opcode field value 0xF). Voice
Lazzaro/Wawrzynek [Page 125]
INTERNET-DRAFT 22 June 2004
commands code the musical gestures for each timbre in a composition.
Systems commands perform housekeeping functions, such as System Reset
(the one-octet command 0xFF).
E.1 Commands Types
Voice commands execute on one of 16 MIDI channels, as coded by its 4-bit
channel field (field cccc in Figure E.1). In most applications, notes
for different timbres are assigned to different channels. To support
applications that require more than 16 channels, MIDI systems use
several MIDI command streams in parallel, to yield 32, 48, or 64 MIDI
channels.
As an example of a voice command, consider a NoteOn command (opcode
0x9), with binary encoding 1001cccc 0nnnnnnn 0aaaaaaa. This command
signals the start of a musical note on MIDI channel cccc. The note has
a pitch coded by the note number nnnnnnn, and an onset amplitude coded
by note velocity aaaaaaa.
Other voice commands signal the end of notes (NoteOff, opcode 0x8), map
a specific timbre to a MIDI channel (PChange, opcode 0xC), or set the
value of parameters that modulate the timbral quality (all other voice
commands). The exact meaning of most voice channel commands depends on
the rendering algorithms the MIDI receiver uses to generate sound. In
most applications, a MIDI sender has a model (in some sense) of the
rendering method used by the receiver.
E.2 Running Status
All MIDI command bitfields share a special structure: the leading bit of
the first octet is set to 1, and the leading bit of all subsequent
octets is set to 0. This structure supports a data compression system,
called running status [1], that improves the coding efficiency of MIDI.
In running status coding, the first octet of a MIDI voice command may be
dropped if it is identical to the first octet of the previous MIDI voice
command. This rule, in combination with a convention to consider NoteOn
commands with a null third octet as NoteOff commands, supports the
coding of note sequences using two octets per command.
E.3 Command Timing
The bitfield formats in Figure E.1 do not encode the execution time for
a command. Timing information is not a part of the MIDI command syntax
itself; different applications of the MIDI command language use
different methods to encode timing.
Lazzaro/Wawrzynek [Page 126]
INTERNET-DRAFT 22 June 2004
For example, the MIDI command set acts as the transport layer for MIDI
1.0 DIN cables [1]. MIDI cables are short asynchronous serial lines
that facilitate the remote operation of musical instruments and audio
equipment. Timestamps are not sent over a MIDI 1.0 DIN cable. Instead,
the standard uses an implicit "time of arrival" code. Receivers execute
MIDI commands at the moment of arrival.
In contrast, Standard MIDI Files (SMFs, [1]), a file format for
representing complete musical performances, add a explicit timestamp to
each MIDI command, using a delta encoding scheme that is optimized for
statistics of musical performance. SMF timestamps usually code timing
using the metric notation of a musical score. SMF meta-events are used
to add a tempo map to the file, so that score beats may be accurately
converted into units of seconds during rendering.
F. Acknowledgements
We thank the networking, media compression, and computer music community
members who have commented or contributed to the effort, including Kurt
B, Cynthia Bruyns, Steve Casner, Paul Davis, Robin Davies, Joanne Dow,
Dominique Fober, Philippe Gentric, Michael Godfrey, Chris Grigg, Todd
Hager, Michel Jullian, Phil Kerr, Young-Kwon Lim, Jessica Little, Jan
van der Meer, Colin Perkins, Charlie Richmond, Herbie Robinson, Larry
Rowe, Eric Scheirer, Dave Singer, Martijn Sipkema, William Stewart, Kent
Terry, Magnus Westerlund, Tom White, Jim Wright, Doug Wyatt, and Giorgio
Zoia. We also thank the members of the San Francisco Bay Area music and
audio community for creating the context for the work, including Don
Buchla, Chris Chafe, Richard Duda, Dan Ellis, Adrian Freed, Ben Gold,
Jaron Lanier, Roger Linn, Richard Lyon, Dana Massie, Max Mathews, Keith
McMillen, Carver Mead, Nelson Morgan, Tom Oberheim, Malcolm Slaney, Dave
Smith, Julius Smith, David Wessel, and Matt Wright.
Lazzaro/Wawrzynek [Page 127]
INTERNET-DRAFT 22 June 2004
G. Security Considerations
Implementors should carefully read the Security Considerations sections
of the RTP [2] and RTP/AVP [3] documents, as the issues discussed in
these sections directly apply to RTP MIDI streams. Implementors should
also review the Secure Real-time Transport Protocol (SRTP, [12]), an RTP
profile that addresses the security issues discussed in [2] [3].
In this Appendix, we discuss security issues that are unique to the RTP
MIDI payload format.
When using RTP MIDI, authentication of incoming RTP and RTCP packets is
RECOMMENDED. Per-packet authentication may be provided by SRTP or by
other means. Without the use of authentication, attackers could forge
MIDI commands into an ongoing stream, damaging speakers and eardrums.
An attacker could also craft RTP and RTCP packets to exploit known bugs
in the client, and take effective control of a client machine.
Session management tools (such as SIP [16]) SHOULD use authentication
during the transport of all session descriptions containing RTP MIDI
media streams. For SIP, the Security Considerations section in [16]
provides an overview of possible authentication mechanisms. RTP MIDI
session descriptions should use authentication because the session
descriptions may code initialization data using the MIME parameters
described in Appendix C. If an attacker inserts bogus initialization
data into a session description, he can corrupt the session or forge an
client attack.
Session descriptions may also code renderer initialization data by
reference, via the url (Appendix C.5.4) and smf_url (Appendix C.5.5.2)
parameters. If the coded URL is spoofed, both session and client are
open to attack, even if the session description itself is authenticated.
The zerosync MIME parameter (described in Appendix C.4.2) impairs a
security feature of RTP. In standard RTP, the RTP timestamp is
initialized to a randomly chosen value, to reduce the predictability of
the header. If zerosync is used in a media description, this security
feature is partially (for non-zero zerosync values) or totally (if
zerosync is set to zero) disabled.
Lazzaro/Wawrzynek [Page 128]
INTERNET-DRAFT 22 June 2004
H. IANA Considerations
In this Appendix, we register the audio/rtp-midi and audio/asc MIME
types, and we extend the audio/mpeg4-generic MIME type [4]. The
audio/rtp-midi and audio/asc registrations are in the IETF tree.
H.1 rtp-midi MIME Registration
This section registers rtp-midi as a MIME subtype for the audio type.
MIME media type name:
audio
MIME subtype name:
rtp-midi
Required parameters:
rate: The RTP timestamp clock rate, as specified in the rtpmap
line. See Sections 2.1 and 6.1 for usage details.
Optional parameters:
Standard SDP attributes:
maxptime: See Appendix C.3.1 for usage details.
ptime: See Appendix C.3.1 for usage details.
Non-extensible parameters:
ch_active: See Appendix C.1.3 for usage details.
ch_anchor: See Appendix C.1.3 for usage details.
ch_default: See Appendix C.1.3 for usage details.
ch_never: See Appendix C.1.3 for usage details.
ch_unused: See Appendix C.1.3 for usage details.
chanmask: See Appendix C.5.5.3 for usage details.
cid: See Appendix C.5.4 for usage details.
dirsense: See Appendix C.5.2 for usage details.
guardtime: See Appendix C.3.2 for usage details.
inline: See Appendix C.5.4 for usage details.
Lazzaro/Wawrzynek [Page 129]
INTERNET-DRAFT 22 June 2004
linerate: See Appendix C.2 for usage details.
mperiod: See Appendix C.2 for usage details.
multimode: See Appendix C.5.1 for usage details.
musicport: See Appendix C.4.1 for usage details.
octpos: See Appendix C.2 for usage details.
tsmode: See Appendix C.2 for usage details.
smf_cid: See Appendix C.5.5.2 for usage details.
smf_inline: See Appendix C.5.5.2 for usage details.
smf_url: See Appendix C.5.5.2 for usage details.
url: See Appendix C.5.4 for usage details.
zerosync: See Appendix C.4.2 for usage details.
Extensible parameters:
j_sec, j_update:
See Appendix C.1 for usage details. The parameters
may only be extended via an IETF standards-track
document.
render:
See Appendix C.5 for usage details. The parameter may
only be extended via an IETF standards-track document.
rinit:
The parameter MUST be assigned to a MIME type/subtype [8].
Thus, the MIME registration mechanism acts as the extension
mechanism for the parameter. Appendix C.5.3 provides
guidance to authors of MIME subtype registrations for
use with the rinit parameter.
smf_info:
See Appendix C.5.5.1 for usage details. The parameter
may only be extended via an IETF standards-track document.
Encoding considerations:
This type is only defined for real-time transfers of MIDI
streams via RTP. Stored-file semantics for rtp-midi may
be defined in the future.
Security considerations:
Lazzaro/Wawrzynek [Page 130]
INTERNET-DRAFT 22 June 2004
See Appendix G of this memo.
Interoperability considerations:
None.
Published specification:
This memo and [1] serve as the normative specification. In
addition, references [14], [15], and [19] provide non-normative
implementation guidance.
Applications which use this media type:
Audio content-creation hardware, such as MIDI controller piano
keyboards and MIDI audio synthesizers. Audio content-creation
software, such as music sequencers, digital audio workstations,
and soft synthesizers. Computer operating systems, for network
support of MIDI Application Programmer Interfaces. Content
distribution servers and terminals may use this media type for
low bit-rate music coding.
Additional information:
None.
Person & email address to contact for further information:
John Lazzaro <lazzaro@cs.berkeley.edu>
Intended usage:
COMMON.
Author/Change controller:
John Lazzaro <lazzaro@cs.berkeley.edu>
H.2 mpeg4-generic MIME Registration
Lazzaro/Wawrzynek [Page 131]
INTERNET-DRAFT 22 June 2004
The mpeg4-generic MIME type [4] permits extensions to support new modes.
The registration below defines mode rtp-midi for mpeg4-generic, to
support the MPEG Audio codecs [5] that use MIDI.
MIME media type name:
audio
MIME subtype name:
mpeg4-generic
Required parameter extensions:
We extend the mpeg4-generic required parameter mode, by
adding the value=parameter syntax:
mode=rtp-midi
to the list of legal mode values defined in [4]. See
Section 6.2 for usage details.
rate: In mode rtp-midi, rate is a required parameter. Rate
specifies the RTP timestamp clock rate on the rtpmap line.
See Sections 2.1 and 6.2 for usage details.
Optional parameters:
Standard SDP attributes:
maxptime: See Appendix C.3.1 for usage details.
ptime: See Appendix C.3.1 for usage details.
Non-extensible parameters:
ch_active: See Appendix C.1.3 for usage details.
ch_anchor: See Appendix C.1.3 for usage details.
ch_default: See Appendix C.1.3 for usage details.
ch_never: See Appendix C.1.3 for usage details.
ch_unused: See Appendix C.1.3 for usage details.
chanmask: See Appendix C.5.5.3 for usage details.
cid: See Appendix C.5.4 for usage details.
dirsense: See Appendix C.5.2 for usage details.
Lazzaro/Wawrzynek [Page 132]
INTERNET-DRAFT 22 June 2004
guardtime: See Appendix C.3.2 for usage details.
inline: See Appendix C.5.4 for usage details.
linerate: See Appendix C.2 for usage details.
mperiod: See Appendix C.2 for usage details.
multimode: See Appendix C.5.1 for usage details.
musicport: See Appendix C.4.1 for usage details.
octpos: See Appendix C.2 for usage details.
rinit: See Appendix C.5.3 for usage details.
Also, Appendix C.5.6 defines rinit as
non-extensible in mpeg4-generic use.
tsmode: See Appendix C.2 for usage details.
smf_cid: See Appendix C.5.5.2 for usage details.
smf_inline: See Appendix C.5.5.2 for usage details.
smf_url: See Appendix C.5.5.2 for usage details.
url: See Appendix C.5.4 for usage details.
zerosync: See Appendix C.4.2 for usage details.
Extensible parameters:
j_sec, j_update:
See Appendix C.1 for usage details. The parameters
may only be extended via an IETF standards-track
document.
render:
See Appendix C.5 for usage details. The parameter may
only be extended via an IETF standards-track document.
smf_info:
See Appendix C.5.5.1 for usage details. The parameter
may only be extended via an IETF standards-track document.
Encoding considerations:
Only defined for real-time transfers of audio/mpeg4-generic
RTP streams with mode=rtp-midi.
Security considerations:
See Appendix G of this memo.
Lazzaro/Wawrzynek [Page 133]
INTERNET-DRAFT 22 June 2004
Interoperability considerations:
Except for the marker bit (Section 2.1), the packet formats
for audio/rtp-midi and audio/mpeg4-generic (mode rtp-midi)
are identical. The formats differ in use: audio/mpeg4-generic
is for MPEG work, audio/rtp-midi is for all other work.
Published specification:
This memo, [1], and [5] are the normative references. In
addition, references [14], [15], and [19] provide non-normative
implementation guidance.
Applications which use this media type:
MPEG 4 servers and terminals that support [5].
Additional information:
None.
Person & email address to contact for further information:
John Lazzaro <lazzaro@cs.berkeley.edu>
Intended usage:
COMMON.
Author/Change controller:
John Lazzaro <lazzaro@cs.berkeley.edu>
H.3 asc MIME Registration
This section registers asc as a MIME subtype for the audio type.
MIME media type name:
Lazzaro/Wawrzynek [Page 134]
INTERNET-DRAFT 22 June 2004
audio
MIME subtype name:
asc
Required parameters:
none
Optional parameters:
none
Encoding considerations:
This type is only defined for data object (stored file)
transfer. The native form of the data object is binary
data, zero-padded to an octet boundary. The most common
transports for the type are HTTP and SMTP.
Security considerations:
See Appendix G of this memo.
Interoperability considerations:
None.
Published specification:
The audio/asc data object is the AudioSpecificConfig
binary data structure, which is normatively defined in [7].
Applications which use this media type:
MPEG 4 Audio servers and terminals which support
audio/mpeg4-generic RTP streams for mode rtp-midi.
Lazzaro/Wawrzynek [Page 135]
INTERNET-DRAFT 22 June 2004
Additional information:
None.
Person & email address to contact for further information:
John Lazzaro <lazzaro@cs.berkeley.edu>
Intended usage:
COMMON.
Author/Change controller:
John Lazzaro <lazzaro@cs.berkeley.edu>
I. References
I.1 Normative References
[1] MIDI Manufacturers Association. "The Complete MIDI 1.0 Detailed
Specification", 1996.
[2] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson.
"RTP: A transport protocol for real-time applications", RFC 3550, July
2003.
[3] Schulzrinne, H., and S. Casner. "RTP Profile for Audio and Video
Conferences with Minimal Control", RFC 3551, July 2003.
[4] van der Meer, J., Mackie, D., Swaminathan, V., Singer, D., and
P. Gentric. "RTP Payload Format for Transport of MPEG-4 Elementary
Streams", RFC 3640, November 2003.
[5] International Standards Organization. "ISO/IEC 14496 MPEG-4",
Part 3 (Audio), Subpart 5 (Structured Audio), 2001.
[6] Handley, M., Jacobson, V., and C. Perkins. "SDP: Session
Description Protocol", draft-ietf-mmusic-sdp-new-18.txt.
[7] International Standards Organization. "ISO 14496 MPEG-4", Part 3
(Audio), 2001.
[8] Freed, N. and N. Borenstein. "MIME Part One: Format of Internet
Lazzaro/Wawrzynek [Page 136]
INTERNET-DRAFT 22 June 2004
Message Bodies", RFC 2045, November 1996.
[9] MIDI Manufacturers Association. "The MIDI Downloadable Sounds
Specification", v98.2, 1998.
[10] Crocker, D. and P. Overell. "Augmented BNF for Syntax
Specifications: ABNF.", RFC 2234, November 1997.
[11] Bradner, S. "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
[12] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. Norrman.
"The Secure Real-time Transport Protocol (SRTP)", RFC 3711, March 2004.
[13] J. Rosenberg and H. Schulzrinne. "An Offer/Answer Model with
SDP", RFC 3264, June 2002.
I.2 Informative References
[14] Lazzaro, J. and J. Wawrzynek. "A Case for Network Musical
Performance", 11th International Workshop on Network and Operating
Systems Support for Digital Audio and Video (NOSSDAV 2001) June 25-26,
2001, Port Jefferson, New York.
[15] Fober, D., Orlarey, Y. and S. Letz. "Real Time Musical Events
Streaming over Internet", Proceedings of the International Conference
on WEB Delivering of Music 2001, pages 147-154.
[16] Rosenberg, J, Schulzrinne, H., Camarillo, G., Johnston, A.,
Peterson, J., Sparks, R., Handley, M., and E. Schooler. "SIP: Session
Initiation Protocol", RFC 3261, June 2002.
[17] Schulzrinne, H., Rao, A., and R. Lanphier. "Real Time Streaming
Protocol (RTSP)", RFC 2326, April 1998.
[18] Clark, D. D. and D. L. Tennenhouse. "Architectural considerations
for a new generation of protocols", SIGCOMM Symposium on
Communications Architectures and Protocols , (Philadelphia,
Pennsylvania), pp. 200--208, IEEE, Sept. 1990.
[19] Lazzaro, J., and J. Wawrzynek. "An Implementation Guide for RTP
MIDI", draft-ietf-avt-rtp-midi-guidelines-05.txt.
[20] Braden, R. et al. "Resource ReSerVation Protocol (RSVP) --
Version 1 Functional Specification", RFC 2205, September 1997.
[21] Freed, N., Klensin, J., and J. Postel. "MIME Part Four:
Lazzaro/Wawrzynek [Page 137]
INTERNET-DRAFT 22 June 2004
Registration Procedures", RFC 2048, November 1996.
[22] Lazzaro, J. "Framing RTP and RTCP Packets over
Connection-Oriented Transport",
draft-ietf-avt-rtp-framing-contrans-02.txt.
J. Authors' Addresses
John Lazzaro (corresponding author)
UC Berkeley
CS Division
315 Soda Hall
Berkeley CA 94720-1776
Email: lazzaro@cs.berkeley.edu
John Wawrzynek
UC Berkeley
CS Division
631 Soda Hall
Berkeley CA 94720-1776
Email: johnw@cs.berkeley.edu
K. Intellectual Property Rights Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in this
document or the extent to which any license under such rights might or
might not be available; nor does it represent that it has made any
independent effort to identify any such rights. Information on the
procedures with respect to rights in RFC documents can be found in BCP
78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an attempt
made to obtain a general license or permission for the use of such
proprietary rights by implementers or users of this specification can be
obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary rights
that may cover technology that may be required to implement this
standard. Please address the information to the IETF at ietf-
ipr@ietf.org.
Lazzaro/Wawrzynek [Page 138]
INTERNET-DRAFT 22 June 2004
L. Full Copyright Statement
Copyright (C) The Internet Society (2004). This document is subject to
the rights, licenses and restrictions contained in BCP 78, and except as
set forth therein, the authors retain all their rights.
This document and the information contained herein are provided
on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT
THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR
ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A
PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.
Lazzaro/Wawrzynek [Page 139]
INTERNET-DRAFT 22 June 2004
M. Change Log for <draft-ietf-avt-rtp-midi-format-05.txt>
[Note to RFC Editors: this Appendix, and its Table of Contents listing,
should be removed from the final version of the memo]
Changes from -04.txt address specific requests made by Colin Perkins, as
posted to avt@ietf.org. Each request is listed below, along with a
pointer to the -05.txt change that we believe addresses it.
---
[Colin] The IETF IPR rules have changed since these drafts were
submitted. Could you please update the copyright and IPR boilerplate
according to RFCs 3667 and 3668?
[Reply] The "Status of this Memo" section, the Intellectual Property
Rights Statement (Appendix K) and the Full Copyright Statement (Appendix
L) have been updated to reflect RFC 3667 and RF 3668.
---
[Colin] In Section 2.1, top of page 7, can you check that the parameter
mentioned is "srate" rather than "rate"?
[Reply] The parameter in draft-ietf-mmusic-sdp-new-17.txt is named
"clock rate". All uses of "srate" in the document have been changed to
"clock rate".
---
[Colin] In Section 2.1, consider starting the second paragraph on page 7
"Timestamps of consecutive packets do not..."
[Reply] Done.
---
[Colin] In Section 2.2, 3rd paragraph on page 8, mentions that senders
MAY transmit an RTP stream with missing or out-of-order packets in some
circumstances. Doing this will disrupt RTCP statistics, so unless
there's a pressing need to allow this, I suggest it is disallowed.
[Reply] Suggestion taken, Section 2.2 (last sentence of paragraph 5),
Appendix C.1.1 (final paragraph) modified to reflect the change.
---
[Colin] Section 3 lists the P bit in the header, but it's not defined
Lazzaro/Wawrzynek [Page 140]
INTERNET-DRAFT 22 June 2004
until Section 3.2. It might be clearer if Section 3 mentioned the P
bit, giving a forward pointer to its definition. Please also check the
appendices for similar issues.
[Reply] Done -- see the Section 3 text that follows Figure 2 and
precedes Figure 3. Forward pointers were also added to the text
following Figure A.3.1 in Appendix A.3, the text following Figure A.6.1
in Appendix A.6, the text following Figure A.7.1 in Appendix A.7, and
the text following Figure B.3.1 in Appendix B.3.
---
[Colin] The "Checkpoint Packet Seqnum" is defined on page 19, but there
is no mention of the required byte order. Please specify that this
field is network byte order.
[Reply] Done, see altered text in the paragraphs preceding Figure 9 in
Section 5.
---
[Colin] The SDP examples include spaces between the "a=rtpmap:" and
"a=fmtp:" keywords and their parameters. White space is not permitted
here, and the examples need to be updated to reflect this. Note also
that all MIME parameters should be specified on a single "a=fmtp:" line,
rather than on multiple lines.
[Reply] All examples have been changed to reflect these rules. Also,
please look over the first paragraphs of Appendix D; I re-formulated the
ABNF fmtp syntax to match the rules, and added text that spelled out the
ABNF in plain English.
---
[Colin] The text refers to "fmtp parameters" throughout. The usual term
is "MIME parameters".
[Reply] Fixed.
---
[Colin] Section 8 "Congestion Control" should reference the discussion
in the RTP specification and (for example) the A/V profile. It's
important to specify that, when using UDP, the sender should monitor
packet loss to ensure that it competes approximately fairly with TCP.
[Reply] Done, see the new Section 8.
Lazzaro/Wawrzynek [Page 141]
INTERNET-DRAFT 22 June 2004
---
[Colin] In Section A.1, 2nd paragraph, maybe clarify that the reference
is to a "previous" packet with sequence number C?
[Reply] Done, along with a caveat for the C = I case.
---
[Colin] Appendix C defines a number of MIME parameters, which affect the
operation of the system. It's not clear which parameters are mandatory
to implement, which are optional, and which features are required for
all implementations or optional. Could you consider adding a table to
summarize this? I'm a little concerned that there'll be
interoperability issues due to the large number of parameters and
options, unless the draft is very clear on what is intended.
[Reply] The new Appendix C.6 defines the feature set that an RTP MIDI
client be expected to support, and also gives detailed advice on how to
conduct Offer/Answer negotiations to maximize interoperability. Figure
C.6.1 is a table that summarizes the advice in Appendix C.6.
Hopefully this addition addresses your concerns.
In the process of writing Appendix C.6, several changes were made in
other parts of the document:
o In the main text, Sections 6.1-2, which describe
the default (no MIME parameters) streams, now
discuss the implications of sendrecv, sendonly,
and recvonly RTP MIDI sessions (see the end of
each section).
o The preamble to Appendix C stresses that none of
the parameters are REQUIRED to appear in an RTP
MIDI session description -- all are OPTIONAL.
o In Appendix C preamble, the requirement for a
"common view of the session by all parties" is
defined and explained, particularly its impact
on sendrecv sessions. Several copies of this
requirement were removed from other parts of
Appendix C.
o In C.4, the use of identity relationships to
create "virtual" sendrecv sessions was clarified.
o Normative text was added to parts of Appendix
Lazzaro/Wawrzynek [Page 142]
INTERNET-DRAFT 22 June 2004
C.5, to (1) clarify the use of the "unknown"
value for "render", (2) state when an offer
using "smf_info" MUST be rejected, and state
where in the session description "smf_info",
"smf_inline", "smf_url", and "smf_cid" MUST
appear, and (3) state when an offer using
"chanmask" MUST be rejected, and where in a
session description "chanmask" MUST appear.
o In C.5.1, the multimode parameter now has its
own section (formerly in the preamble of C.5).
o A new section, C.5.2, describes a new MIME
parameter, dirsense. This parameter lets a
session description distinguish between
a telephony-like usage of sendrecv streams
(network musical performance, two peers
conversing with notes) and the classic MIDI 1.0
usage of sendrecv streams (MIDI In and MIDI
Out cable pairs forming a client-server system).
o Lots of small editorial changes, mostly pointers
to the new Appendix C.6.
---
[Colin] Does the ABNF in appendix D validate according to
http://www.apps.ietf.org/abnf.html?
[Reply] Yes, the updated ABNF has been validated.
---
[Colin] The security considerations section should reference the RTP
specification and any applicable RTP profile (e.g. AVP, SRTP).
[Reply] Done, see the updated Appendix G.
---
[Colin] Finally, please read http://www.ietf.org/ID-Checklist.html and
check that all the issues noted are addressed in the drafts.
[Reply] Done. Note that the RFC 3667 Section 5.1 IPR Disclosure
Acknowledgement appears in "Status of this Memo", even though the latest
version of "idnits" claims that the paragraph text doesn't exactly match
the RFC. I did a direct copy-paste from the RFC, the idnits string-
matcher seems to have trouble handling whatever nroff -ms does to the
Lazzaro/Wawrzynek [Page 143]
INTERNET-DRAFT 22 June 2004
text. No other idnits complaints, except for hyphenated words that
happen to be broken across two lines.
---
[Colin] I'd like to see a review of the payload format by a MIDI expert,
before we advance the draft. Understanding the recovery journal
semantics requires a lot of MIDI-specific knowledge, and I'd like
confirmation that makes sense (I don't doubt that it does, but I don't
know enough about MIDI to validate it myself).
[Reply] OK, would you like this to happen before the "we need an
internal working group review before advance to Last Call" posting to
AVT, or were you requesting that one of the replies to that posting
needs to be from a MIDI expert?
Lazzaro/Wawrzynek [Page 144]
