CORE M. Boucadair
Internet-Draft Orange
Intended status: Standards Track J. Shallow
Expires: November 25, 2020 May 24, 2020
Constrained Application Protocol (CoAP) Block-Wise Transfer Options for
Faster Transmission
draft-bosh-core-new-block-01
Abstract
This document specifies new Constrained Application Protocol (CoAP)
Block-Wise transfer options: Block3 and Block4 Options.
These options are similar to the CoAP Block1 and Block2 Options, but
enable faster transmission rates for large amounts of data with less
packet interchanges as well as supporting faster recovery should any
of the blocks get lost in transmission.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on November 25, 2020.
Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
Boucadair & Shallow Expires November 25, 2020 [Page 1]
Internet-Draft New Block-Wise Transfer Options May 2020
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Existing CoAP Block-Wise Transfer Options . . . . . . . . 2
1.2. New CoAP Block-Wise Transfer Options . . . . . . . . . . 3
1.3. New CoAP Response Code . . . . . . . . . . . . . . . . . 4
1.4. Applicability Scope . . . . . . . . . . . . . . . . . . . 4
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. The Block3 and Block4 Options . . . . . . . . . . . . . . . . 5
3.1. Properties of Block3 and Block4 Options . . . . . . . . . 5
3.2. Structure of Block3 and Block4 Options . . . . . . . . . 6
3.3. Using the Block3 Option . . . . . . . . . . . . . . . . . 9
3.4. Using the Block 4 Option . . . . . . . . . . . . . . . . 11
3.5. Working with Observe and Block4 Options . . . . . . . . . 12
3.6. Working with Size1 and Size2 Options . . . . . . . . . . 12
3.7. Streaming Support for the Block3 and Block4 Options . . . 13
3.8. Working with ETag Option . . . . . . . . . . . . . . . . 13
3.9. Use of Block3 and Block4 Options Together . . . . . . . . 13
4. TBA3 (Missing Payloads) Response Code . . . . . . . . . . . . 14
5. Caching Considerations . . . . . . . . . . . . . . . . . . . 14
6. HTTP-Mapping Considerations . . . . . . . . . . . . . . . . . 15
7. Examples of Selective Block Recovery . . . . . . . . . . . . 15
7.1. Block3 Option: Non-Confirmable Example . . . . . . . . . 15
7.2. Block4 Option: Non-Confirmable Example . . . . . . . . . 17
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18
8.1. New CoAP Options . . . . . . . . . . . . . . . . . . . . 18
8.2. New CoAP Response Code . . . . . . . . . . . . . . . . . 19
9. Security Considerations . . . . . . . . . . . . . . . . . . . 19
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 19
11.1. Normative References . . . . . . . . . . . . . . . . . . 19
11.2. Informative References . . . . . . . . . . . . . . . . . 20
Appendix A. Examples with Confirmable Messages . . . . . . . . . 21
A.1. Block3 Option . . . . . . . . . . . . . . . . . . . . . . 21
A.2. Block4 Option . . . . . . . . . . . . . . . . . . . . . . 22
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24
1. Introduction
1.1. Existing CoAP Block-Wise Transfer Options
The Constrained Application Protocol (CoAP) [RFC7252], although
inspired by HTTP, was designed to use UDP instead of TCP. The
message layer of CoAP over UDP includes support for reliable
Boucadair & Shallow Expires November 25, 2020 [Page 2]
Internet-Draft New Block-Wise Transfer Options May 2020
delivery, simple congestion control, and flow control. [RFC7959]
introduced the CoAP Block1 and Block2 Options to handle data records
that cannot fit in a single IP packet, so not having to rely on IP
fragmentation.
The CoAP Block1 and Block2 Options work well in environments where
there are no or minimal packet losses. These options operate
synchronously where each block has to be requested and can only ask
for (or send) the next block when the request for the previous block
has completed. Packet, and hence block transmission rate, is
controlled by Round Trip Times (RTTs).
There is a requirement for these blocks of data to be transmitted
under network conditions where there may be transient packet loss.
An example is when a network is subject to a Distributed Denial of
Service (DDoS) attack and there is a need for DDoS mitigation agents
relying upon CoAP to communicate with each other (e.g.,
[I-D.ietf-dots-telemetry]). As a reminder, [RFC7959] recommends use
of Confirmable (CON) responses to handle potential packet loss; which
does not work with a flooded pipe DDoS situation.
1.2. New CoAP Block-Wise Transfer Options
This document introduces the CoAP Block3 and Block4 Options. These
options are similar in operation to the CoAP Block1 and Block2
Options respectively, but enable faster transmissions of sets of
blocks of data with less packet interchanges as well as supporting
faster recovery should any of the Blocks get lost in transmission.
Using Non-confirmable (NON) messages, the faster transmissions occur
as all the Blocks can be transmitted serially (as are IP fragmented
packets) without having to wait for an acknowledgement or next
request from the remote CoAP peer. Recovery of missing Blocks is
faster in that multiple missing Blocks can be requested in a single
CoAP packet.
Note that the same performance benefits can be applied to Confirmable
messages if the value of NSTART is increased from 1 (Section 4.7 of
[RFC7252]). Some sample examples with Confirmable messages are
provided in Appendix A.
A CoAP endpoint can acknowledge all or a subset of the blocks.
Concretely, the receiving CoAP endpoint informs the CoAP endpoint
sender either successful receipt or reports on all blocks in the body
that have been not yet been received. The CoAP endpoint sender will
then retransmit only the blocks that have been lost in transmission.
Boucadair & Shallow Expires November 25, 2020 [Page 3]
Internet-Draft New Block-Wise Transfer Options May 2020
Alternative CoAP options may be considered to cover the extra
functionality of the Block3 and Block4 Options using Block1 and
Block2 Options, but then the semantic and usage of the Block1 and
Block2 Options would have to change.
The deviation from Block1 and Block2 Options are specified in
Section 3. Pointers to appropriate [RFC7959] sections are provided.
The specification refers to the base CoAP methods defined in
Section 5.8 of [RFC7252] and the new CoAP methods, FETCH, PATCH, and
iPATCH introdcued in [RFC8132].
1.3. New CoAP Response Code
This document defines a new CoAP Response Code (Section 5.9 of
[RFC7252]), called TBA3 (Missing payloads), to report on payloads
using the Block3 Option that are not received by the server.
See Section 4 for more details.
1.4. Applicability Scope
The mechanism specified in the document includes guards to prevent a
CoAP agent from overloading the network by adopting an aggressive
sending rate. These guards MUST be followed in addition to the
existing CoAP congestion control as specified in Section 4.7 of
[RFC7252].
This mechanism primally targets applications such as DDoS Open Threat
Signaling (DOTS) that can't use Confirmable (CON) responses to handle
potential packet loss and that support application-specific
mechanisms to assess whether the remote peer is able to handle the
messages sent by a CoAP endpoint (e.g., DOTS heartbeats in
Section 4.7 of [I-D.ietf-dots-signal-channel]).
2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119][RFC8174] when, and only when, they appear in all
capitals, as shown here.
Readers should be familiar with the terms and concepts defined in
[RFC7252].
The terms "payload" and "body" are defined in [RFC7959]. The term
"payload" is thus used for the content of a single CoAP message
Boucadair & Shallow Expires November 25, 2020 [Page 4]
Internet-Draft New Block-Wise Transfer Options May 2020
(i.e., a single block being transferred), while the term "body" is
used for the entire resource representation that is being transferred
in a block-wise fashion.
3. The Block3 and Block4 Options
3.1. Properties of Block3 and Block4 Options
The properties of Block3 and Block4 Options are shown in Table 1.
The formatting of this table follows the one used in Table 4 of
[RFC7252] (Section 5.10). The C, U, N, and R columns indicate the
properties Critical, Unsafe, NoCacheKey, and Repeatable defined in
Section 5.4 of [RFC7252]. Only C column is marked for the Block3
Option. Only C and R columns are marked for the Block4 Option.
+--------+---+---+---+---+-----------+--------+--------+---------+
| Number | C | U | N | R | Name | Format | Length | Default |
+========+===+===+===+===+===========+========+========+=========+
| TBA1 | x | | | | Block3 | uint | 0-7 | (none) |
| TBA2 | x | | | x | Block4 | uint | 0-7 | (none) |
+--------+---+---+---+---+-----------+--------+--------+---------+
Table 1: CoAP Block3 and Block4 Option Properties
The Block3 and Block4 Options can be present in both the request and
response messages. The Block3 Option pertains to the request payload
and the Block4 Option pertains to the response payload. The Content-
Format Option applies to the body, not to the payload (i.e., it must
be the same for all payloads of the same body).
Block3 is useful with the payload-bearing POST, PUT, PATCH, and
iPATCH requests and their responses (2.01 and 2.04). Block4 Option
is useful with GET, POST, PUT, and FETCH requests and their payload-
bearing responses (2.01, 2.03, 2.04, and 2.05) (Section 5.5 of
[RFC7252]).
To indicate support for Block4 responses, the CoAP client MUST
include the Block4 Option in a GET or FETCH request so that the
server knows that the client supports this Block4 functionality.
Otherwise, the server would use the Block2 Option (if supported) to
send back a message body that is larger than what can fit into a
single IP packet [RFC7959].
If Block3 Option is present in a request or Block4 Option in a
response (i.e., in that message to the payload of which it pertains),
it indicates a block-wise transfer and describes how this specific
block-wise payload forms part of the entire body being transferred
(referred to as "descriptive usage"). If it is present in the
Boucadair & Shallow Expires November 25, 2020 [Page 5]
Internet-Draft New Block-Wise Transfer Options May 2020
opposite direction, it provides additional control on how that
payload will be formed or was processed (referred to as "control
usage").
Implementation of Block3 (or Block4) Option is intended to be
optional. However, when it is present in a CoAP message, it MUST be
processed (or the message rejected). Therefore, Block3 and Block4
Options are identified as Critical options.
The Block3 and Block4 Options are safe to forward. That is, a CoAP
proxy that does not understand the Block3 (or Block4) Option should
forward the option on.
The Block4 Option is repeatable when requesting re-transmission of
missing Blocks, but not otherwise. Except that case, any request
carrying multiple Block3 (or Block4) Options MUST be handled
following the procedure specified in Section 5.4.5 of [RFC7252].
PROBING_RATE parameter in CoAP indicates the average data rate that
must not be exceeded by a CoAP endpoint in sending to a peer endpoint
that does not respond. The body of blocks will be subjected to
PROBING_RATE (Section 4.7 of [RFC7252]).
The Block3 and Block4 Options are of class E and U for OSCORE
[RFC8613].
3.2. Structure of Block3 and Block4 Options
The structure of Block3 and Block4 Options follows the structure
defined in Section 2.2 of [RFC7959] with two additional fields:
Block Identifier (BID): This number associates all the Blocks
(individual payloads) that make up the body that is being
transferred.
Streaming bit (S bit): This bit is used to indicate whether the
client (or server) is continuously sending blocks of data.
As such, five items of information may need to be transferred in a
Block3 or Block4 Option:
o the size of the block (size exponent (SZX)),
o whether more blocks are following (More (M)),
o the relative number of the block (Block Number (NUM)) within a
sequence of blocks with the given size,
Boucadair & Shallow Expires November 25, 2020 [Page 6]
Internet-Draft New Block-Wise Transfer Options May 2020
o whether this is about streaming data (Streaming (S)), and
o the block identifier number (BID) that is common to the sequence
of blocks with the same given size. The BID is different for each
body or set of sequence of blocks.
The value of the Block3 or Block4 Option is a variable-size (0 to 7
byte) unsigned integer (uint) (Section 3.2 of [RFC7252]). This
integer value encodes the aforementioned five fields as shown in
Figure 1. Note that, due to the CoAP uint-encoding rules, when all
of NUM, M, SZX, S, and BID happen to be zero, a zero-byte integer
will be sent.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| BID |S| NUM |M| SZX |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| BID |S| NUM
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
NUM |M| SZX |
+-+-+-+-+-+-+-+-+
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| BID |S| NUM
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
NUM |M| SZX |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| BID |S|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| NUM |M| SZX |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1: Structure of Block3 and Block4 Options
The twenty-fourth least significant bit is the S bit ("val &
0x1000000").
Boucadair & Shallow Expires November 25, 2020 [Page 7]
Internet-Draft New Block-Wise Transfer Options May 2020
The option value shifted right by 25 ("(val >> 25)")(the BID field)
is the Block identifier that identifies which sequence of blocks this
particular block is in.
The current transfer is about the "size" bytes starting at byte "NUM
<< (SZX + 4)".
Within the option value of a Block3 or Block4 Option, the meaning of
the option fields is defined below. Note that the block size (SZX,
size exponent), the M bit, and the NUM fields are defined in
Section 2.2 of [RFC7959], but are provided below for the reader's
convenience:
BID: This block identifier is the same for all of the blocks in the
body of data that is being transferred. It is used when a
particular block needs to be re-transmitted.
This value MUST be different for distinct sets of blocks of data
and SHOULD be incremented whenever a new body of data is being
transmitted for a CoAP session between peers. The initial BID
value SHOULD be randomly generated and MUST NOT have a value of 0.
When a Block4 Option is used to request a complete transfer of a
body, BID MUST be set to 0. If the request is for one or more
missing blocks, then the BID is set to the value of the BID of the
partially received body.
S: Streaming bit. This bit is set only if data is being streamed
as opposed to having a finite body size. Usually this bit is
unset. See Section 3.7 for further information on streaming.
NUM: Block Number, indicating the block number being requested or
provided. Block number '0' indicates the first block of a body
(i.e., starting with the first byte of the body).
M: More bit (i.e., "not last block"). For descriptive usage, this
bit, if unset, indicates that the payload in this message is the
last block in the body; when set, it indicates that there are one
or more additional blocks available.
When a Block4 Option is used in a request to retrieve a specific
block number ("control usage"), the M bit MUST be sent as zero and
ignored on reception. In a Block3 Option in a response, the M bit
is used to indicate atomicity, similar to Block1 Option
([RFC7959]).
SZX: Block Size. The block size is represented as a three-bit
unsigned integer indicating the size of a block to the power of
Boucadair & Shallow Expires November 25, 2020 [Page 8]
Internet-Draft New Block-Wise Transfer Options May 2020
two. Thus, block size = 2**(SZX + 4). The allowed values of SZX
are 0 to 6, i.e., the minimum block size is 2**(0+4) = 16 and the
maximum is 2**(6+4) = 1024. The value 7 for SZX (which would
indicate a block size of 2048) is used as a BERT Option in
Section 6 of [RFC8323].
There is no default value for the Block3 and Block4 Options. Absence
of one of these options is equivalent to an option value of 0 with
respect to the value of NUM, M, S, and BID that could be given in the
option, i.e., it indicates that the current block is the first and
only block of the transfer (block number is set to 0, M and S bits
are unset, and BID is set to 0). However, in contrast to the
explicit value 0, which would indicate an SZX of 0, and thus a size
value of 16 bytes, there is no specific explicit size implied by the
absence of the option -- the size is left unspecified. (As for any
uint, the explicit value 0 is efficiently indicated by a zero-length
option; this, therefore, is different in semantics from the absence
of the option).
3.3. Using the Block3 Option
The Block3 Options is used when the client wants to send a large
amount of data to the server using the POST, PUT, PATCH, or iPATCH
methods where the data and headers do not fit into a single packet.
The client sends all the individual payloads of the body using the
same BID, only expecting a response when all the payloads have been
sent. It is RECOMMENDED that after transmission of every set of
MAX_PAYLOADS payloads of a single body, a delay is introduced of
ACK_TIMEOUT (Section 4.8.2 of [RFC7252]) before the next set of
payload transmissions to manage potential congestion issues.
MAX_PAYLOADS should be configurable with a default value of 10.
Note: The default value is chosen for reasons similar to those
discussed in Section 5 of [RFC6928].
For NON transmissions, it is permissible, but not required, to send
the penultimate payload of a MAX_PAYLOADS set as a Confirmable
packet. If a Confirmable packet is used, then the client MUST wait
for the ACK to be returned before sending the next set of payloads,
which can be in time terms less than the ACK_TIMEOUT delay.
Also, for NON transmissions, it is permissible, but not required, to
send a Confirmable packet for the final payload of a body (that is, M
bit unset). If a Confirmable packet is used, then the client MUST
wait for the 2.01 (Created) or 2.04 (Changed) Response Codes to be
returned for successful transmission, or TBA3 to then resend the
missing blocks (if any).
Boucadair & Shallow Expires November 25, 2020 [Page 9]
Internet-Draft New Block-Wise Transfer Options May 2020
With NON transmission, the server acknowledges receipt of all of the
payloads that make up the body or respond at any time during the
receipt of the payloads to acknowledge that some of the payloads have
arrived, but others are missing. It is RECOMMENDED that, unless
there are receipt issues, the server only responds when the final
payload (i.e., M bit unset) is received.
For Confirmable transmission, the server MUST continue to acknowledge
each packet. NSTART will also need to be increased from the default
(1) to get faster transmission rates.
Non-empty Tokens MUST be included. Each individual payload of the
body MUST have a different Token.
A 2.01 (Created) or 2.04 (Changed) Response Code indicates successful
receipt of the entire body. The 2.31 (Continue) Response Code MUST
NOT be used.
A 4.02 (Bad Option) Response Code MUST be returned if the server does
not support the Block3 Option.
Use of 4.08 (Request Entity Incomplete) Response Code is discouraged
when using Block3 Option because packets may arrive out of sequence;
TBA3 (Missing Payloads) Response Code (Section 4) SHOULD be used
instead. However, 4.08 (Request Entity Incomplete) Response Code is
still valid to reject a Content-Format mismatch.
A 4.13 (Request Entity Too Large) Response Code can be returned under
similar conditions to those discussed in Section 2.9.3 of [RFC7959].
A TBA3 (Missing Payloads) Response Code indicates that some of the
payloads are missing and need to be resent. The client then re-
transmits the missing payloads using Block3 to specify the BID, M
bit, block number, SZX, and M bit unset. As discussed above, the
sending of the list of missing blocks is subject to MAX_PAYLOAD.
If the server has not received the final payload (i.e., a block with
M bit unset), but one or other payloads have been received, it SHOULD
wait for up to MAX_TRANSMIT_SPAN (Section 4.8.2 of [RFC7252]) before
sending the TBA3 (Missing Payloads) Response Code. However, this
timer MAY be reduced to two times ACK_TIMEOUT before sending a TBA3
(Missing Payloads) Response Code to cover the situation where
MAX_PAYLOADS has been triggered by the client causing a break in
transmission.
In all cases, multiple TBA3 (Missing Payloads) Response Codes are
traffic limited by PROBING_RATE.
Boucadair & Shallow Expires November 25, 2020 [Page 10]
Internet-Draft New Block-Wise Transfer Options May 2020
If the client transmits a new body of data with a new BID to the same
server, the server MUST remove any partially received body held for a
previous BID for that resource.
If the server receives a duplicate block with the same BID, it SHOULD
silently ignore the packet.
A server SHOULD only maintain a partial body (missing payloads) for
up to EXCHANGE_LIFETIME (Section 4.8.2 of [RFC7252]).
3.4. Using the Block 4 Option
Support for the receipt of Block4 Option by the client is indicated
by using the Block4 Option in the GET or FETCH request. If the
Block4 Option is not included in the request, the server MUST NOT
send data using the Block4 Option.
The initiating request that contains Block4 Option MUST set both BID
and 'M' to 0. All other Block4 Options in a request MUST have the M
bit unset.
The payloads sent back from the server as a response MUST all have
the same BID which MUST NOT be 0. The sending of the payloads is
subject to MAX_PAYLOADS. If MAX_PAYLOADS is exceeded, the server
MUST introduce an ACK_TIMEOUT delay before transmitting the next set
of payloads.
For NON transmission, it is permissible, but not required, to send
the penultimate payload of a MAX_PAYLOADS set as a Confirmable
packet. If a Confirmable packet is used, then the server MUST wait
for the ACK to be received before sending the next set of payloads,
which can be in time terms less than the ACK_TIMEOUT delay.
Also, for NON transmission, it is permissible, but not required, to
send a Confirmable packet for the final payload of a body (i.e., M
bit unset). If a Confirmable packet is used, the server MUST wait
for the ACK to be returned for successful transmission.
If the client detects that some of the payloads are missing, the
missing payloads are requested by issuing a new GET or FETCH request
that contains one or more Block4 Options that define the missing
blocks. A new Token MUST be used for this request. The rate of
requests for missing blocks is subject to PROBING_RATE.
The client may elect to request the missing blocks or just ignore the
partial body.
Boucadair & Shallow Expires November 25, 2020 [Page 11]
Internet-Draft New Block-Wise Transfer Options May 2020
All the payload responses to a specific GET or FETCH request MUST
have the same Token as in the request.
With NON transmission, the client only needs to indicate that some of
the payloads are missing by issuing a GET or FETCH request for the
missing blocks.
For Confirmable transmission, the client SHOULD continue to
acknowledge each packet as well as issuing a separate GET or FETCH
for the missing blocks. NSTART will also need to be increased from
the default (1) to get faster transmission rates.
If the server transmits a new body of data with a new BID to the same
client with the same Token, the client MUST remove any partially
received body held for a previous BID for that Token.
If the client receives a duplicate block with the same BID, it SHOULD
silently ignore the packet.
A client SHOULD only maintain a partial body (missing payloads) for
up to EXCHANGE_LIFETIME (Section 4.8.2 of [RFC7252]) or as defined by
the Max-Age Option whichever is the less.
3.5. Working with Observe and Block4 Options
As the blocks of the body are sent without waiting for
acknowledgement of the individual blocks, the Observe value [RFC7641]
MUST be the same for all the blocks of the same body.
Likewise, the Tokens MUST all have the same value for all the blocks
of the same body. This is so that if any of the blocks gets lost
during transmission (including the first one), the receiving CoAP
endpoint can take the appropriate decisions as to how to continue
(implementation specific).
If the client requests missing blocks, the client MUST use a
different Token; all repeated missing blocks for that new request
MUST use the new Token.
3.6. Working with Size1 and Size2 Options
Section 4 of [RFC7959] defines two CoAP options: Size1 for indicating
the size of the representation transferred in requests and Size2 for
indicating the size of the representation transferred in responses.
It is RECOMMENDED that the Size1 Option is used with the Block3
Option and that the Size2 Option is used with the Block4 Option.
Boucadair & Shallow Expires November 25, 2020 [Page 12]
Internet-Draft New Block-Wise Transfer Options May 2020
If Size1 or Size2 Options are used, they MUST be used in all payloads
of the body and MUST have the same value.
3.7. Streaming Support for the Block3 and Block4 Options
If the Size1 Option is not specified, there is support for streaming
data to the server by always setting the S and M bits in the payload.
The server can still indicate missing blocks with the TBA3 (Missing
Payloads) Response Code but is NOT RECOMMENDED. By unsettling the M
bit, the client can indicate that this is the last payload and hence
the end of the streaming of data.
Likewise, if the Size2 Option is not specified, there is support for
streaming data to the client by always setting the S and M bits in
the payload. The client can request missing blocks but is NOT
RECOMMENDED. The client needs to indicate to the server that it can
handle streamed data by setting the S bit in the initial request. If
the client no longer wants to receive data, it can send a GET request
to the same resource with the same BID, but both M and S bits unset.
When the Block Number (NUM) reaches the maximum(2**20 -1), it simply
wraps around to 0 and then continues to be incremented for each
payload.
As a reminder, the use of MAX_PAYLOADS has still to be followed so
that the network is not overloaded with too much traffic.
3.8. Working with ETag Option
The ETag Option defined in Section 5.10.6 of [RFC7252] applies to the
whole representation of the resource, and thus to the body of the
response.
If ETag Option is used, it MUST be used in all payloads of the body
and MUST have the same value.
If the ETag changes, but the BID does not, then the payload MUST be
ignored as it is invalid and cannot be included in the partial body.
If ETag is used in the request, then Section 5.10.6.2 of [RFC7252]
still applies. As such, 2.03 (Valid) Response Code can be returned.
3.9. Use of Block3 and Block4 Options Together
The behavior is similar to the one defined in Section 3.3 of
[RFC7959] with Block3 substituted for Block1 and Block4 for Block2.
Boucadair & Shallow Expires November 25, 2020 [Page 13]
Internet-Draft New Block-Wise Transfer Options May 2020
4. TBA3 (Missing Payloads) Response Code
TBA3 (Missing Payloads) Response Code is a new client error status
code (Section 5.9.2 of [RFC7252]) used to indicate that the server
has not received the blocks of the request body that it needs to
proceed.
Likely causes are the client has not sent all blocks, some blocks
were dropped during transmission, or the client has sent them long
enough ago that the server has already discarded them.
The body of the TBA3 (Missing Payloads) Response Code contains a
count of the missing block numbers followed by set of 1 or more
missing block numbers encoded in CBOR format (0 unsigned) for
compactness. If the size of the TBA3 (Missing Payloads) response
packet is larger than that defined by Section 4.6 [RFC7252], then the
number of missing blocks MUST be limited so that the response can fit
into a single packet. If necessary, multiple TBA3 (Missing Payloads)
Response Codes can be sent back; each covering a unique set of
missing blocks.
The Content-Format Option (Section 5.10.3 of [RFC7252]) MUST be used
in the TBA3 (Missing Payloads) Response Code. It MUST be set to
"application/cbor".
5. Caching Considerations
The Block3 and Block4 Options are part of the cache key. As such, a
CoAP proxy that does not understand the Block3 and Block4 Options
must follow the recommendations in Section 5.7.1 of [RFC7252] for
caching.
If the S bit is set, the data SHOULD NOT be cached.
This specification does not require a proxy to obtain the complete
representation before it serves parts of it to the client.
Otherwise, the considerations discussed in Section 2.10 of [RFC7959]
apply for the Block3 and Block4 Options (with Block3 substituted for
Block1 and Block4 substituted for Block2) for proxies that support
Block3 and Block4 Options.
A proxy that supports Block4 Option MUST be prepared to receive a GET
message indicating one or more missing blocks. The proxy can serve
from its cache missing blocks that are available in its cache in a
set as a server would send all the Block4s. If one or more requested
blocks are not available in the cache, the proxy SHOULD update the
GET request with the blocks that it can serve from the cache, and
then forward on the request, after modifying it to remove the missing
Boucadair & Shallow Expires November 25, 2020 [Page 14]
Internet-Draft New Block-Wise Transfer Options May 2020
block references it has in cache, to the next hop. Alternatively,
the original request unmodified (from the missing block perspective)
MAY be forwarded on to the server. All the responses are then passed
back to the client with the cache getting updated.
How long a CoAP endpoint (or proxy) keeps the body in its cache is
implementation specific (e.g., it may be based on Max-Age).
6. HTTP-Mapping Considerations
As a reminder, the basic normative requirements on HTTP/CoAP mappings
are defined in Section 10 of [RFC7252]. The implementation
guidelines for HTTP/CoAP mappings are elaborated in [RFC8075].
The rules defined in Section 5 of [RFC7959] are to be followed.
7. Examples of Selective Block Recovery
This section provides some sample flows to illustrate the use of
Block3 and Block4 Options. These conventions are used in the
following subsections:
T: Token value
O: Observe Option value
M: Message ID
B3: Block3 Option values BID/S/NUM/More/SZX
B4: Block3 Option values BID/S/NUM/More/SZX
\: Trimming long lines
[[]]: Comments
-->X: Message loss
X<--: Message loss
7.1. Block3 Option: Non-Confirmable Example
Figure 2 depicts an example of a NON PUT request conveying Block3
Option. All the blocks are received by the server.
Boucadair & Shallow Expires November 25, 2020 [Page 15]
Internet-Draft New Block-Wise Transfer Options May 2020
CoAP CoAP
Client Server
| |
+--------->| NON PUT /path M:0x01 T:0xf0 B3:10/0/0/1/1024
+--------->| NON PUT /path M:0x02 T:0xf1 B3:10/0/1/1/1024
+--------->| NON PUT /path M:0x03 T:0xf2 B3:10/0/2/1/1024
+--------->| NON PUT /path M:0x04 T:0xf3 B3:10/0/3/0/1024
|<---------+ NON 2.04 M:0xf1 T:0xf3
...
Figure 2: Example of NON Request with Block3 Option (Without Loss)
Consider now a scenario where a new body of data is to be sent by the
client, but some blocks are dropped in transmission as illustrated in
Figure 3.
CoAP CoAP
Client Server
| |
+--------->| NON PUT /path M:0x05 T:0xe0 B3:11/0/0/1/1024
+--->X | NON PUT /path M:0x06 T:0xe1 B3:11/0/1/1/1024
+--->X | NON PUT /path M:0x07 T:0xe2 B3:11/0/2/1/1024
+--------->| NON PUT /path M:0x08 T:0xe3 B3:11/0/3/0/1024
| |
...
Figure 3: Example of NON Request with Block3 Option (With Loss)
The server realizes that some blocks are missing and asks for the
missing ones in one go (Figure 4). It does so by indicating which
blocks have been received in the data portion of the response.
CoAP CoAP
Client Server
| |
...
|<---------+ NON TBA3 M:0xf2 T:0xe3 [Missing 1,2]
+--------->| NON PUT /path M:0x09 T:0xe4 B3:11/0/1/1/1024
+--->X | NON PUT /path M:0x0a T:0xe5 B3:11/0/2/1/1024
| |
|<---------+ NON TBA3 M:0xf3 T:0xe4 [Missing 2]
+--------->| NON PUT /path M:0x0b T:0xe6 B3:11/0/2/1/1024
|<---------+ NON 2.04 M:0xf4 T:0xe6
| |
...
Figure 4: Example of NON Request with Block3 Option (Blocks Recovery)
Boucadair & Shallow Expires November 25, 2020 [Page 16]
Internet-Draft New Block-Wise Transfer Options May 2020
Under high levels of traffic loss, the client can elect not to retry
sending missing blocks of data. This decision is implementation
specific.
7.2. Block4 Option: Non-Confirmable Example
Figure 5 illustrates the example of Block4 Option. The client sends
a NON GET carrying an Observe and a Block4 Options. The Block4
Option indicates a size hint (1024 bytes). This request is replied
by the serer using four (4) blocks that are transmitted to the client
without any loss. Each of these blocks carries a Block4 Option. The
same process is repeated when an Observe is triggered, but no loss is
experienced by any of the notification blocks.
CoAP CoAP
Client Server
| |
+--------->| NON GET /path M:0x01 T:0xf0 O:0 B4:0/0/0/0/1024
|<---------+ NON 2.05 M:0xf1 T:0xf0 O:1234 B4:21/0/0/1/1024
|<---------+ NON 2.05 M:0xf2 T:0xf0 O:1234 B4:21/0/1/1/1024
|<---------+ NON 2.05 M:0xf3 T:0xf0 O:1234 B4:21/0/2/1/1024
|<---------+ NON 2.05 M:0xf4 T:0xf0 O:1234 B4:21/0/3/0/1024
...
[[Observe triggered]]
|<---------+ NON 2.05 M:0xf5 T:0xf0 O:1235 B4:22/0/0/1/1024
|<---------+ NON 2.05 M:0xf6 T:0xf0 O:1235 B4:22/0/1/1/1024
|<---------+ NON 2.05 M:0xf7 T:0xf0 O:1235 B4:22/0/2/1/1024
|<---------+ NON 2.05 M:0xf8 T:0xf0 O:1235 B4:22/0/3/0/1024
...
Figure 5: Example of NON Notifications with Block4 Option (Without
Loss)
Figure 6 shows the example of an Observe that is triggered but for
which some notification blocks are lost. The client detects the
missing blocks and request their retransmission. It does so by
indicating the blocks that were successfully received.
Boucadair & Shallow Expires November 25, 2020 [Page 17]
Internet-Draft New Block-Wise Transfer Options May 2020
CoAP CoAP
Client Server
| |
...
[[Observe triggered]]
|<---------+ NON 2.05 M:0xf9 T:0xf0 O:1236 B4:23/0/0/1/1024
| X<---+ NON 2.05 M:0xfa T:0xf0 O:1236 B4:23/0/1/1/1024
| X<---+ NON 2.05 M:0xfb T:0xf0 O:1236 B4:23/0/2/1/1024
|<---------+ NON 2.05 M:0xfc T:0xf0 O:1236 B4:23/0/3/0/1024
| |
[[Client realises blocks are missing and asks for the missing
ones in one go]]
+--------->| NON GET /path M:0x02 T:0xf1 B4:23/0/1/0/1024\
| | B4:23/0/2/0/1024
| X<---+ NON 2.05 M:0xfd T:0xf1 B4:23/0/1/1/1024
|<---------+ NON 2.05 M:0xfe T:0xf1 B4:23/0/2/1/1024
| |
[[Get final missing block]]
+--------->| NON GET /path M:0x03 T:0xf2 B4:23/0/1/0/1024
|<---------+ NON 2.05 M:0xff T:0xf2 B4:23/0/1/1/1024
...
Figure 6: Example of NON Notifications with Block4 Option (Blocks
Recovery)
Under high levels of traffic loss, the client can elect not to retry
getting missing blocks of data. This decision is implementation
specific.
8. IANA Considerations
8.1. New CoAP Options
IANA is requested to add the following entries to the "CoAP Option
Numbers" sub-registry available at https://www.iana.org/assignments/
core-parameters/core-parameters.xhtml#option-numbers:
+--------+------------------+-----------+
| Number | Name | Reference |
+========+==================+===========+
| TBA1 | Block3 | [RFCXXXX] |
| TBA2 | Block4 | [RFCXXXX] |
+--------+------------------+-----------+
Table 2: CoAP Block3 and Block4 Option Numbers
This document suggests 21 (TBA1) and 25 (TBA2) as a values to be
assigned for the new option numbers.
Boucadair & Shallow Expires November 25, 2020 [Page 18]
Internet-Draft New Block-Wise Transfer Options May 2020
8.2. New CoAP Response Code
IANA is requested to add the following entry to the "CoAP Response
Codes" sub-registry available at https://www.iana.org/assignments/
core-parameters/core-parameters.xhtml#response-codes:
+------+------------------+-----------+
| Code | Description | Reference |
+======+==================+===========+
| TBA3 | Missing Payloads | [RFCXXXX] |
+------+------------------+-----------+
Table 3: New CoAP Response Code
This document suggests 4.18 (TBA3) as a value to be assigned for the
new Response Code.
9. Security Considerations
Security considerations discussed in Section 9 of [RFC7959] should be
taken into account.
[[discuss if any security issues related to the incremental BID
values. Lifetime of a BID (pointer to RFC8200)]]
10. Acknowledgements
Thanks to Achim Kraus, Christian Amsuess, Carsten Bormann, and Jim
Schaad for the comments on the mailing list.
Some text from [RFC7959] is reused for readers convenience.
11. References
11.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained
Application Protocol (CoAP)", RFC 7252,
DOI 10.17487/RFC7252, June 2014,
<https://www.rfc-editor.org/info/rfc7252>.
Boucadair & Shallow Expires November 25, 2020 [Page 19]
Internet-Draft New Block-Wise Transfer Options May 2020
[RFC7641] Hartke, K., "Observing Resources in the Constrained
Application Protocol (CoAP)", RFC 7641,
DOI 10.17487/RFC7641, September 2015,
<https://www.rfc-editor.org/info/rfc7641>.
[RFC7959] Bormann, C. and Z. Shelby, Ed., "Block-Wise Transfers in
the Constrained Application Protocol (CoAP)", RFC 7959,
DOI 10.17487/RFC7959, August 2016,
<https://www.rfc-editor.org/info/rfc7959>.
[RFC8075] Castellani, A., Loreto, S., Rahman, A., Fossati, T., and
E. Dijk, "Guidelines for Mapping Implementations: HTTP to
the Constrained Application Protocol (CoAP)", RFC 8075,
DOI 10.17487/RFC8075, February 2017,
<https://www.rfc-editor.org/info/rfc8075>.
[RFC8132] van der Stok, P., Bormann, C., and A. Sehgal, "PATCH and
FETCH Methods for the Constrained Application Protocol
(CoAP)", RFC 8132, DOI 10.17487/RFC8132, April 2017,
<https://www.rfc-editor.org/info/rfc8132>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8323] Bormann, C., Lemay, S., Tschofenig, H., Hartke, K.,
Silverajan, B., and B. Raymor, Ed., "CoAP (Constrained
Application Protocol) over TCP, TLS, and WebSockets",
RFC 8323, DOI 10.17487/RFC8323, February 2018,
<https://www.rfc-editor.org/info/rfc8323>.
[RFC8613] Selander, G., Mattsson, J., Palombini, F., and L. Seitz,
"Object Security for Constrained RESTful Environments
(OSCORE)", RFC 8613, DOI 10.17487/RFC8613, July 2019,
<https://www.rfc-editor.org/info/rfc8613>.
11.2. Informative References
[I-D.ietf-dots-signal-channel]
Reddy.K, T., Boucadair, M., Patil, P., Mortensen, A., and
N. Teague, "Distributed Denial-of-Service Open Threat
Signaling (DOTS) Signal Channel Specification", draft-
ietf-dots-signal-channel-41 (work in progress), January
2020.
Boucadair & Shallow Expires November 25, 2020 [Page 20]
Internet-Draft New Block-Wise Transfer Options May 2020
[I-D.ietf-dots-telemetry]
Boucadair, M., Reddy.K, T., Doron, E., chenmeiling, c.,
and J. Shallow, "Distributed Denial-of-Service Open Threat
Signaling (DOTS) Telemetry", draft-ietf-dots-telemetry-08
(work in progress), May 2020.
[RFC6928] Chu, J., Dukkipati, N., Cheng, Y., and M. Mathis,
"Increasing TCP's Initial Window", RFC 6928,
DOI 10.17487/RFC6928, April 2013,
<https://www.rfc-editor.org/info/rfc6928>.
Appendix A. Examples with Confirmable Messages
These examples assume NSTART has been increased to at least 4.
A.1. Block3 Option
Let's now consider the use Block3 Option with a CON request as shown
in Figure 7. All the blocks are acknowledged (ACK).
CoAP CoAP
Client Server
| |
+--------->| CON PUT /path M:0x01 T:0xf0 B3:10/0/0/1/1024
+--------->| CON PUT /path M:0x02 T:0xf1 B3:10/0/1/1/1024
+--------->| CON PUT /path M:0x03 T:0xf2 B3:10/0/2/1/1024
+--------->| CON PUT /path M:0x04 T:0xf3 B3:10/0/3/0/1024
|<---------+ ACK 0.00 M:0x01
|<---------+ ACK 0.00 M:0x02
|<---------+ ACK 0.00 M:0x03
|<---------+ ACK 0.00 M:0x04
Figure 7: Example of CON Request with Block3 Option (Without Loss)
Now, suppose that a new body of data is to sent but with some blocks
dropped in transmission as illustrated in Figure 8. The client will
retry sending blocks for which no ACK was received.
Boucadair & Shallow Expires November 25, 2020 [Page 21]
Internet-Draft New Block-Wise Transfer Options May 2020
CoAP CoAP
Client Server
| |
+--------->| CON PUT /path M:0x05 T:0xf4 B3:11/0/0/1/1024
+--->X | CON PUT /path M:0x06 T:0xf5 B3:11/0/1/1/1024
+--->X | CON PUT /path M:0x07 T:0xf6 B3:11/0/2/1/1024
+--------->| CON PUT /path M:0x08 T:0xf7 B3:11/0/3/1/1024
|<---------+ ACK 0.00 M:0x05
|<---------+ ACK 0.00 M:0x08
| |
[[The client retries sending packets not acknowledged]]
+--------->| CON PUT /path M:0x06 T:0xf5 B3:11/0/1/1/1024
+--->X | CON PUT /path M:0x07 T:0xf6 B3:11/0/2/1/1024
|<---------+ ACK 0.00 M:0x06
| |
[[The client retransmits messages not acknowledged
(exponential backoff)]]
+--->? | CON PUT /path M:0x07 T:0xf6 B3:11/0/2/1/1024
| |
[[Either transmission failure (acknowledge retry timeout)
or successfully transmitted.]]
Figure 8: Example of CON Request with Block3 Option (Blocks Recovery)
It is implementation dependent as to whether a CoAP session is
terminated following acknowledge retry timeout, or whether the CoAP
session continues to be used under such adverse traffic conditions.
If there is likely to be the possibility of network transient losses,
then the use of Non-confirmable traffic should be considered.
A.2. Block4 Option
An example of the use of Block4 Option with Confirmable messages is
shown in Figure 9.
Boucadair & Shallow Expires November 25, 2020 [Page 22]
Internet-Draft New Block-Wise Transfer Options May 2020
Client Server
| |
+--------->| CON GET /path M:0x01 T:0xf0 O:0 B4:0/0/0/0/1024
|<---------+ ACK 2.05 M:0x01 T:0xf0 O:1234 B4:21/0/0/1/1024
|<---------+ ACK 2.05 M:0xe1 T:0xf0 O:1234 B4:21/0/1/1/1024
|<---------+ ACK 2.05 M:0xe2 T:0xf0 O:1234 B4:21/0/2/1/1024
|<---------+ ACK 2.05 M:0xe3 T:0xf0 O:1234 B4:21/0/3/0/1024
...
[[Observe triggered]]
|<---------+ CON 2.05 M:0xe4 T:0xf0 O:1235 B4:22/0/0/1/1024
|<---------+ CON 2.05 M:0xe5 T:0xf0 O:1235 B4:22/0/1/1/1024
|<---------+ CON 2.05 M:0xe6 T:0xf0 O:1235 B4:22/0/2/1/1024
|<---------+ CON 2.05 M:0xe7 T:0xf0 O:1235 B4:22/0/3/0/1024
|--------->+ ACK 0.00 M:0xe4
|--------->+ ACK 0.00 M:0xe5
|--------->+ ACK 0.00 M:0xe6
|--------->+ ACK 0.00 M:0xe7
...
[[Observe triggered]]
|<---------+ CON 2.05 M:0xe8 T:0xf0 O:1236 B4:23/0/0/1/1024
| X<---+ CON 2.05 M:0xe9 T:0xf0 O:1236 B4:23/0/1/1/1024
| X<---+ CON 2.05 M:0xea T:0xf0 O:1236 B4:23/0/2/1/1024
|<---------+ CON 2.05 M:0xeb T:0xf0 O:1236 B4:23/0/3/0/1024
|--------->+ ACK 0.00 M:0xe8
|--------->+ ACK 0.00 M:0xeb
| |
[[Server retransmits messages not acknowledged]]
|?---------+ CON 2.05 M:0xe9 T:0xf0 O:1236 B4:23/0/1/1/1024
| X<---+ CON 2.05 M:0xea T:0xf0 O:1236 B4:23/0/2/1/1024
|--------->+ ACK 0.00 M:0xe9
| |
[[Server retransmits messages not acknowledged
(exponential backoff)]]
| X<---+ CON 2.05 M:0xea T:0xf0 O:1236 B4:23/0/2/1/1024
| |
[[Either transmission failure (acknowledge retry timeout)
or successfully transmitted.]]
Figure 9: Example of CON Notifications with Block4 Option
It is implementation-dependent as to whether a CoAP session is
terminated following acknowledge retry timeout, or whether the CoAP
session continues to be used under such adverse traffic conditions.
If there is likely to be the possibility of network transient losses,
then the use of Non-confirmable traffic should be considered.
Boucadair & Shallow Expires November 25, 2020 [Page 23]
Internet-Draft New Block-Wise Transfer Options May 2020
Authors' Addresses
Mohamed Boucadair
Orange
Rennes 35000
France
Email: mohamed.boucadair@orange.com
Jon Shallow
United Kingdom
Email: supjps-ietf@jpshallow.com
Boucadair & Shallow Expires November 25, 2020 [Page 24]