NFS Protocol Extension: Retrospect and Prospect
draft-dnoveck-nfs-extension-00
The information below is for an old version of the document.
| Document | Type |
This is an older version of an Internet-Draft whose latest revision state is "Expired".
|
|
|---|---|---|---|
| Author | David Noveck | ||
| Last updated | 2013-09-09 | ||
| RFC stream | (None) | ||
| Formats | |||
| Additional resources | |||
| Stream | Stream state | (No stream defined) | |
| Consensus boilerplate | Unknown | ||
| RFC Editor Note | (None) | ||
| IESG | IESG state | I-D Exists | |
| Telechat date | (None) | ||
| Responsible AD | (None) | ||
| Send notices to | (None) |
draft-dnoveck-nfs-extension-00
NFSv4 D. Noveck
Internet-Draft EMC
Intended status: Informational September 09, 2013
Expires: March 13, 2014
NFS Protocol Extension: Retrospect and Prospect
draft-dnoveck-nfs-extension-00
Abstract
This document surveys the processes by which the NFS protocol has
been extended in the past and considers how the mechanisms by which
the protocol is extended might best be modified in the future.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on March 13, 2014.
Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Noveck Expires March 13, 2014 [Page 1]
Internet-Draft nfs-extension September 2013
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Protocol Extension . . . . . . . . . . . . . . . . . . . . . 3
3. Protocol Extension Mechanisms . . . . . . . . . . . . . . . . 3
3.1. Specific Protocol Mechanisms Designed for Extension . . . 4
3.2. Protocol Extension by XDR Replacement . . . . . . . . . . 4
3.3. Protocol Extension by XDR Extension . . . . . . . . . . . 5
3.4. Combination of Protocol Extension Mechanisms . . . . . . 6
4. Pre-IETF NFS Versioning . . . . . . . . . . . . . . . . . . . 6
4.1. The Pre-IETF NFS Environment . . . . . . . . . . . . . . 6
4.2. Transition from NFSv2 to NFSv3 . . . . . . . . . . . . . 7
5. NFS Versioning (so far) Within IETF . . . . . . . . . . . . . 7
5.1. Transition from NFSv3 to NFSv4 . . . . . . . . . . . . . 7
5.2. Initial Minor Versioning Model for NFSv4 . . . . . . . . 8
5.3. Transition from NFSv4.0 to NFSv4.1 . . . . . . . . . . . 10
5.4. Transition from NFSv4.1 to NFSv4.2 . . . . . . . . . . . 11
5.5. Evolution of Minor Versioning Model within NFSv4 . . . . 12
5.6. Current Minor Versioning Model for NFSv4 . . . . . . . . 13
5.7. Review of NFSv4 Versioning so far . . . . . . . . . . . . 14
6. NFSv4 Versioning Now . . . . . . . . . . . . . . . . . . . . 15
6.1. Current NFS Versioning Practices . . . . . . . . . . . . 15
6.2. Problems with Current NFS Versioning Approach . . . . . . 15
7. Going Forward with a New NFSv4 Extension Approach . . . . . . 17
7.1. Extension Mechanisms used for Protocol Updates . . . . . 17
7.2. Requirements for a New NFSv4 Extension Approach . . . . . 18
7.3. Principles upon which to Base a New NFSv4 Extension
Approach . . . . . . . . . . . . . . . . . . . . . . . . 19
7.4. Work Going Forward in Creating a New NFSv4 Extension
Approach . . . . . . . . . . . . . . . . . . . . . . . . 20
8. Security Considerations . . . . . . . . . . . . . . . . . . . 21
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 21
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 21
11.1. Normative References . . . . . . . . . . . . . . . . . . 21
11.2. Informative References . . . . . . . . . . . . . . . . . 22
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 23
1. Introduction
This document examines the subject of protocol extension within the
NFS family of protocols. In order to better understand the issues
that exist going forward with NFSv4, we examine the history of
protocol extension throughout the development of NFS including both
the pre-IETF period and the development of successive NFSv4 minor
versions.
Noveck Expires March 13, 2014 [Page 2]
Internet-Draft nfs-extension September 2013
We then use this history as a basis upon which to explore the issues
involved in providing a modified extension paradigm that builds on
the work already done, but is more flexible.
2. Protocol Extension
Often, protocols require means by which they can be extended. Such
extension may be needed to meet new requirements, to correct protocol
weaknesses exposed by experience, or even to correct protocol bugs
(as can happen when protocols are published as RFC's without fully
fleshed-out implementations).
We need to distinguish here between protocol "extension" and
"versioning". Versioning is a form of protocol extension but not
every form of protocol extension can be accommodated within a
versioning paradigm.
When a versioning paradigm is in place, groups of extensions are
conceived of as ordered, allowing extensions in subsequent versions
to build upon those in previous versions. When multiple extensions
are combined into a single version, each of the extensions may be
built assuming that the others will be present as well. In such
cases, there can be the opportunity to make design changes in the
protocol, allowing elements of the protocol to be restructured,
sometimes in major ways.
When a versioning paradigm is in effect and extensions are optional,
extensions cannot build upon one another, since the presence of any
particular extension cannot be assumed. In such cases, the ability
to restructure the protocol is reduced, but smaller changes may be
introduced more easily.
In this latter case, it is not clear that the word "versioning" is
appropriate. Nevertheless, in this document, we will, as in the
phrase "NFSv4 minor versioning" use the existing terminology without
necessarily subscribing to the view that "versioning" is the
appropriate description.
3. Protocol Extension Mechanisms
Some factors that are often relevant in deciding on the means by
which a protocol will be extended.
o Whether extensions are to be individually selectable (i.e.
optional) or assumed to be always present, allowing one to build
upon another earlier one.
o The size and scope of extensions that will be made.
Noveck Expires March 13, 2014 [Page 3]
Internet-Draft nfs-extension September 2013
o Compatibility issues with existing implementations.
o Issues that relate to ensuring that when individual extensions,
separately arrived at, are each optionally allowed, that the ones
used are compatible and can be used effectively together.
o The overall implementation framework. For example, RPC-based
protocols may do extension by means of the RPC version mechanism.
While it is possible to use different sorts of extension mechanisms
for different sorts of extensions, protocols typically do not take
advantage of that flexibility.
On the other hand, protocols do, as NFS has done, change their
preferred extension mechanisms in response to long-term changes in
requirements. However, once having done so, they rarely switch back.
Changing extension mechanisms is a big step, both conceptually and in
implementation terms, and is not frequently repeated.
3.1. Specific Protocol Mechanisms Designed for Extension
Often, protocols will be designed with specific mechanisms, designed
to allow protocol extension. An example is the provision for TCP
options (see [RFC0793] and [RFC2780].) Most often, such mechanisms
are designed to allow individual extensions to be designed and
implemented independently, with any dependency relations between
extensions specified separately and not enforced by the extension
mechanism itself.
3.2. Protocol Extension by XDR Replacement
RPC-based protocols may, and often do, provide for protocol extension
by replacing the XDR for one version with that for another and using
the RPC versioning mechanism to manage selection of the proper
protocol variant. The use of the RPC versioning mechanism enforces a
versioning paradigm of this sort on protocols using this extension
mechanism.
This extension mechanism allows very extensive protocol changes, up
to and including the replacement of one protocol by an entirely
different one. For some kinds of protocol extensions, this seems the
only way to effect the change.
Noveck Expires March 13, 2014 [Page 4]
Internet-Draft nfs-extension September 2013
3.3. Protocol Extension by XDR Extension
It is possible to replace an XDR definition by one which is an
extension in the sense that
o The set of messages described by the second definition is a
superset of that described by the first.
o Each message within the set of messages described by the first
definition is recognized as having exact the same structure/
interpretation by the second definition.
o Each message within the set of messages described by the second
definition but not the first definition must be recognized as part
of an unsupported extension.
Within an XDR/RPC framework, extensions can be arrived at by:
o Addition of previously unspecified RPC requests.
o Addition of new, previously unused, values to existing enums.
o Addition of previously unassigned bit values to a flag word.
o Addition of new cases to existing switches, provided that the
existing switch did not contain a default case.
Such an extension relation between XDR descriptions is reflexive and
transitive and thus defines a partial order. In practice, provisions
have to be made to make sure that two extensions of the same
description are compatible (i.e. either one is an extension of the
other, or there is a another description that is a valid extension of
both).
To put things in concrete terms, such compatibility can be assured if
measures are taken to ensure:
o That the same request number is not used for two different
requests.
o That the same enum value is not assigned two different meanings.
o That the same bit flag value is not assigned two different
meanings.
o That whenever the same case value is added to the same switch in
two different extensions, the content assigned to the two matching
added cases is the same.
Noveck Expires March 13, 2014 [Page 5]
Internet-Draft nfs-extension September 2013
o That default cases are never added to existing switches.
3.4. Combination of Protocol Extension Mechanisms
It is possible to use multiple of these means of protocol extension
simultaneously. When this is done, the result is a composite
extension mechanism. For example, if the XDR replacement or XDR
extension mechanism is adopted, a protocol-specific mechanism can be
added to it, if the protocol-specific mechanism is built on objects
whose XDR definition is sufficiently generic. (e.g. opaque arrays or
feature bitmasks).
It can be argued that the NFSv4 attribute model provides such an
embedded protocol-specific extension mechanism, since sets of
attribute values are specified as XDR opaque arrays and attribute
sets are specified as variable-length arrays of 32-bit integers
allowing new attribute bits to be defined outside of the XDR
definition framework.
Note that there exists specification text that suggests that
attributes are part of the XDR specification, making it hard to reach
a firm conclusion on the matter. However, the resolution of this
question does not affect the other matters discussed below, since, in
either case, we have an extension mechanism that allows optional
extensions.
4. Pre-IETF NFS Versioning
4.1. The Pre-IETF NFS Environment
NFSv2 and NFSv3 were described by the informational RFC's [RFC1094]
and [RFC1813]. These documents each described existing
interoperating client and server implementations. Thus they started
with running code. If there was a rough consensus in effect, it was
that these were useful protocols to use and thus that someone
building a client or server had to interoperate with the existing
implementations.
The following characteristics of protocol development during that
period are noteworthy.
o Most client implementations were implemented on very similar
systems, in terms of the API's supported and many specifics of the
local filesystems exported by servers.
o Often, the important client and server implementations were done
by the same organization.
Noveck Expires March 13, 2014 [Page 6]
Internet-Draft nfs-extension September 2013
As a result of these commonalities, specifications tended to avoid a
lot of detail that would have been required in a more diverse
environment. New features were thought of in terms of generally
understood client and server implementation frameworks and it was
generally clear which of those could be implemented without markedly
changing that framework.
4.2. Transition from NFSv2 to NFSv3
There were a number of significant changes involved, but only the
first two were of major importance.
o Converting file sizes and offsets from 32-bit to 64-bit unsigned
integers.
o Support for uncommitted WRITEs and the COMMIT request.
o Provision for WRITE to return atomic pre- and post-write file
attributes, in order to allow a client to determine whether
another client was writing the file.
Interestingly enough, this feature was not carried over into
NFSv4.
o The READDIRPLUS request.
o The addition of NFS3ERR_JUKEBOX (the precursor of NFS4ERR_DELAY).
Of these only the first actually needed something as drastic as the
XDR replacement model. The others could have been handled simply by
adding new RPC requests and an enum value to an existing NFSv2 XDR.
Since, NFS's extension mechanism was then XDR replacement, such
choices were not available.
5. NFS Versioning (so far) Within IETF
5.1. Transition from NFSv3 to NFSv4
NFSv4 was the first NFS version published as a Standards track
document. Although an initial [RFC3010], entitled "NFS version 4
protocol" was published as a Proposed Standard, it was never
implemented due to issues with the design of locking.
Subsequently, [RFC3530], entitled "Network File System (NFS) version
4 Protocol" was published as a Proposed Standard, obsoleting
[RFC3010]. Currently, there are bis documents, [RFC3530bis] and
[RFC3530bis-dotx], nearing publication.
Noveck Expires March 13, 2014 [Page 7]
Internet-Draft nfs-extension September 2013
The set of changes made to create NFSv4 was larger by far than that
for NFSv3. A partial list follows.
o Conversion to a stateful protocol, including support for locking.
Locking included support for OPENs (with share reservations),
byte-range locking (optionally including mandatory byte-range
locks) and delegation.
o The COMPOUND operation.
o Conversion to a bi-directional protocol, by the addition of
(optional) callbacks.
o Internationalization.
o Support for filesystems doing case-insensitive name matching.
o A new, extensible file attribute model, including support for
acls, and conversion of user and group to a string model, opening
the way for multi-domain support.
o Support for named attributes.
o Merger of ancillary protocols (e.g. MOUNT, NSM, NLM) into the NFS
Protocol proper.
o Support for crossing of filesystem boundaries within a server's
file name space (originally done for the incorporation of MOUNT
functionality).
o Support for such multi-server operations as migration,
replication, and referral.
Referrals were not explicitly mentioned in [RFC3530] and are
explained in [RFC3530bis].
o Creation of a minor versioning model (to be discussed in
Section 5.2) to allow further protocol extension.
These features/extensions were implemented via the XDR replacement
model. This was the only realistic alternative, not only because of
the size of the list, but also because some of the changes undercut
some central design elements of the pre-IETF NFS protocol.
5.2. Initial Minor Versioning Model for NFSv4
The minor versioning model for NFSv4 is an XDR extension model. It
was presented within a versioning paradigm but the fact that all the
Noveck Expires March 13, 2014 [Page 8]
Internet-Draft nfs-extension September 2013
added features were to be (at least initially) optional indicated
that features were intended to be built independently, and that
clients were expected to deal with their presence or absence. Note
that the term "features" is not explicitly defined. We assume that
the definition includes operations within COMPOUND or CB_COMPOUND,
attributes, flag bits enum values, and new cases of XDR switch
definitions.
Now let's look at some specifics of the minor version rules
established for NFSv4 in [RFC3530]. Note that some of these were
significantly modified by [RFC5661] and [NFSv42], as discussed in
Section 5.6.
o No RPC requests may be added. Thus COMPOUND (and NULL) are to be
the only requests within all NFSv4 minor versions.
Similarly for callbacks, CB_COMPOUND and CB_NULL are the only
requests within callback program.
o The arguments for COMPOUND and CB_COMPOUND contain a 32-bit
minorversion field.
Although this field is part of the minor versioning paradigm, it
is not clear how useful it is, as long as all extensions are
optional. For a more detailed discussion of this issue, see
Section 5.6.
o Features may be defined as optional, recommended, or mandatory.
These designations apply to implementation by the server. For
clients, no operations are mandatory, although it is hard to
imagine an NFSv4.0 client that does not use PUTFH or SETCLIENTID,
for example.
o Features may be upgraded or downgraded along the optional/
recommended/mandatory scale.
o Features may be declared "mandatory to not implement". This
allows the deletion of a feature while retaining as reserved the
value previously assigned.
o Clients and servers that support a particular minor version must
support all previous minor versions as well.
o New features may not be designated as mandatory in the minor
version in which they are introduced.
Noveck Expires March 13, 2014 [Page 9]
Internet-Draft nfs-extension September 2013
o Clients are not allowed to use stateids, filehandles, or similar
returned objects from the COMPOUND procedure with one minor within
another COMPOUND procedure with a different value of the
minorversion field.
This model was subsequently modified in [RFC5661] and in [NFSv42].
See Section 5.3 and Section 5.4 for details.
Many of the events anticipated in the model presented above have
never been realized and it may be that they never will be realized.
See Section 5.5 for some details. Examples are:
o There have never been recommended operations.
o There have never been optional attributes.
o Features have never been upgraded or downgraded in the transition
between minor versions.
5.3. Transition from NFSv4.0 to NFSv4.1
NFSv4.1 made a major change to NFSv4.0. It was able to do so using
an XDR extension model although it did not follow the rules laid out
in Section 5.2. Specifically, some features were declared
"infrastructural" and thus mandatory upon introduction.
Note that at the same time, the requirement that clients and servers
support previous minor versions changed from a "must" to a "SHOULD".
Presumably, this change reflects the fact that a minor version with
substantial infrastructural changes is essentially a new protocol,
making the "must" seem dubious. Whether the "SHOULD" here meets the
requirements of [RFC2119] needs to be explored.
NFSv4.1 was described in [RFC5661] and [RFC5662], each of which was
published as a Proposed Standard.
The following features were added as infrastructural features.
o Support for a sessions model including support for EOS (exactly-
once semantics).
Note that COMPOUND was taken advantage of to avoid adding slot and
sequence information to the request header. Instead this
information is packaged in a SEQUENCE or CB_SEQUENCE operation at
the start of the COMPOUND or CB_COMPOUND.
o A new set of operations were added which enable the client and
server to identify themselves to one another.
Noveck Expires March 13, 2014 [Page 10]
Internet-Draft nfs-extension September 2013
Although these are often thought of as part of the sessions model,
in fact they are logically distinct.
o RECLAIM_COMPLETE to allow better server sequencing of lock reclaim
operations.
There also a number of optional features.
o Parallel NFS
o WANT_DELEGATION to allow delegations to be obtained apart from
opens.
o Directory delegations and notifications.
o The FS_LOCATIONS_INFO and FS_STATUS attributes.
Note that there has been little implementation work on the last two
of these.
Parallel NFS created an alternate protocol extension mechanism for
NFS. New pNFS mapping types could be added. Existing mapping types
might have their own extension mechanisms. There also exists the
possibility that features might be added within the NFSv4 protocol
proper, designed to, or capable of, interacting with particular
mapping types. This document will not address these issues but
eventually, the NFSv4 Protocol will have to deal with them.
5.4. Transition from NFSv4.1 to NFSv4.2
While NFSv4.2 has not been defined in an RFC, it is fairly close to
completion. The descriptions in [NFSv42] and [NFSv42-dotx] can
serve as useful references.
The following features (all optional) are provided for in NFSv4.2:
o Support for labeled NFS.
o Server-side copy.
o An operation fence option on EXCHANGE_ID.
o Application data holes (formerly application data blocks).
o Disk-space reservation (nominally "recommended" since it is
implemented by an attribute and attributes have never been
declared "optional").
Noveck Expires March 13, 2014 [Page 11]
Internet-Draft nfs-extension September 2013
o Hole-punching operations.
o READ_PLUS
Note that there are two piece of infrastructure that are used by
multiple features above. These are not "infrastructural" in the
sense mentioned in Section 5.3 (i.e. they are not mandatory), but
they do serve an infrastructural role in that are required to be
present if one of the optional features that use them are supported.
o WRITE_PLUS used to implement (ordinary) hole punching and
application data holes.
o OFFLOAD operations/callbacks used to support WRITE_PLUS and
server-side copy.
5.5. Evolution of Minor Versioning Model within NFSv4
As noted above, there have been changes made by [RFC5661] and
[NFSv42] in the NFSV4 minor versioning model.
o NFSv4.1 (in [RFC5661] introduced the concept of "infrastructural"
features (i.e. those defined as mandatory at initial
introduction).
o NFSv4.2 (in [NFSv42] added the concept of feature obsolescence
allowing implementers to get early notice of the expectation that
some features are on the path to becoming mandatory-to-not-
implement.
With these changes, we can classify potential minor versions,
starting with those that currently exist.
o Minor version zero which introduced a new (major) version of the
NFS protocol. All of the operations within it are new and a
subset are effectively mandatory.
o Minor versions which introduce a new operation and make it
mandatory (based on its being infrastructural).
Currently, the only such version is minor version one, although
there may be others in the future.
o All other minor versions. These add only optional/recommended
features, each present or absent on the server with clients
needing to be able to deal individually with their presence or
absence.
Noveck Expires March 13, 2014 [Page 12]
Internet-Draft nfs-extension September 2013
Currently, the only such version is minor version two. It is
likely there will be others in the future and it may be that all
future minor versions will be of this character.
We term versions in the first two categories "infrastructure-level
versions" Such versions form an ascending sequence in which the
difference between, for example, NFSv4.0 and NFSv4.1, is very similar
to the difference between NFSv2 and NFSv3. Clients may be designed
for one or the other, or for both but a client capable of interacting
with both is really choosing between two different protocols.
We term versions in the last category "optional-feature-only
versions".
Note that although the concept of optional features being upgraded to
mandatory status remains, it is likely that it will not be used very
much, if at all, in the future. The situation is similar for the
case of features being downgraded to mandatory-to-not-implement.
Given the diversity of NFS clients and servers, it is highly unlikely
that a new non-infrastructural feature will be so broadly necessary/
desirable that a consensus to make it mandatory would be likely to
arise. Such a decision would prevent servers not implementing such a
feature from incorporating other later-developed features. It is
only when a feature is judged so useful by users that people will not
use servers without it, that adoption will become universal. At this
point, a decision to make it mandatory would merely ratify what had
already happened on its own.
Except in the case of a universally recognized mistake, any
downgrading to mandatory-to-not-implement, would only happen when a
replacement becomes mandatory so the considerations above make that
situation equally unlikely to occur.
Still, it is possible that versions making such feature status
changes will be created in the future. We will call any such
"mandatory-feature-change" versions.
5.6. Current Minor Versioning Model for NFSv4
Minor versions which are infrastructure-level or which are,
mandatory-feature-change versions form an ascending sequence in which
we also have a versioning paradigm, implemented using XDR extension.
Optional-feature-only versions are fundamentally different. Each
NFSv4.2 server implements the same protocol as NFSv4.1 with a
particular set of optional features beyond those that are mandatory.
This set may range from the null set all the way to all of the
Noveck Expires March 13, 2014 [Page 13]
Internet-Draft nfs-extension September 2013
optional features. Here, it appears that the versioning paradigm is
not appropriate to the reality of the extension mechanism.
As a way of illustrating the basic point here, let us consider two
servers each of which only supports operations within NFSv4.1:
o The first server "supports" NFSv4.2 but none of the optional
features added in [NFSv42]. In this case, any attempt by a client
to use one of those features will result in an NFS4ERR_OPNOTSUPP
being returned.
o Let us say that the second server does not support NFSv4.2 and
supports precisely the same set of features. In this case, a
request will be rejected (with error NFS4RR_....) if its COMPOUND
minorversion field is two and if the field is one, any unsupported
NFSv4.2 operation will be rejected with NFS4ERR_OP_INVAL.
Although this obeys the rules as they stand, there is no real value
for the client, the server, or the protocol in making these
artificial distinctions. Optional-feature-only minor versions such
as NFSv4.2 are not minor versions in the same sense that NFSv4.0 and
NFSv4.1 are. In this case the minorversion field is not providing
any information, while the set of operations supported is the
important thing that the server implementer chooses and the client
needs to know.
In later sections we will discuss how this mismatch might be best
addressed as NFSv4 development proceeds.
5.7. Review of NFSv4 Versioning so far
To summarize protocol extension as it applies to the NFSV4 protocols:
o NFSV4.0 was implemented using the XDR replacement approach
inherited from NFSv2 and NFSv3. As was to be expected given the
nature and scope of the changes, its development took considerable
time.
It defined a protocol extension approach based on the XDR
extension mechanism which was designed to enable future
development of minor versions. However, this mechanism was not
used as part of the implementation of NFSv4.0.
o NFSV4.1 was implemented using the XDR extension mechanism. To
implement sessions, it was forced to modify the extension approach
in the only way that was viable in the circumstances. As a
result, the specification process took a long time, since it made
significant structural changes to the protocol and also because it
Noveck Expires March 13, 2014 [Page 14]
Internet-Draft nfs-extension September 2013
had to specify the entre protocol, and not just a set of
extensions.
o NFSV4.2 returned to the original XDR extension mechanism and was
intended to be a small incremental update with a one-hundred page
(or less) specification. The fact that this turned out to be a
multi-year effort has occasioned concern and we will attempt to
see how the process can be streamlined.
6. NFSv4 Versioning Now
6.1. Current NFS Versioning Practices
The following pattern was followed for NFSv4.2, and, unless changes
are made, seems likely to persist.
o Various features are sketched out in individual drafts
o The working group reaches a decision (i.e. by rough consensus) as
to the extensions/features to be included in the minor version.
o The existing individual drafts are combined into a draft of a
working group document intended to eventually become the RFC
describing the new minor version.
o This document goes though further refinement and cycles of working
group document review. At some point a companion -dot-x document
is prepared and reviewed by the working group as well.
o The two documents go through working group last call, IESG review,
and RFC publication.
This pattern of development is not a good fit for the kind of minor
version that NFSv4.2 is and many future such minor versions will be.
Such versions consist of a set of mostly unrelated features, each
individually selectable or not by implementers, artificially yoked
together. In essence, we have a "feature batch" rather than a minor
version.
6.2. Problems with Current NFS Versioning Approach
A number of issues have been noted with the current process for
NFSv4.2, leading to the conclusion that the process needs to be
revised in some way for future minor versions, of the same sort.
o It takes too long to get a minor version drafted and through
working group IESG review. Despite the fact that NFSv4.2 was
intended to be a fairly minimal minor version, describable in a
Noveck Expires March 13, 2014 [Page 15]
Internet-Draft nfs-extension September 2013
one-hundred-page spec, it looks like the pace of development is
such that there will be about a four-year gap between the time
that NFSv4.2 was started and an equivalent point for NFSv4.3, if
that pattern is maintained.
o We still do not have significant active implementations in which
proposed last-minute protocol changes can be tested for validity.
As an example of the problem, consider the decision to pass source
stateids to the COPY op. If there were an implementation of
inter-server server-side copy, the problems that this created
(since stateids are tied to clientids in NFSv4.1 and beyond) would
have quickly become manifest.
o Many features within NFSv4.2 have not received the kind of
searching review appropriate to this stage of specification
development. Some examples are discussed below.
Some instances of problems/issues ascribable to a lack of searching
document review:
o The state of the IO hints feature is most unsatisfactory. It is
not clear how, or even if, it is possible to specify in a way that
interoperable clients and servers can be written.
o It was the general understanding within the group that labeled NFS
required use of RPCSEC_GSSv3, while RPCSEC_GSSv3 was not being
worked on, and had little chance of being worked on.
o The security for inter-server copy was specified to be dependent
on RPCSEC_GSSv3, yet, when it was found that RPCSEC_GSSv3 was not
on the horizon, it turned out that a simple alternative was
available, and, the functionality needed for inter-server copy was
not really anticipated for RPCSERC_GSSv3 either.
If we look at the problems above, we can understand better how such
problems can arise. In short, the decision as to what features to
include within a minor version, is not a good use of the rough
consensus model and in proceeding on that basis, the group created a
set of perverse incentives that undercut the process. Also, as the
process goes on for a long time, as is likely, these perverse
incentives are intensified. Consider the following points:
o It is not clear exactly what the consensus as to proposed minor
version contents actually means. Working group members might
interpret it as meaning "These features are worth pursuing and
they should be pursued". However, if they thought the definition
was more like, "each of these features is so important that, if it
is not ready, any other feature, including the one I'm interested
Noveck Expires March 13, 2014 [Page 16]
Internet-Draft nfs-extension September 2013
in, should be delayed also", then it is hard to imagine any such
rough consensus existing. Note that, given the minor versioning
implementation laid out above (in Section 6.1), the latter
definition is, for functional purposes, the effective meaning, of
the minor version content consensus.
o Given that many features are linked together, any delay in one
feature, once it is accepted as part of the feature batch, delays
all of the features, making it hard for people to comment
forthrightly on any significant specification inadequacies. Not
only will it delay your preferred feature, but if the problems are
not fixed, the only recourse is an extreme penalty. As a result,
it often seems not worth pursuing these sorts of issues.
o As the version turnaround cycle is so long, it is very difficult
to remove a feature from a minor version feature batch. Given
that these are all features that have enough interest to be in the
minor version, it is hard to kick the feature into the next minor
version, given that will certainly mean a multi-year delay, even
if the feature could be guaranteed admission to the next feature
batch.
o Given that responsibility for a minor version is transferred to
the editor of minor version definition documents at an early
stage, we have a process in which it is not clear who has
responsibility to follow up on the work necessary other than the
minor version editor who may not have the required time and
expertise in all cases. There is not a designated feature owner
with responsibility to make the feature happen.
7. Going Forward with a New NFSv4 Extension Approach
As we work to correct the issues noted above, and fill out the
details of a modified extension paradigm, we will have to take note
of the design considerations put forth in [RFC6709].
7.1. Extension Mechanisms used for Protocol Updates
It is generally held to be the case, that a document updating a minor
version RFC, is not allowed to extend the XDR. Sometimes typedefs
are added to make it clearer how particular fields are used. Also
comments have been added and minimal reformatting done, but even
addition of new error codes, as opposed to adding existing error
codes to the list of those allowed to be returned by a given
operation, has not been allowed.
Noveck Expires March 13, 2014 [Page 17]
Internet-Draft nfs-extension September 2013
Given that we have an XDR extension paradigm in place, it does not
make sense to prohibit XDR extensions to be made in these update
documents.
It should be noted that the prohibition of such XDR changes is not
explicitly mentioned anywhere, to my knowledge. Rather, it seems to
be a piece of NFSv4 versioning folklore which needs to be either
justified or discarded.
Acknowledging this change would allow us to do the following sorts of
things (in addition to the specification bug fixes now allowed) in
bis documents and other documents which update minor version
definition RFC's. While it would be theoretically possible to add
entirely new features, working group and IESG review should keep
additions limited to the following two sorts of items.
o Adding new operations to correct protocol bugs, subject to the
proviso that a server implementing a replacement operation must
also support the operation replaced. In this case, uniqueness for
values such as operation and attribute numbers would assured by
defining them in a later minor version's XDR definition document,
or some update thereof. If the bug fix does not apply to that
later minor version, it can be treated there as mandatory-to-not-
implement, to match what it replaces.
o Backporting of features whose usefulness has been proven in a
subsequent minor version and can be easily made available in an
earlier minor version. In this case, values such as operation and
attribute numbers are already assured of uniqueness, due to their
assignment as unique in the later minor version.
Doing things this way would address the issues that have given rise
to the perceived need for "micro-versioning". Note that the sorts of
changes we would be making would not require any change in the
minorversion field, at all.
7.2. Requirements for a New NFSv4 Extension Approach
The following requirements will govern construction of a possible new
protocol extension approach for NFSv4.
o That individual extensions not be documented together (i.e. in the
same document) unless there are good reasons to do so (e.g. the
extensions use common facilities not documented anywhere else,
there is a dependency relationship among the extensions allowing
one to be used only when others are available). This would allow
for shorter documents and a less trying review process.
Noveck Expires March 13, 2014 [Page 18]
Internet-Draft nfs-extension September 2013
o The process should allow for protocol bugs to be fixed, even if
the problem is found after RFC publication. Just as we have the
errata process to fix spec issues, we should be able to fix bugs/
oversights in the XDR, as long as compatibility issues with
existing implementations are addressed.
o That the process gives the working group an appropriate
opportunity to review extensions and reject those that are
architecturally inappropriate.
o That the process gives the working group and IESG an appropriate
opportunity to review extension specifications and get them fixed,
without adversely affecting other unrelated features.
o That the process appropriately assigns the responsibility for
making proposed extensions real on those proposing them. This
should include, at an appropriate time, work to get "running code"
(i.e. client and server implementations) to demonstrate
implementation feasibility.
7.3. Principles upon which to Base a New NFSv4 Extension Approach
The following principles seem the best way to meet these requirements
without a disruptive major-version-scale shift in the NFSv4
definition (i.e. something as big as the shift from NFSv3 to NFSv4 or
from NFFSv4.0 to NFSv4.1)
o That the versioning paradigm not be used where it cannot actually
be taken advantage of (i.e. where all extensions are optional).
o That the extension mechanism be usable to correct protocol bugs in
bis documents and other RFC's updating existing RFC's. See
Section 7.1 for details.
o That a new, more flexible workflow be designed for ongoing
protocol extension development. This should include some
recognition of the role of the development of implementations in
the maturing of a feature.
o That appropriate version negotiation/discovery features be added
to allow clients to discover what facilities a server supports,
without having to attempt to use them all.
Noveck Expires March 13, 2014 [Page 19]
Internet-Draft nfs-extension September 2013
7.4. Work Going Forward in Creating a New NFSv4 Extension Approach
The following set of steps are necessary in many possible ways of
proceeding along the way to a new extension approach for NFSv4.
Details and actual sequencing will reflect choices that the working
group makes.
o Creation of a new standards-track document defining how protocol
extension and versioning are to work in NFSv4.
Since it would supersede existing treatments of the issues, it
should be recorded as updating specifications for NFSv4.0
([RFC3530] or the RFC arising from [RFC3530bis], when approved),
for NFSv4.1 ([RFC5661]), and for NFSv4.2 (the RFC arising from
[NFSv42] when approved).
o Establishment of a framework for extension discovery (and
negotiation if that is judged necessary) to replace testing of
operations for an NFS3ERR_OPNOTSUPP response.
The operations and attributes for this framework might be
documented in the document defining the protocol extension
framework, in a free-standing standards-track document, or as an
infrastructural feature in a new minor version. In any of those
cases, they could be incorporated as optional in earlier minor
versions by documents updating the minor version specification
RFC's.
o Design of processes to review proposed extensions for
architectural suitability, to reserve necessary operation codes,
attribute numbers, and enum and flag values, and to decide when
and if minor versions should be created to upgrade or downgrade
features.
Such processes might well be documented in a working group
informational document, but any such documentation should probably
be done only after the working group is satisfied that the
processes are working well.
As far as the value reservation issue, we will have to decide
whether an IANA-based approach, as recommended by [RFC6709], is
required or whether simpler procedures, implemented within the
working group, are adequate.
As far as the document specifying the NFSv4 extension and versioning
framework, the following are important elements:
Noveck Expires March 13, 2014 [Page 20]
Internet-Draft nfs-extension September 2013
o Clearly separate the concepts of protocol extension and versioning
to allow the basic XDR extension approach to be used in contexts
other than creation of a minor version.
o Clearly specify rules and expectations for updates to already
defined minor versions.
o Discourage any future incorporation of the definition of protocol
extensions in minor version definition documents, except where the
extension is infrastructural. Thus the basic function of minor
version definition documents would be to specify what features
already defined are included and their status (experimental,
optional, recommended or mandatory) in that minor version.
o Resolve and clarify cases where the original minor versioning
rules don't match the extension/versioning model as it has evolved
(e.g. how far the obligation of a client to support earlier
versions extends, across what sorts of version changes does it
make sense to require non-use of filehandles, stateids, etc.)
8. Security Considerations
Since no substantive protocol changes are proposed here, no security
considerations apply.
As extensions are designed and specified, their security issues will
be addressed and each extension will receive the appropriate security
review from the NFSv4 working group and IESG.
9. IANA Considerations
The current document does not require any actions by IANA.
Depending on decisions that the working group makes about how to
address the issues raised here, future documents may require actions
by IANA.
10. Acknowledgements
The author wishes to thank Chuck Lever of Oracle for his helpful
document review and many important suggestions.
11. References
11.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
Noveck Expires March 13, 2014 [Page 21]
Internet-Draft nfs-extension September 2013
11.2. Informative References
[NFSv42-dotx]
Haynes, T., Ed., "NFS Version 4 Minor Version 2 External
Data Representation Standard (XDR) Description ", 2013,
<http://www.ietf.org/id/draft-ietf-
nfsv4-minorversion2-dot-x-19.txt>.
Work in progress.
[NFSv42] Haynes, T., Ed., "NFS Version 4 Minor Version 2 ", 2013,
<http://www.ietf.org/id/draft-ietf-
nfsv4-minorversion2-20.txt>.
Work in progress.
[RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC
793, September 1981.
[RFC1094] Nowicki, B., "NFS: Network File System Protocol
specification", RFC 1094, March 1989.
[RFC1813] Callaghan, B., Pawlowski, B., and P. Staubach, "NFS
Version 3 Protocol Specification", RFC 1813, June 1995.
[RFC2780] Bradner, S. and V. Paxson, "IANA Allocation Guidelines For
Values In the Internet Protocol and Related Headers", BCP
37, RFC 2780, March 2000.
[RFC3010] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R.,
Beame, C., Eisler, M., and D. Noveck, "NFS version 4
Protocol", RFC 3010, December 2000.
[RFC3530] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R.,
Beame, C., Eisler, M., and D. Noveck, "Network File System
(NFS) version 4 Protocol", RFC 3530, April 2003.
[RFC3530bis-dotx]
Haynes, T., Ed. and D. Noveck, Ed., "Network File System
(NFS) Version 4 Protocol External Data Representation
Standard (XDR) Description ", 2013, <http://www.ietf.org/
id/draft-ietf-nfsv4-rfc3530bis-dot-x-18.txt>.
Work in progress.
[RFC3530bis]
Noveck Expires March 13, 2014 [Page 22]
Internet-Draft nfs-extension September 2013
Haynes, T., Ed. and D. Noveck, Ed., "Network File System
(NFS) Version 4 Protocol ", 2013, <http://www.ietf.org/id/
draft-ietf-nfsv4-rfc3530bis-27.txt>.
Work in progress.
[RFC5661] Shepler, S., Eisler, M., and D. Noveck, "Network File
System (NFS) Version 4 Minor Version 1 Protocol", RFC
5661, January 2010.
[RFC5662] Shepler, S., Eisler, M., and D. Noveck, "Network File
System (NFS) Version 4 Minor Version 1 External Data
Representation Standard (XDR) Description", RFC 5662,
January 2010.
[RFC6709] Carpenter, B., Aboba, B., and S. Cheshire, "Design
Considerations for Protocol Extensions", RFC 6709,
September 2012.
Author's Address
David Noveck
EMC Corporation
228 South Street
Hopkinton, MA 01748
US
Phone: +1 508 249 5748
Email: david.noveck@emc.com
Noveck Expires March 13, 2014 [Page 23]