Operation of the IP Flow Information Export (IPFIX) Protocol on IPFIX Mediators
RFC 7119
| Document | Type | RFC - Proposed Standard (February 2014) | |
|---|---|---|---|
| Authors | Benoît Claise , Atsushi Kobayashi , Brian Trammell | ||
| Last updated | 2015-10-14 | ||
| RFC stream | Internet Engineering Task Force (IETF) | ||
| Formats | |||
| Additional resources | Mailing list discussion | ||
| IESG | Responsible AD | Joel Jaeggli | |
| Send notices to | (None) |
RFC 7119
Domain IDs; the IPFIX Mediator, in this case, comprises its own set
of Observation Domain(s) independent of the Observation Domain(s) of
the Original Exporters.
The second option is to provide or maintain a Template Mapping for
received (Options) Template Records and exported inferred (Options)
Template Records, along with the appropriate Observation Domain IDs
per Transport Session, which ensures that the combination of
Observation Domain IDs and Template IDs do not collide on export.
In some cases where the IPFIX Message Header can't contain a
consistent Observation Domain for the entire IPFIX Message, but the
Flow Records exported from the IPFIX Mediator should contain the
Observation Domain of the Original Exporter anyway, the (Options)
Template Record must contain the originalObservationDomainId
Information Element, specified in Section 6.1. When an IPFIX
Mediator receives Flow Records containing the
originalObservationDomainId Information Element, the IPFIX Mediator
MUST NOT modify its value(s) when composing new Flow Records with the
originalObservationDomainId Information Element.
6.1. originalObservationDomainId Information Element
Name: originalObservationDomainId
Description: The Observation Domain ID reported by the Exporting
Process on an Original Exporter, as seen by the Collecting Process
on an IPFIX Mediator. Used to provide information about the
Original Observation Domain to a downstream Collector. When
cascading through multiple Mediators, this identifies the initial
Observation Domain in the cascade.
Data Type: unsigned32
Data Type Semantics: identifier
ElementId: 405
7. Timing Considerations
The IPFIX Message Header "Export Time" field is the time in seconds
since 0000 UTC Jan 1, 1970, at which the IPFIX Message leaves the
IPFIX Mediator. However, in the specific case of an IPFIX Mediator
containing an Intermediate Conversion Process, the IPFIX Mediator MAY
use the export time received from the incoming Transport Session.
Claise, et al. Standards Track [Page 21]
RFC 7119 IPFIX MED-PROTO February 2014
It is RECOMMENDED that IPFIX Mediators handle time using absolute
timestamps (e.g., flowStartSeconds, flowStartMilliseconds, or
flowStartNanoseconds), which are specified relative to the UNIX epoch
(00:00 UTC 1 Jan 1970) [POSIX.1], where possible rather than relative
timestamps (e.g., flowStartSysUpTime or flowStartDeltaMicroseconds),
which are specified relative to protocol structures such as system
initialization or message export time.
The latter are difficult to manage for two reasons. First, they
require constant translation, as the system initialization time of an
intermediate system and the export time of an intermediate message
will change across mediation operations. Further, relative
timestamps introduce range problems. For example, when using the
flowStartDeltaMicroseconds and flowEndDeltaMicroseconds Information
Elements [IANA-IPFIX], the Data Record must be exported within a
maximum of 71 minutes after its creation. Otherwise, the 32-bit
counter would not be sufficient to contain the flow start time
offset. Those time constraints might be incompatible with some of
the application requirements of some Intermediate Processes.
Intermediate Processes MUST NOT assume that received records appear
in flowStartTime, flowEndTime, or observationTime order. An
Intermediate Process processing timing information (e.g., an
Intermediate Aggregation Process) MAY ignore records that are
significantly out of order, in order to meet application-specific
state and latency requirements, but SHOULD report that records were
dropped.
When an Intermediate Process aggregates information from different
Flow Records, the timestamps on exported records SHOULD be the
minimum of the start times and the maximum of the end times in the
general case. However, if the Flow Records do not overlap, i.e., if
there is a time gap between the times in the Flow Records, then the
report may be inaccurate. The IPFIX Mediator is only reporting what
it knows, on the basis of the information made available to it, and
there may not have been any data to observe during the gap. Then
again, if there is an overlap in timestamps, there's the potential of
double-accounting: different Observation Points may have observed the
same traffic simultaneously. The specification of the precise rules
for applying Flow Record timestamps at IPFIX Mediators for all the
different situations is out of the scope of this document.
Note that [RFC7015] provides additional specifications for handling
of timestamps at an Intermediate Aggregation Process.
Claise, et al. Standards Track [Page 22]
RFC 7119 IPFIX MED-PROTO February 2014
8. Transport Considerations
SCTP [RFC4960] using the Partially Reliable SCTP (PR-SCTP) extension
specified in [RFC3758] MUST be implemented by all compliant IPFIX
Mediator implementations. TCP [RFC0793] MAY also be implemented by
implementations compliant with the IPFIX Mediator. UDP [RFC0768] MAY
also be implemented by compliant IPFIX Mediator implementations.
Transport-specific considerations for IPFIX Exporters as specified in
Sections 8.3, 8.4, 9.1, 9.2, and 10 of [RFC7011] apply to IPFIX
Mediators as well.
SCTP SHOULD be used in deployments where IPFIX Mediators and
Collectors are communicating over links that are susceptible to
congestion. SCTP is capable of providing any required degree of
reliability. TCP MAY be used in deployments where IPFIX Mediators
and Collectors communicate over links that are susceptible to
congestion, but SCTP is preferred due to its ability to limit back
pressure on Exporters and its message versus stream orientation. UDP
MAY be used, although it is not a congestion-aware protocol.
However, in this case, the IPFIX traffic between IPFIX Mediator and
Collector MUST run in an environment where IPFIX traffic has been
provisioned for and/or separated from non-IPFIX traffic, whether
physically or virtually.
9. Collecting Process Considerations
Any Collecting Process compliant with [RFC7011] can receive IPFIX
Messages from an IPFIX Mediator. If the IPFIX Mediator uses IPFIX
Structured Data [RFC6313] to export Original Exporter Information, as
in Section 5, the Collecting Process MUST support [RFC6313].
10. Specific Reporting Requirements
IPFIX provides Options Templates for the reporting the reliability of
processes within the IPFIX Architecture. As each Mediator includes
at least one IPFIX Exporting Process, they MAY use the Exporting
Process Reliability Statistics Options Template, as specified in
[RFC7011].
Analogous to the Metering Process Reliability Statistics Options
Template, also specified in [RFC7011], Mediators MAY implement the
Intermediate Process Reliability Statistics Options Template,
specified in Sections 10.1, 10.3, and 10.4 define Information
Elements used by this Options Template.
The Flow Keys Options Template, as specified in [RFC7011], may
require special handling at an IPFIX Mediator, as described in
Section 10.2.
Claise, et al. Standards Track [Page 23]
RFC 7119 IPFIX MED-PROTO February 2014
In addition, each Intermediate Process may have its own specific
reporting requirements (e.g., Anonymization Records as in [RFC6235],
or the Aggregation Counter Distribution Options Template as in
[RFC7015]); these SHOULD be implemented as necessary, as described in
the specification for each Intermediate Process.
10.1. Intermediate Process Reliability Statistics Options Template
The Intermediate Process Statistics Options Template specifies the
structure of a Data Record for reporting Intermediate Process
statistics. It SHOULD contain the following Information Elements;
the intermediateProcessId Information Element is defined in
Section 10.3 and the ignoredDataRecordTotalCount Information Element
is defined in Section 10.4:
Claise, et al. Standards Track [Page 24]
RFC 7119 IPFIX MED-PROTO February 2014
+-----------------------------+-------------------------------------+
| IE | Description |
+-----------------------------+-------------------------------------+
| observationDomainId [scope] | An identifier of the Observation |
| | Domain (of messages exported by |
| | this Mediator), locally unique to |
| | the Intermediate Process, to which |
| | this statistics record applies. |
| | ---------------------------------- |
| intermediateProcessId | An identifier for the Intermediate |
| [scope] | Process to which this statistics |
| | record applies. |
| | ---------------------------------- |
| ignoredDataRecordTotalCount | The total number of Data Records |
| | received but not processed by the |
| | Intermediate Process. |
| | ---------------------------------- |
| time first record ignored | The timestamp of the first record |
| | that was ignored by the |
| | Intermediate Process. For Data |
| | Records containing timestamp |
| | ranges, this SHOULD be taken from |
| | the start timestamp of the range; |
| | for data records containing no |
| | timing information, this SHOULD be |
| | taken from the Export Time in the |
| | message header of the IPFIX Message |
| | that contains it. For this |
| | timestamp, any of the following |
| | timestamp can be used: |
| | observationTimeSeconds, |
| | observationTimeMilliseconds, |
| | observationTimeMicroseconds, or |
| | observationTimeNanoseconds. |
+-----------------------------+-------------------------------------+
Claise, et al. Standards Track [Page 25]
RFC 7119 IPFIX MED-PROTO February 2014
+-----------------------------+-------------------------------------+
| IE | Description |
+-----------------------------+-------------------------------------+
| time last record ignored | The timestamp of the last record |
| | that was ignored by the |
| | Intermediate Process. For Data |
| | Records containing timestamp |
| | ranges, this SHOULD be taken from |
| | the end timestamp of the range; for |
| | data records containing no timing |
| | information, this SHOULD be taken |
| | from the Export Time in the message |
| | header of the containing IPFIX |
| | Message. For this timestamp, any |
| | of the following timestamp can be |
| | used: observationTimeSeconds, |
| | observationTimeMilliseconds, |
| | observationTimeMicroseconds, or |
| | observationTimeNanoseconds. |
+-----------------------------+-------------------------------------+
10.2. Flow Key Options Template
The Flow Keys Options Template specifies the structure of a Data
Record for reporting the Flow Keys of reported Flows. A Flow Keys
Data Record extends a particular Template Record that is referenced
by its templateId identifier. The Template Record is extended by
specifying which of the Information Elements contained in the
corresponding Data Records describe Flow properties that serve as
Flow Keys of the reported Flow. This Options Template is defined in
Section 4.4 of [RFC7011] and SHOULD be used by Mediators for export
as defined there.
When an Intermediate Process exports Data Records containing
different Flow Keys from those received from the Original Exporter,
and the Original Exporter sent a Flow Keys Options record to the
IPFIX Mediator, the IPFIX Mediator MUST export a Flow Keys Options
record defining the new set of Flow Keys.
10.3. intermediateProcessId Information Element
Name: intermediateProcessId
Description: An identifier of an Intermediate Process that is
unique per IPFIX Device. Typically, this Information Element is
used for limiting the scope of other Information Elements. Note
that process identifiers may be assigned dynamically; that is, an
Intermediate Process may be restarted with a different ID.
Claise, et al. Standards Track [Page 26]
RFC 7119 IPFIX MED-PROTO February 2014
Data Type: unsigned32
Data Type Semantics: identifier
ElementId: 406
10.4. ignoredDataRecordTotalCount Information Element
Name: ignoredDataRecordTotalCount
Description: The total number of received Data Records that the
Intermediate Process did not process since the (re-)initialization
of the Intermediate Process; includes only Data Records not
examined or otherwise handled by the Intermediate Process due to
resource constraints, not Data Records that were examined or
otherwise handled by the Intermediate Process but those that
merely do not contribute to any exported Data Record due to the
operations performed by the Intermediate Process.
Data Type: unsigned64
Data Type Semantics: totalCounter
ElementId: 407
11. Operations and Management Considerations
In general, using IPFIX Mediators to combine information from
multiple Original Exporters requires a consistent configuration of
the Metering Processes behind these Original Exporters. The details
of this consistency are specific to each Intermediate Process.
Consistency of configuration should be verified out of band, with the
MIB modules ([RFC6615] and [RFC6727]) or with the Configuration Data
Model for IPFIX and PSAMP [RFC6728].
From an operational perspective, this specification provides all the
information required to set up IPFIX Mediators and Collectors behind
IPFIX Mediators. While configuring the IPFIX Mediators, care must be
taken to include all the relevant information so that the Collectors
deduce the Data Records precise semantic. This is covered by the
Template Mapping specifications in Section 4.1. Also, caution must
be taken that if something is not carefully configured in the
processing chain, this can lead to the wrong interpretation of
collected IPFIX data, and the associated applications can produce
results that are not operationally meaningful.
Claise, et al. Standards Track [Page 27]
RFC 7119 IPFIX MED-PROTO February 2014
12. Security Considerations
As they act as both IPFIX Collecting Processes and Exporting
Processes, the Security Considerations for the IPFIX Protocol
[RFC7011] also apply to IPFIX Mediators. The Security Considerations
for IPFIX Files [RFC5655] also apply to IPFIX Mediators that write
IPFIX Files or use them for internal storage. However, there are a
few specific considerations that IPFIX Mediator implementations must
also take into account.
By design, IPFIX Mediators are "men in the middle": they intercede in
the communication between an Original Exporter (or another upstream
IPFIX Mediator) and a downstream Collecting Process. This has two
important implications for the level of confidentiality provided
across an IPFIX Mediator and the ability to protect data integrity
and Original Exporter authenticity across an IPFIX Mediator. These
are addressed in more detail in the Security Considerations for IPFIX
Mediators in [RFC6183].
Note that while IPFIX Mediators can use the exporterCertificate and
collectorCertificate Information Elements defined in [RFC5655] as
described in Section 9.3 of [RFC6183] to export information about
X.509 identities in upstream TLS-protected Transport Sessions, this
mechanism cannot be used to provide true end-to-end assertions about
a chain of IPFIX Mediators: any IPFIX Mediator in the chain can
simply falsify the information about upstream Transport Sessions. In
situations where information about the chain of mediation is
important, it must be determined out of band. Note as well that an
Exporting Process has no in-band way to determine whether or not a
given Collecting Process will act as a Mediator. Trust placed in
Collecting Processes is absolute, so care should be taken when
exporting IPFIX Messages between Exporting Processes and Collecting
Processes controlled by different entities.
13. IANA Considerations
This document specifies new IPFIX Information Elements,
originalExporterIPv4Address in Section 5.1,
originalExporterIPv6Address in Section 5.2,
originalObservationDomainId in Section 6.1, intermediateProcessId in
Section 10.3, and ignoredDataRecordTotalCount in Section 10.4, which
have been added to the IPFIX Information Element registry
[IANA-IPFIX].
Claise, et al. Standards Track [Page 28]
RFC 7119 IPFIX MED-PROTO February 2014
14. Acknowledgments
We would like to thank the IPFIX contributors, specifically Paul
Aitken (THE ultimate IPFIX document reviewer) and Andrew Feren for
their thorough reviews; Nevil Brownlee and Juergen Quittek for
shepherding this document and chairing the IPFIX Working Group; and
to Rahul Patel, Meral Shirazipour, and Juergen Schoenwaelder for
their feedback and comments. This work is materially supported by
the European Union Seventh Framework Programme under grant agreements
257315 (DEMONS) and 318627 (mPlane).
15. References
15.1. Normative References
[RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768,
August 1980.
[RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC
793, September 1981.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P.
Conrad, "Stream Control Transmission Protocol (SCTP)
Partial Reliability Extension", RFC 3758, May 2004.
[RFC4960] Stewart, R., "Stream Control Transmission Protocol", RFC
4960, September 2007.
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an
IANA Considerations Section in RFCs", BCP 26, RFC 5226,
May 2008.
[RFC5655] Trammell, B., Boschi, E., Mark, L., Zseby, T., and A.
Wagner, "Specification of the IP Flow Information Export
(IPFIX) File Format", RFC 5655, October 2009.
[RFC6313] Claise, B., Dhandapani, G., Aitken, P., and S. Yates,
"Export of Structured Data in IP Flow Information Export
(IPFIX)", RFC 6313, July 2011.
[RFC6615] Dietz, T., Kobayashi, A., Claise, B., and G. Muenz,
"Definitions of Managed Objects for IP Flow Information
Export", RFC 6615, June 2012.
Claise, et al. Standards Track [Page 29]
RFC 7119 IPFIX MED-PROTO February 2014
[RFC6727] Dietz, T., Claise, B., and J. Quittek, "Definitions of
Managed Objects for Packet Sampling", RFC 6727, October
2012.
[RFC6728] Muenz, G., Claise, B., and P. Aitken, "Configuration Data
Model for the IP Flow Information Export (IPFIX) and
Packet Sampling (PSAMP) Protocols", RFC 6728, October
2012.
[RFC7011] Claise, B., Trammell, B., and P. Aitken, "Specification of
the IP Flow Information Export (IPFIX) Protocol for the
Exchange of Flow Information", STD 77, RFC 7011, September
2013.
[RFC7012] Claise, B. and B. Trammell, "Information Model for IP Flow
Information Export (IPFIX)", RFC 7012, September 2013.
[RFC7013] Trammell, B. and B. Claise, "Guidelines for Authors and
Reviewers of IP Flow Information Export (IPFIX)
Information Elements", BCP 184, RFC 7013, September 2013.
[RFC7014] D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow
Selection Techniques", RFC 7014, September 2013.
[RFC7015] Trammell, B., Wagner, A., and B. Claise, "Flow Aggregation
for the IP Flow Information Export (IPFIX) Protocol", RFC
7015, September 2013.
15.2. Informative References
[RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander,
"Requirements for IP Flow Information Export (IPFIX)", RFC
3917, October 2004.
[RFC3954] Claise, B., "Cisco Systems NetFlow Services Export Version
9", RFC 3954, October 2004.
[RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
"Architecture for IP Flow Information Export", RFC 5470,
March 2009.
[RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP
Flow Information Export (IPFIX) Applicability", RFC 5472,
March 2009.
[RFC5473] Boschi, E., Mark, L., and B. Claise, "Reducing Redundancy
in IP Flow Information Export (IPFIX) and Packet Sampling
(PSAMP) Reports", RFC 5473, March 2009.
Claise, et al. Standards Track [Page 30]
RFC 7119 IPFIX MED-PROTO February 2014
[RFC5476] Claise, B., Johnson, A., and J. Quittek, "Packet Sampling
(PSAMP) Protocol Specifications", RFC 5476, March 2009.
[RFC5610] Boschi, E., Trammell, B., Mark, L., and T. Zseby,
"Exporting Type Information for IP Flow Information Export
(IPFIX) Information Elements", RFC 5610, July 2009.
[RFC5982] Kobayashi, A. and B. Claise, "IP Flow Information Export
(IPFIX) Mediation: Problem Statement", RFC 5982, August
2010.
[RFC6183] Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi,
"IP Flow Information Export (IPFIX) Mediation: Framework",
RFC 6183, April 2011.
[RFC6235] Boschi, E. and B. Trammell, "IP Flow Anonymization
Support", RFC 6235, May 2011.
[NAT-LOGGING]
Sivakumar, S. and R. Penno, "IPFIX Information Elements
for logging NAT Events", Work in Progress, November 2013.
[IANA-IPFIX]
IANA, "IP Flow Information Export (IPFIX) Entities",
<http://www.iana.org/assignments/ipfix>.
[POSIX.1] IEEE, "IEEE Standard for Information Technology - Portable
Operating System Interface", IEEE 1003.1-2008, 2008.
Claise, et al. Standards Track [Page 31]
RFC 7119 IPFIX MED-PROTO February 2014
Authors' Addresses
Benoit Claise
Cisco Systems, Inc.
De Kleetlaan 6a b1
1831 Diegem
Belgium
Phone: +32 2 704 5622
EMail: bclaise@cisco.com
Atsushi Kobayashi
NTT Information Sharing Platform Laboratories
3-9-11 Midori-cho
Musashino-shi, Tokyo 180-8585
Japan
Phone: +81 422 59 3978
EMail: akoba@nttv6.net
Brian Trammell
Swiss Federal Institute of Technology Zurich
Gloriastrasse 35
8092 Zurich
Switzerland
Phone: +41 44 632 70 13
EMail: trammell@tik.ee.ethz.ch
Claise, et al. Standards Track [Page 32]