PSAMP working group Internet Draft EDITOR: B. Claise draft-ietf-psamp-protocol-00.txt Cisco Systems Expires: April 2003 Otcober 2003 Packet Sampling (PSAMP) Protocol Specifications Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsolete by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This document specifies the export of packet information from a PSAMP exporting process to a PSAMP colleting process. For export of packet information the IP Flow Information eXport (IPFIX) protocol is used. It is shown that The IPFIX protocol is well suited for this purpose, because the IPFIX architecture matches the PSAMP architecture very well and the means provided by the IPFIX protocol are sufficient. The document specifies in detail how the IPFIX protocol is used for PSAMP export of packet information. Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Claise, et. al Standard Track [Page 1]
PSAMP Protocol Specifications October 2003 Table of Contents 1. Open Issues.................................................2 2. Introduction................................................2 3. Terminology.................................................3 4. Relationship between PSAMP and IPFIX........................3 4.1 IPFIX Overview............................................3 4.2 IPFIX and PSAMP Differences and Similarities..............4 4.2.1 Export Point of View....................................4 4.2.2 Information Model Point of View.........................4 5. Using IPFIX for PSAMP.......................................5 5.1 High Level View of the Integration........................5 5.2 Partial or Entire IPFIX Protocol Specifications Support...6 6. PSAMP Requirements versus the IPFIX Solution................6 6.1 IPFIX Solution for the PSAMP Requirements.................7 7. Low Level View of the Integration...........................9 7.1 Sampling Case, PSAMP Base Level of Functionality..........9 7.1.1 Example................................................10 7.2 Sampling Case............................................10 7.2.1 Example................................................11 7.3 Filtering Case...........................................11 7.3.1 Example................................................11 8. Security Considerations....................................12 9. References.................................................12 10. Acknowledgments...........................................12 11. AuthorsÆ Addresses........................................13 1. Open Issues This section covers the open issues, still to be resolved/updated in this draft: - For section 6 "PSAMP requirements versus the IPFIX solution", check if there are any other requirements in the [PSAMP-FRAMEWORK]. 2. Introduction The packet sampling (PSAMP) Working Group and the IP flow information export (IPFIX) Working Group both aim at standardizing technology for observing traffic from network devices and for exporting some part of the observation. Also, both Working Groups consider packet sampling as a component of their technology. While for the IPFIX Working Group packet sampling is just one out of many components considered, it is the focus of the PSAMP Working Group. The PSAMP Working Group has agreed to use the IPFIX reporting protocol if it's suitable for the PSAMP requirements. Therefore, a Claise, et. al Standard Track [Page 2]
PSAMP Protocol Specifications October 2003 detailed analysis on the IPFIX protocol needs to be done and if IPFIX is not suitable, then the reason should be stated exactly. This document evaluates if the IPFIX protocol specifications could fit the export format requirements for PSAMP device, how PSAMP could use the IPFIX protocol, and whether the part of or the full IPFIX protocol specifications are actually required. As we will conclude that the IPFIX protocol is suitable as export protocol for PSAMP, this document finally specifies in details how to use IPFIX. 3. Terminology To be copied in from [PSAMP-FRAMEWORK4]. 4. Relationship between PSAMP and IPFIX 4.1 IPFIX Overview The output of the IPFIX working group relevant for this draft, is structured into three documents: - IP flow information architecture [IPFIX-ARCH] - IPFIX Protocol Specifications [IPFIX-PROTO] - IP flow information export information model [IPFIX-INFO] This table will help summarizing the IPFIX protocol specifications [IPFIX-PROTO]. FlowSet Template Record Data Record +----------------------------------------------------------------+ | | | Flow Data Record(s) | | Data FlowSet | / | or | | | | Options Data Record(s) | +----------------------------------------------------------------+ | Template FlowSet | Template Record(s) | / | +----------------------------------------------------------------+ | Options Template | Options Template | / | | FlowSet | Record(s) | | +----------------------------------------------------------------+ A Data FlowSet is composed of an Options Data Record(s) or Flow Data Record(s); no Template Record is included. The Flow Data Record is linked to a Template Record, and the Options Data Record is linked to an Options Template Record. Claise, et. al Standard Track [Page 3]
PSAMP Protocol Specifications October 2003 A Template FlowSet is composed of Template Record(s); no Flow or Options Data Record is included. An Options Template FlowSet is composed of Options Template Record(s); no Flow or Options Data Record is included. The Options Template Record (and its corresponding Options Data Record) is used to supply information about the metering process configuration or specific data, rather than supplying information about IP flows. The Options Data Records are sent on a regular basis, but not with every Flow Data Record. 4.2 IPFIX and PSAMP Differences and Similarities IPFIX achieves data reduction by aggregating per-packet IP layer information into flow records. IPFIX produces and exports flow records containing information per flow. This information is created based on the observation of a potentially large number of packets. In contrast, PSAMP achieves data reduction by reducing the packet population via sampling. PSAMP generates and exports information per packet. For more details please see the [PSAMP-FRAMEWORK] and [PSAMP-SAMPLE-TECH]. 4.2.1 Export Point of View From a pure export point of view, IPFIX will not distinguish a flow record composed of several packets aggregated together, from a flow record composed of a single packet. As a conclusion, the PSAMP export can be seen as special IPFIX flow record containing information about a single packet. PSAMP doesn't have the notion of flow. But in order to avoid any duplication in the terminology and as a consequence a redefinition of the IPFIX protocol specifications, the IPFIX terminology [IPFIX- PROTO] is kept unchanged, even if some obvious pointers to the notion of flow is made. For example: Flow Data Record, FlowSet, etc... 4.2.2 Information Model Point of View On one hand, the IPFIX export probably contains data types like source IP address, destination IP address, ToS, etc. Refer to Claise, et. al Standard Track [Page 4]
PSAMP Protocol Specifications October 2003 [IPFIX-INFO] for more details. On the other hand, the PSAMP export contains only the packet fragment in the base level of functionality. Refer to [PSAMP-INFO] for more details. As the templates are flexible, IPFIX will not distinguish from a export point of view a flow record composed of several data types, from a flow record composed of just a few data types (for example: the packet fragment and the selector ID). The information model data types exported in an IPFIX device and a PSAMP device are not completely different but most of the time overlapping. Note that, according to [PSAMP-FRAMEWORK] section 5.2 "Recommended Contents for Packet Reports", the PSAMP reporting process SHOULD also report fields relating to the protocols used in the packets, to the packet treatment and to the selection state associated with the packet. Thus the PSAMP reporting process will not limit itself to the export the data types defined in [PSAMP-INFO], and can benefit from the data types already defined in [IPFIX-INFO]. From the IPFIX point of view, the new PSAMP information model will augment the data types that could be exported; for example, the hash value, the selector ID or the packet-sampled. If a IPFIX metering process create some flow records by sampling some packets, and if both the IPFIX and PSAMP specifications are implemented on the device, the IPFIX flow records could be augmented with extra data types like the selector ID, the selector ID parameters, etc. As the PSAMP information model is basically an extension to the IPFIX information model, a formal process must be in place for the addition of data types. The draft draft-bryant-ipfix-vendor-ie-00.tx (not yet out) discusses some possibilities. 5. Using IPFIX for PSAMP 5.1 High Level View of the Integration The Template Record in the Template FlowSet is used to describe the different PSAMP data types that will be exported to the Collector. The Collector decodes the Template FlowSet and knows which data types to expect when it receives the Flow Data Records in the Data FlowSet, i.e. the PSAMP Packet Reports. Typically, in the base level Claise, et. al Standard Track [Page 5]
PSAMP Protocol Specifications October 2003 of the PSAMP functionality, the Template FlowSet will contain the input sequence number, the packet fragment (some number of contiguous bytes from the start of the packet) and the selector ID. The Options Template Record in the Options Template FlowSet is used to describe the different PSAMP data types that concern the metering process itself: sampling and/or filtering functions, plus the associated parameters. The Collector decodes the Options Template FlowSet and knows which data types to expect when it receives the Options Data Records in the Data FlowSet, i.e. the PSAMP Report Interpretation. Typically, the Options Template would contain the Selector ID, the sampling or filtering functions, and the sampling or filtering associated parameters. 5.2 Partial or Entire IPFIX Protocol Specifications Support The "High level view of the integration" section 5.1 concludes that PSAMP requires all the different possibilities of the IPFIX protocol specifications [IPFIX-PROTO]. That is the 3 types of FlowSet (Data FlowSet, Template FlowSet and Options Templates FlowSet), the 2 types of Templates Records (Template Record and Options Template Record), and the 2 types of Data Record (Flow Data Record, Options Data Record), as described again in the table below. FlowSet Template Record Data Record +----------------------------------------------------------------+ | | | Flow Data Record(s) | | Data FlowSet | / | or | | | | Options Data Record(s) | +----------------------------------------------------------------+ | Template FlowSet | Template Record(s) | / | +----------------------------------------------------------------+ | Options Template | Options Template | / | | FlowSet | Record(s) | | +----------------------------------------------------------------+ As a consequence, PSAMP can't rely on a subset of the IPFIX protocol specifications are described in [IPFIX-PROTO]. The entire IPFIX protocol specifications MUST be implemented for the PSAMP export. 6. PSAMP Requirements versus the IPFIX Solution Claise, et. al Standard Track [Page 6]
PSAMP Protocol Specifications October 2003 [PSAMP-FRAMEWORK] describes some requirements that affect directly the export protocol. Refer to the following sections: section 3.2 "Reporting Process Requirements" section 3.3 "Exporting Process Requirements" section 5 "Reporting Process" [PSAMP-FRAMEWORK] also describes in the section 3.1 one requirement that, if not directly related to the export protocol, will put some constraints on it: Selection Process Requirements: - Parallel Measurements: multiple independent measurement processes at the same entity." [PSAMP-FRAMEWORK] finally describes in the section 5 some requirements regarding the reporting process. This series of requirements specifies the different data types that MUST and SHOULD reported to the collector. Nevertheless IPFIX, being a generic export protocol, can export any data types as long as there are described in the information model. So these requirements are mainly targeted for the [PSAMP-INFO] document. 6.1 IPFIX Solution for the PSAMP Requirements Let's address the PSAMP requirements one by one. * Parallel Measurements: multiple independent measurement processes at the same entity. Refer to [PSAMP-FRAMEWORK] section 3.1 "Selection Process Requirements". This requirement is addressed by exporting the Selector ID data type in every packet report, so part of every Flow Data Records. Note that without this requirement, exporting the Scope [IPFIX-PROTO] part of every single packet report could have been sufficient. * Transparency: allow transparent interpretation of measurements as communicated by PSAMP reporting, without any need to obtain additional information concerning the observed packet stream. Refer to [PSAMP-FRAMEWORK] section 3.2 "Reporting Process Requirements". This requirement is addressed by exporting the Selector ID in every Flow Data Records (packet report) and exporting the associated SAMPLING_ALGORITHM and SAMPLING PARAMETERS in the Options Data Record Claise, et. al Standard Track [Page 7]
PSAMP Protocol Specifications October 2003 (packet interpretation). So the all the metering process parameters are linked to the Flow Data Records. * Robustness to Information Loss: allow robust interpretation of measurements with respect to reports missing due to data loss, e.g. in transport, or within the measurement, reporting or exporting processes. Inclusion in reporting of information that enables the accuracy of measurements to be determined. Refer to [PSAMP-FRAMEWORK] section 3.2 "Reporting Process Requirements". An Options Templates MUST be sent on regular basis. This Options Template contains for example the total number of packet report exported from the PSAMP device, the total number of packet observed, etc... Thus the Collector can compare the number of packet report received per selector ID with the number actually metered and/or sent. In case of discrepancy, a new sampling rate could be computed. * Faithfulness: all reported quantities that relate to the packet treatment MUST reflect the router state and configuration encountered by the packet at the time it is received by the measurement process. Refer to [PSAMP-FRAMEWORK] section 3.2 "Reporting Process Requirements". This requirement doesn't concern the export protocol itself but the metering process, even if described in the "Reporting Process Requirements" section. * Privacy: selection of the content of packet reports will be cognizant of privacy and anonymity issues while being responsive to the needs of measurement applications, and in accordance with RFC 2804. Full packet capture of arbitrary packet streams is explicitly out of scope. Refer to [PSAMP-FRAMEWORK] section 3.2 "Reporting Process Requirements". This requirement doesn't concern the export protocol itself, even if described in the "Reporting Process Requirements" section. * Timeliness: reports on selected packets MUST be made available to the collector quickly enough to support near real time applications. Specifically, any report on a packet MUST be dispatched within 1 second of the time of receipt of the packet by the measurement process. Refer to [PSAMP-FRAMEWORK] section 3.3 "Export Process Requirements". Claise, et. al Standard Track [Page 8]
PSAMP Protocol Specifications October 2003 The IPFIX protocol specifications [IPFIX-PROTO] describe an inactivity timeout for the flow expiration. This inactivity timeout is configurable, with a minimum value of 0 for immediate expiration. Note that this minimum value of 0 will force every single Flow Data Record to contain information about a single packet and not an aggregation of packets. * Congestion Avoidance: export of a report stream across a network MUST be congestion avoiding in compliance with RFC 2914. Refer to [PSAMP-FRAMEWORK] section 3.3 "Export Process Requirements". IPFIX, by its charter, MUST also respect this requirement. * Secure Export: - confidentiality: the option to encrypt exported data MUST be provided. - integrity: alterations in transit to exported data MUST be detectable at the collector - authenticity: authenticity of exported data MUST be verifiable by the collector in order to detect forged data. The motivation here is the same as for security in IPFIX export. Refer to [PSAMP-FRAMEWORK] section 3.3 "Export Process Requirements". 7. Low Level View of the Integration 7.1 Sampling Case, PSAMP Base Level of Functionality EDITORÆS NOTE: LET'S ASSUME THAT THE [PSAMP-INFO] DEFINES THE FOLLOWING DATA TYPES SEQUENCE-NUMBER: the input sequence number, PACKET-SAMPLE: some number of contiguous bytes from the start of the packet SELECTOR-ID: SAMPLING-ALGORITHM: SAMPLING-PARAMETER1, SAMPLING-PARAMETERS2, ETC... As described in the section 5.1 "Mandatory Contents of Packet Reports" of [PSAMP-FRAMEWORK], the packet reports must contain: - the input sequence number(s), denoted the SEQUENCE-NUMBER in [PSAMP-INFO] Claise, et. al Standard Track [Page 9]
PSAMP Protocol Specifications October 2003 - some number of contiguous bytes from the start of the packet, denoted the PACKET-SAMPLE in [PSAMP-INFO]. Thus the Template FlowSet defines a Template Record composed of SEQUENCE-NUMBER, PACKET-SAMPLE and SELECTOR-ID. The report interpretation must contain: - the sampling algorithm, denoted SAMPLING-ALGORITHM in [PSAMP-INFO] - the sampling parameters denoted SAMPLING-PARAMETER1, SAMPLING- PARAMETER2, etc... in [PSAMP-INFO] The Options Template FlowSet defines a Options Template Record composed of SELECTOR-ID, SAMPLING-ALGORITHM, SAMPLING-PARAMETERS. Finally the Data FlowSet is used to export the Flow Data Record(s) containing the real values of SEQUENCE-NUMBER, PACKET-SAMPLE and SELECTOR-ID. The Data FlowSet is also used to export the Options Data Record(s) containing the real values of SELECTOR-ID, SAMPLING-ALGORITHM, SAMPLING-PARAMETERS. By means of the SELECTOR-ID, the Collector can link any Flow Data Record to the corresponding Options Data Record. That is, any Flow Data Record to the metering process funtion and parameters. 7.1.1 Example EDITORÆS THIS MUST BE A FULL EXAMPLE LIKE IN SECTION 13 OF [IPFIX- PROTO]. THE [PSAMP-INFO] MUST BE FIRST PUBLISHED. 7.2 Sampling Case The PSAMP reporting process SHOULD also report fields relating to the protocols used in the packets, to the packet treatment and to the selection state associated with the packet, as specified in [PSAMP-FRAMEWORK] section 5.2 "Recommended Contents for Packet Reports". Let's take the same example as in the section 7.1, but let's add the export of the destination BGP Autonomous System (AS) [1771] and of the input interface The packet reports MUST contain: - the input sequence number(s), denoted the SEQUENCE-NUMBER in [PSAMP-INFO] Claise, et. al Standard Track [Page 10]
PSAMP Protocol Specifications October 2003 - some number of contiguous bytes from the start of the packet, denoted the PACKET-SAMPLE in [PSAMP-INFO] - the destination BGP AS , denoted destinationAS in [IPFIX-INFO] - the input interface, denoted ingressPort in [IPFIX-INFO] Thus the Template FlowSet defines a Template Record composed of SEQUENCE-NUMBER, PACKET-SAMPLE and SELECTOR-ID, destinationAS and ingressPort. The report interpretation will remain unchanged and must contain: - the sampling algorithm, denoted SAMPLING-ALGORITHM in [PSAMP-INFO] - the sampling parameters denoted SAMPLING-PARAMETER1, SAMPLING- PARAMETER2, etc... in [PSAMP-INFO] The Options Template FlowSet is used to define this template composed of SELECTOR-ID, SAMPLING-ALGORITHM, SAMPLING-PARAMETERS. Finally Data FlowSet is used to export the Flow Data Record(s) containing the real values of SEQUENCE-NUMBER, PACKET-SAMPLE and SELECTOR-ID, destinationAS and ingressPort. The Data FlowSet is also used to export the Options Data Record(s) containing the real values of SELECTOR-ID, SAMPLING-ALGORITHM, SAMPLING-PARAMETERS. As a consequence, the collector can link any Flow Data Record to the sampling algorithm and sampling parameters, by means of the SELECTOR-ID value. 7.2.1 Example EDITORÆS NOTE: THIS MUST BE A FULL EXAMPLE LIKE IN SECTION 13 OF [IPFIX-PROTO]. THE [PSAMP-INFO] MUST BE FIRST PUBLISHED. 7.3 Filtering Case EDITORÆS NOTE: ACTUALLY THE EXAMPLE WILL BE QUITE SIMILAR TO 7.1 AND 7.2 BUT WILL DEPEND A LOT ON HOW WE WILL DEFINE THE FILTERING IN [IPFIX-INFO]. 7.3.1 Example EDITORÆS NOTE: THIS MUST BE A FULL EXAMPLE LIKE IN SECTION 13 OF [IPFIX-PROTO]. THE [PSAMP-INFO] MUST BE FIRST PUBLISHED. Claise, et. al Standard Track [Page 11]
PSAMP Protocol Specifications October 2003 8. Security Considerations As IPFIX has been selected as the PSAMP export protocol and as the PSAMP security requirements are not stricter than the IPFIX security requirements, refer to the IPFIX export protocol [IPFIX-PROTO] for the security considerations. 9. References [PSAMP-FRAMEWORK] N. Duffield, D. Chiou, B. Claise, A. Greenber, M. Grossglauser "A Framework for Passive Packet Measurement" draft- ietf-psamp-framework-03.txt [PSAMP-FRAMEWORK4] N. Duffield, D. Chiou, B. Claise, A. Greenber, M. Grossglauser "A Framework for Passive Packet Measurement" draft- ietf-psamp-framework-04.txt [PSAMP-SAMPLE-TECH] T. Zseby, M. Molina, F. Raspall, N. Duffield "Sampling and Filtering Techniques for IP Packet Selection" draft- ietf-psamp-sample-tech-02.txt [PSAMP-MIB] T. Dietz, D. Romascanu, B. Claise "Definitions of Managed Objects for Packet Sampling" draft-ietf-psamp-mib-00.txt [PSAMP-INFO] T. Dietz, F. Dressler, G. Carle, B. Claise, "Information Model for Packet Sampling Exports", draft-ietf-psamp- info-00.txt [IPFIX-ARCH] G. Sadasivan, N. Brownlee "Architecture Model for IP Flow Information Export" draft-ietf-ipfix-arch-01.txt", June 2003 [IPFIX-INFO] P. Calato, J. Meyer, J. Quittek, "Information Model for IP Flow Information Export" draft-ietf-ipfix-info-01, August 2003 [IPFIX-PROTO] B. Claise, M. Fullmer, P. Calato, R. Penno, "IPFIX Protocol Specifications", draft-ietf-ipfix-protocol-00.txt, June 2003 [RFC1771] Y. Rekhter, T. Li, "A Border Gateway Protocol 4 (BGP- 4)", RFC 1771, March 1995. 10. Acknowledgments To be completed. Claise, et. al Standard Track [Page 12]
PSAMP Protocol Specifications October 2003 11. AuthorsÆ Addresses Benoit Claise Cisco Systems De Kleetlaan 6a b1 1831 Diegem Belgium Phone: +32 2 704 5622 E-mail: bclaise@cisco.com Juergen Quittek NEC Europe Ltd. Network Laboratories Kurfuersten-Anlage 36 69115 Heidelberg Germany Phone: +49 6221 90511-15 Email: quittek@ccrle.nec.de Claise, et. al Standard Track [Page 13]
