Internet Draft J. Quittek <draft-quittek-pcap-ext-00.txt> NEC Europe Ltd. Expires: May 2001 G. Carle GMD FOKUS 17 November 2000 Remote Packet Capture Extensions <draft-quittek-pcap-ext-00.txt> Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Abstract This memo suggests two extensions of the recommendations for remote packet capture given in [RemPCAP]. The extensions concern the suggested captured packet encapsulation header and the configuration of packet capturing devices. The captured packet encapsulation header is extended by flags indicating simple steps of pre-processing captured packet headers. Most indicated pre-processing actions are merging actions of headers with common properties. For configuration of packet capturing devices, a configuration record is suggested. Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 1]
Internet Draft PCAP extensions 17 November 2000 Table of content 1. Introduction 2. Packet Merging 2.1. Fragment Merge (FM) 2.2. TCP Socket merge (TS) 2.3. UDP Socket merge (US) 2.4. Peer Pair merge (PP) 2.5. Source Merge (SM) and Destination Merge (DM) 2.6. Source Peer merge (SP) Destination Peer merge (DP) 3. Packet (Pre-)Filtering 4. Merging Extensions and Extended Captured Packet Encapsulation Header 4.1. Actions bit field 4.2. Number of merged packets 4.3. Total length of all packets 4.4. Time Stamp (last) 4.5. Filter Identifier 5. Packet Capturing Configuration Record 5.1. General capturing configuration 5.2. Receivers of captured packet data 5.3. Filter specifications 6. Security Considerations 7. References 8. Authors' Addresses 1. Introduction The recommendations for remote packet capturing given in [RemPCAP] include a captured packet encapsulation header which can be used for transmitting captured packets or portions of those. Usually the portion of the packet that is transmitted contains the IP header, the transport header and parts of the application header. The captured packet encapsulation header was defined assuming that each packet portion will be transmitted separately. Considering the processing power of today's network interfaces, it appears to be appropriate to reflect in a captured packet encapsulation header also possible ways of packet header pre-processing by the capturing device. Particularly merging of packets and filtering of packets are tasks many network interfaces are capable of or - at least- prepared for. For example interfaces with integrated packet classifier or traffic shapers are equipped with sufficient capabilities to perform packet (pre-)filtering and merging of packet headers. The first extension suggested in this memo extends the captured packet encapsulation header by fields indicating what kind of merging (c.f. section 2) or Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 2]
Internet Draft PCAP extensions 17 November 2000 filtering (c.f. section 3) was applied at the packet capturing device. While these merging functions are suitable for activity detection, extended merging functions may collect additional aggregated information for merged packets, such as flow activity time, number of merged packets, data volume of merged packets. This requires the definition of a Flow idle timeout for identification of a last packet of a flow. An extended captured packet encapsulation header is defined for transmitting the additional information (c.f. section 4.). The second extension deals with configuring packet capturing devices (c.f. section 5). Certainly, there are ways of configuring them manually via command line interfaces or other means. However, an automated remote configuration of these devices appears to be desirable. This memo recommends a packet capturing configuration record for device configuration covering the configuration of packet merging and of packet filtering. 2. Packet Merging For many metering applications merging of packets belonging to the same 'flow' is of interest. (Various types of flow specification appear appropriate.) A package capturing device may already have the abilities to merge packets. These capabilities may vary among different devices. However, it is desirable to exploit the capabilities that are available. The kind of merge required may be different. The following table lists a set of merging actions that appear to be desirable for several applications using packet capturing. +----+------------------------+-------------------------------------+ | FM | Fragment Merge | merge all fragments of the same | | | | original IP packet | +----+------------------------+-------------------------------------+ | TS | TCP Socket merge | merge all packets belonging to the | | | | same TCP socket | +----+------------------------+-------------------------------------+ | US | UDP Socket merge | merge all packets belonging to the | | | | same UDP socket | +----+------------------------+-------------------------------------+ | PP | Peer Pair merge | merge all packets between the same | | | | pair of IP addresses | Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 3]
Internet Draft PCAP extensions 17 November 2000 +----+------------------------+-------------------------------------+ | SM | Source Merge | merge all packets with the same | | | | source IP address and port number | +----+------------------------+-------------------------------------+ | DM | Destination Merge | merge all packets with the same | | | | destination IP address and port | | | | number | +----+------------------------+-------------------------------------+ | SP | Source Peer merge | merge all packets with the same | | | | source IP address | +----+------------------------+-------------------------------------+ | DP | Destination Peer merge | merge all packets with the same | | | | destination IP address | +----+------------------------+-------------------------------------+ Table 1: Kinds of packet merging The suggested extension of the captured packet encapsulation header (described below) signals merging actions applied to packets by references to the actions of this table. Also the suggested packet capturing configuration record uses the listed actions for configuring packet capturing devices. The following subsections describe each merging action. 2.1. Fragment Merge (FM) IP packets may be fragmented while they are forwarded. For some application it is required to count only the number of originally sent 'complete' IP packets and to ignore fragmentation. Merging all IP fragments originally belonging to the same IP datagram is one of the merging actions that can be performed by packet capturing devices before they report about captured packets. After the merge, only the fragment with the lowest fragment offset is reported by the capturing device. 2.2. TCP Socket merge (TS) Many traffic measurement applications require information not on a per packet base, but on a per flow base. TCP Socket merge denotes the merging of all packets that share the same source IP address, source TCP port, destination IP address and destination TCP port. TCP Socket merging can be uni-directional or bi-directional. In uni-directional mode packets for each direction are merged separately, in bi- directional mode all packets independent of the direction (source to destination or vice versa) are merged before the capturing device reports about them. In any case, the report contains only the first header captured. Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 4]
Internet Draft PCAP extensions 17 November 2000 2.3. UDP Socket merge (US) UDP socket merge is defined analogously to TCP socket merge. The packets to be merged must have common source and destination addresses and port numbers. Packets are merged either in uni- directional mode or in bi-directional mode. 2.4. Peer Pair merge (PP) Peer pair merge denotes the merging of all packets sent between a pair of IP addresses. Packets are merged either in uni-directional mode or in bi-directional mode. 2.5. Source Merge (SM) and Destination Merge (DM) These actions merge all packets that share the IP address, the transport type, and the port number for the source or for the destination, respectively. Only the first captured packet is reported. 2.6. Source Peer merge (SP) Destination Peer merge (DP) These actions are similar to source merge and destination merge, respectively. The only difference is that only the IP addresses are considered when packets are merged. Differences in transport type or protocol numbers are ignored. 3. Packet (Pre-)Filtering Many applications requiring packet capturing need only a subset of all packets observed by the capturing device to be captured. For these applications it is desirable to exploit filtering functionality available at the capturing device in order to reduce the amount of data to be exchanged and to increase scalability. A simple packet filter may reduce the number of packets to be reported significantly, before a more advanced filter receives and processes them. Filters can be specified in several ways. For example the the Meter MIB [RFC2720], the RMON MIB [RFC2819], and the DiffServ MIB [DS-MIB] use different filter specifications for IP packets. Some of these specifications may be very long and complex. Because of this, it does not appear to be appropriate for a packet capturing device to add the applied filter specification to the report of a captured packet. However, reporting that a filter was applied when a packet was captured seems to be an important information that should not be omitted. And for specifying what filter has been applied, a filter identifier is a solution, that might provide sufficient information Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 5]
Internet Draft PCAP extensions 17 November 2000 in many cases. It should be noted that the filtering mechanisms available at a packet capturing device are expected to be rather simple and not as powerful as for example the mechanism offered by the Meter MIB. 4. Merging Extensions and Extended Captured Packet Encapsulation Header The extended captured packet encapsulation header contains seven additional fields compared to the header suggested in [RemPCAP]. A device reporting captured packets signals applied aggregation actions and filtering actions by setting bits in the first new field. The further fields contain information on details of the applied actions. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ifIndex | Interface Type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Status | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time Stamp (sec) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time Stamp (nsec) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |F B T U P S D S D F F| Number of merged packets | |M D S S P M M P P A I| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Total length of all packets | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time Stamp (last) (sec) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time Stamp (last) (nsec) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Filter Identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | Captured Packet Data | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Extended Captured Packet Encapsulation Header Format Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 6]
Internet Draft PCAP extensions 17 November 2000 4.1. Actions bit field The first new field is a 16 bit field with bits specifying applied merge actions and an applied filter action. If fragment merge was applied, this is indicated by a set FM bit. A set Bi-Directional (BD) bit indicates the bi-directional mode for TS, US, and PP. If this bit is not set, uni-directional mode is indicated. in combination with other merge actions the BD bit has not meaning. The next seven bits indicate the merge action applied. Most of them exclude each other. The only combinations of two bits that may be set at the same time are TS and US, SM and DM, SP and DP. Combinations of more than two bits may not be set. The next five bits are unused. The remaining two bits indicate further properties of the reported flow. The the Filter Applied (FA) bit indicates that this packet was captured after applying a filter. The Flow Idle timeout (FI) bit indicates that a timeout was observed for the reported flow, i.e. no packet of this flow has been observed for a period of time longer than a given timeout. 4.2. Number of merged packets This 16 bit field indicates the number of packets merged into one. The interpretation of the field depends on the applied merge options: - If the FM bit is set and none of the bits TS, US, PP, BD, SM, DM, SP, and DP is set, then this number indicates the number of fragments that have been merged into one. - If one of the bits TS, US, PP, BD, SM, DM, SP, and DP is set in combination with the FM bit, then the group of all fragments with the same identification field in the IP header is counted as one packet when computing the number of merged packets. - If one of the bits TS, US, PP, BD, SM, DM, SP, and DP is set and the FM bit is not set, then all fragments with the same identification field in the IP header are counted individually for the number of merged packets. 4.3. Total length of all packets This 32 bit field indicates the total length of all packets merged into one. Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 7]
Internet Draft PCAP extensions 17 November 2000 4.4. Time Stamp (last) These two 32 bit fields are defined analogously to the Time Stamp fields in the original captured packet encapsulation header. With the extension, the original Time Stamp fields indicate the time, the first of the merged packets was observed. The new Time Stamp (last) fields indicate the time, the last merged packet was observed. 4.5. Filter Identifier The last field with a length of 32 bit indicates the filter that was applied when the reported packet was captured. For reasons described in section 3. only an identifier is used to specify the filter. The interpretation of this identifier depends on the capturing device and on the configuration of that device. The identifier is only valid, if the FA bit is set. 5. Packet Capturing Configuration Record Packet capturing devices can be configured using the packet capturing configuration record. This record allows to specify which functions for packet merging, packet filtering or merging extensions are applied for specific flows. In the latter case, it is also possible to specify flow idle timeouts and the captured size of a packet. (in cases where the default flow idle timeout or captured size appears not to be appropriate). With the Packet Capturing Configuration Record it can also be specified how captured data is to be transmitted to receivers. So far, there is only one transmission method defined using UDP Encapsulation (UE). Multiple receivers are supported. The packet capturing configuration record contains three sections, of which the second and the third are optional. The first mandatory section specifies a general capturing configuration applied by default to all observed packets. The second section specifies a list of receivers of captured packet data. The third section specifies a list of filters applied to packets before they get captured. Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 8]
Internet Draft PCAP extensions 17 November 2000 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |F B T U P S D S D U|#of re-| Captured size of packet | |M D S S P M M P P E|ceivers| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Number of filters specified | Flow idle timeout | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP address of receiver of captured packet data 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | receiver's port number 1 | padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP address of receiver of captured packet data 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | . . . | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Filter Identifier 1 | Transport | | | type 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source IP address 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination IP address 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source port 1 | Destination port 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source address| Dest. address | Source port | Dest. port | | mask 1 mask 1 mask 1 mask 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Flow idle timeout 1 | Captured size of packet 1 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Filter Identifier 2 | Transport | | | type 2 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | . . . | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Packet Capturing Configuration Record Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 9]
Internet Draft PCAP extensions 17 November 2000 5.1. General capturing configuration The first field of the packet capturing configuration record contains twelve bits. The first nine bits (FM, BD, TS, US, PP, SM, DM, SP, and DP) specify merging actions the capturing device is requested to perform on all observed packets. The next two bits are unused, so far. The last bit of this field (UE) indicates (if set) that UDP encapsulation is to be used for transmitting captured packet data. A UDP packet used for this purpose must contain a sequence of encapsulated captured packets according to the extended captured packet encapsulation headers format. The next 4 bit field is to be interpreted as a positive integer number specifying the number of receivers to which reports on captured packet information are to be sent. The receivers are specified in the second section. If the receiver(s) are already specified by other means, the number should be set to zero. The next 16 bit field specifies the minimal (positive) number of bytes to be captured of a packet. this captured size applies to packet data not belonging to the transport layer or lower layers. So, if the captured size is set to zero, still the IP and transport header (TCP, UDP, ICMP, or IGMP) are reported. If the capturing device is not able to interpret transport layer information, a conservative estimation of the captured size has to be made which will ensure, that the requested parts of the packets are always captured. The next 16 bit field specifies the (positive) number of filters specified in the third section of the capturing configuration record. This number may be zero, specifying, that all observed packets are to be reported. If it is non-zero, then the capturing device is requested to report only on packets which have passed one of the filters. The last field of the first section of the packet capturing configuration record specifies the default flow idle timeout of filtered packets. This 16 bit is to be interpreted an a non-negative number of seconds. If for any of the specified filters no packets has been observed for this number of seconds after the last observed packet, then the flow specified by this filter is assumed to be idle and a report on this flow is to be sent. The default timeout may be overwritten by the individual filter specifications in the third section of the packet capturing configuration record. Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 10]
Internet Draft PCAP extensions 17 November 2000 5.2. Receivers of captured packet data The second section of the packet capturing configuration record contains an optionally empty list of receivers of captured packet data. Each item of the list has a size of 64 bit with the first 32 bit specifying the receivers IPv4 address and the following 16 bit specifying the receivers port number. The remaining 16 bits are unused. 5.3. Filter specifications The third section of the packet capturing configuration record contains an optionally empty list of packet filters. An item of this list specifies a filter and sets two options of processing filtered data. The first 24 bit field serves as filter identifier. The following nine fields identify the packets to be filtered. The fields are - Transport type (8 bit) - Source IP address - Destination IP address - Source port number - Destination port number - Source address mask - Destination address mask - Source port number mask - Destination port number mask The masks for IP addresses and port numbers specify how many of the most relevant bits of the address or mask must be matched by a packet in order to pass the filter. With an address mask value of 0, any address or port number matches; with a mask value of 32, an IP address must be matched exactly. The transport type specifies a value to be matched by the protocol field in the IPv4 header. If it is of value 0, then any transport type is matched. There are two additional fields in the filter specification requesting special processing of packets passing the filter. The flow idle timeout overwrites the default timeout specified in the first section of the packet capturing configuration record. Similarly, the captured size of packets overwrites the corresponding default value for the particular filter. Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 11]
Internet Draft PCAP extensions 17 November 2000 6. Security Considerations Remote packet capturing may cause security threats such as unauthorized access, modification or disclosure of remotely captured packets [RemPCAP]. Protecting privacy of captured data that is being transmitted from the remote capturing node to an authorized entity against listening third parties may be supported by protections through IPSec [RemPCAP]. More important is protecting privacy of captured data against unauthorized initiators of capturing instructions. Similar to restricting access to packet capturing methods at a local node to privileged users, initiation of remote packet capturing methods should be subject to policy control ("is the initiator authorized to perform the remote packet capturing capability"). For certain scenarios such as intra-domain authorization control, SNMPv3 appears adequate. For other scenarios such as third party measurement or accounting services involving inter-domain authorization control, a AAA protocol and AAA servers may be necessary. Simple authorization schemes may distinguish between two user classes (authorized or non-authorized). Complex authorization schemes may require that remote packet capturing requests are subject to policy- based authorization control. Complex authorization control may include fine grain authorization policies, such as which user is allowed to capture which part of the packet (which attributes) depending on flow specifications. For example, authorization policies may specify that the user class 'administrator' may capture remotely within their own domain without restrictions, while the same user class may remotely capture at other domains only packets with source or destination IP addresses of specific networks (typically those belonging to their domain). Authorization policies may also specify that unprivileged users may capture remotely only packets with their source or destination address. Another security issue is the transmission of captured packets. Since they might contain the complete information as the original packet, the security level applied to this transmission must match the security level of the original transmission. Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 12]
Internet Draft PCAP extensions 17 November 2000 7. References [RemPCAP] C. Bullard, "Remote Packet Capture", work in progress, draft-bullard-pcap-00.txt, November 2000. [RFC2720] N. Brownlee, "Traffic Flow Measurement: Meter MIB", RFC 2720, October 1999. [RFC2598] S. Waldbusser, "Remote Network Monitoring Management Information Base", RFC 2819, May 2000. [DS-MIB] F. Baker, K. Chan, A. Smith, "Management Information Base for the Differentiated Services Architecture", work in progress, draft-ietf-diffserv-mib-05.txt, November 2000. 8. Authors' Addresses Juergen Quittek NEC Europe Ltd. C&C Research Laboratories Adenauerplatz 6 69115 Heidelberg Germany Phone: +49 6221 90511-15 EMail: quittek@ccrle.nec.de Georg Carle GMD FOKUS Kaiserin-Augusta-Allee 31 10589 Berlin Germany Phone: +49 30 3463 7149 Email: carle@fokus.gmd.de Quittek and Carle draft-quittek-pcap-ext-00.txt [Page 13]