Internet Engineering Task Force                           Nevil Brownlee
INTERNET-DRAFT                                The University of Auckland

                                                            October 2000
                                                      Expires April 2001




                   RTFM: Implementing New Attributes


        <draft-brownlee-implementing-new-attributes-00.txt>


Status of this Memo

This document is an Internet-Draft and is in full conformance with all
provisions of Section 10 of RFC2026.

Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF), its areas, and its working groups.  Note that other groups
may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time.  It is inappropriate to use Internet-Drafts as reference material
or to cite them other than as "work in progress."

The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt

The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.

Abstract

RFC 2724 presented an analysis and taxonomy of RTFM attributes, and
proposed adding new attributes and new types of attribute.  Many of
those 'new' attributes, especially the distribution-valued attributes,
have now had several years implementation experience.  This document
proposes that these attributes be implemented, i.e that the RTFM
Architecture and Meter MIB RFCs be re-written to include them.


INTERNET-DRAFT       RTFM: Implementing New Attributes      October 2000


Contents

1  Introduction                                                        2

2  New Attribute Types                                                 2
   2.1 Attribute type 'distribution-valued'  . . . . . . . . . . . .   2
   2.2 Attribute type 'list-valued'  . . . . . . . . . . . . . . . .   4

3  New Attributes                                                      4
   3.1 Attributes from RFC 2724  . . . . . . . . . . . . . . . . . .   4
   3.2 Other possible new attributes . . . . . . . . . . . . . . . .   5

4  Extensions to the Meter MIB                                         6
   4.1 Implementing distribution- and list- values . . . . . . . . .   6
   4.2 'Flow size' filters . . . . . . . . . . . . . . . . . . . . .   6

5  Author's Address                                                    7


1  Introduction


RFC 2724 described a range of possibilities for extending the RTFM
Architecture, by adding new types of attribute value, and by proposing
new attributes which could be measured by an RTFM meter.  The new
attributes, particularly the distribution-values ones, have been
implemented and used for several years now, and many of them have proved
very useful.  In addition, mailing-list discussions have shown the need
for another attribute type, list-valued.

There is now a clear need to extend the RTFM architecture to include
these new attribute types, and the attributes which have proved useful.
RFCs covering extensions to the Architecture and to the Meter MIB are
therefore proposed as work items for a new RTFM Working Group.


2  New Attribute Types


2.1  Attribute type 'distribution-valued'


A distribution value is used as a way of gathering a sample distribution
for a specified metric.  The distribution is stored in an array of
'buckets,' with the values being stored as counters between a minimum
and maximum, with defined steps (either linear of logarithmic) for each
bucket, together with one further 'overflow' bucket for values above the
maximum.




Nevil Brownlee                                                  [Page 2]


INTERNET-DRAFT       RTFM: Implementing New Attributes      October 2000


The counters are never reset, hence one can compute the difference
between two consecutive distributions; such a 'difference distribution'
describes the metric's behaviour in the interval between successive
meter readings.

As well as it's buckets, a distribution value must also hold the values
of the parameters which specify the mapping of its values.  RFC 2724
proposed the following parameters, with sizes (in bytes) chosen so as to
fit within two six-byte fields (the minimum size for RTFM Address
Attributes).


 Size   Parameter        Description

   1    Transform        0 = null, 1 = linear, 2 = logarithmic
   1    Scale Factor     Power of 10 multiplier for limits and counts
   2    Lower Limit      Highest value for first bucket
   2    Upper Limit      Highest value for last bucket

   1    Buckets          Number of buckets.  Does not include
                              the 'overflow' bucket
   1    Parameter-1      } Parameter use depends
   2    Parameter-2      } on distribution-valued
   2    Parameter-3      } attribute


This set of parameters has worked very well in practice.  Note that each
bucket receives counts for values within a fixed range - there is no
attempt to adjust the steps dynamically.  This is a simple approach, but
one which has has proved to be very effective in use.

The complete set of parameters and values (including the overflow
bucket) can be retrived by a meter reader, allowing ananlysis software
to compute whatever statistics are required to describe the
distribution.  Within an SNMP PDU a distribution appears as a sequence
of integers.  The first eight integers are the distribution parameters,
these are followed by the bucket counters, and then the overflow
counter.

If a meter reader attempts to get a non-existent distribution value,
e.g. from a flow whose ruleset never saved a particular
distribution-valued attribute, the meter must return a null
distribution, i.e. one which has Transform = 0 and Buckets = 0.  Within
an SNMP PDU a null distribution will be a sequence containing a single
integer whose value is zero.

Note that a separate distribution must be built for each direction of a
flow; in this respect distribution-valued attributes (e.g. ToBitRate
and FromBitRate distribution) are similar to simple counts (e.g. ToPDUs
and FromPDUs).


Nevil Brownlee                                                  [Page 3]


INTERNET-DRAFT       RTFM: Implementing New Attributes      October 2000


2.2  Attribute type 'list-valued'


All of the RTFM address attributes have values in each packet for both
directions of their flow, e.g. every packet has values for both
SourcePeerAddress and DestPeerAddress.  The RTFM packet-matching
algorithm depends on having both of these, so that they can be swapped
when attempting a match in the reverse direction.

Metrics such as DSCodePoint and NetFlow's NextHop have only one value in
each packet, hence they can't be used in packet matching.  However, we
would like to determine what values of these metrics appear while the
flow is active, for each of its two directions.

To do this, we can build tables of (value, count) pairs, one for each
direction of the flow.  Each such table is a list-value, e.g.
ToDSCodePoint is a list of the CodePoint values seen for a flow's
Source->Destination packets, together with counts for each of those
values.  FromDSCodePoint is similar, but for a flow's
Destination->Source packets.

Within an SNMP PDU a list-value is a sequence of integers.  The first
integer gives the number of pairs in the list (zero if no values have
been counted); it is followed by the actual (value, count) pairs.


3  New Attributes


3.1  Attributes from RFC 2724


- Distributions(65)
    Has bits set to indicate which distribution-valued attributes
    are present in the flow.
*** Making SNMP GETs for non-existent distribution-valued attributes
    return null distributions removes the need for this attribute.

+ Packet sizes distributions:
    ToPacketSize(66)         size of PDUs in bytes (i.e. number
    FromPacketSize(67)         of bytes actually transmitted)

+ Inter-arrival time distributions for packets within a flow:
    ToInterarrivalTime(68)   microseconds between successive packets
    FromInterarrivalTime(69)   travelling in the same direction
      Inter-arrival times for packets travelling in the same
      direction are useful for determining variations in packet
      transit times.  Note that they don't require any packet
      matching, all the meter has to do is to remember the
      arrival time (in microseconds) for the last packet in
      each direction,

Nevil Brownlee                                                  [Page 4]


INTERNET-DRAFT       RTFM: Implementing New Attributes      October 2000



+ Short-term bit- and packet-rate distributions:
    ToBitRate(72)            short-term flow rate in bits per second
    FromBitRate(73)            Parameter 1 = rate interval in seconds
    ToPDURate(74)            short-term flow rate in PDUs per second
    FromPDURate(75)            Parameter 1 = rate interval in seconds
      10-second bit rates have proved very useful in observing
      the utilisation of busy links.  They are particularly useful
      for links which allow short-term traffic bursts (e.g. Frame
      Relay links), since they provide a good indication of just
      how bursty the link's traffic is.

+ Autonomous System Numbers (integers):
    SourceASN(113)    Autonomous System Number for flow's source
    DestASN(115)      Autonomous System Number for flow's destination
    SourcePrefix(114) CIDR width used by router for determining
                        flow's source network
    DestPrefix(116)   CIDR width used by router for determining
                        flow's destination network
      The need for ASN-based attributes originally came from RTFM
      meters using Cisco NetFlow data rather than simply observing
      packet headers.  Other versions of the RTFM meter have
      supported these attributes by looking up ASNs in a table of
      data obtained from a router running BGP.

+ DS CodePoint list-values:
    ToDSCodePoint(118)    DS Code Point (6 bits) for packets in
                            this flow from source to destination
    FromDSCodePoint(119)  DS Code Point (6 bits) for packets in
                            this flow from destination to source
      These attributes allow a meter to build tables showing which
      CodePoints are being used in each direction of flows.


Other attributes discussed in RFC 2724 have not received sufficient
implementation experience - they should not be included in these RTFM
extensions.



3.2  Other possible new attributes

The following areas of interest have been proposed on the mailing list
and/or in test implementations of the RTFM meter:


+ Next-hop address list-values:
    ToNextHop
    FromNextHop
      These attributes would build tables of a router's next-hop
      addresses.  They would be useful for verifying that routing

Nevil Brownlee                                                  [Page 5]


INTERNET-DRAFT       RTFM: Implementing New Attributes      October 2000


      was being performed correctly (i.e. as per the router
      configuration and routing table).

+ ASN PDU and Octet list-values:
    ToASNListPDUs
    FromASNListPDUs
    ToASNListOctets
    FromASNListOctets
      These would build lists of the packets and bytes sent to each
      ASN for a flow's source (From list) and destination (To list).

+ Packet Header Trace attribute:
      This was discussed at the RTFM informal meeting in Pittsburg
      last August.

+ Explicit Congestion Notification (ECN) attributes:
      A set of attributes for assessing the performance of Explicit
      Congestion Notification (RFC 2481) could be worth considering.

+ Delay and loss statistics:
      There is considerable interest in inter-arrival statistics.
      InterarrivalTime attributes are described above, and these
      provide sufficient data to compute statistics of a flow's
      inter-arrival times.  Nonetheless, it could be useful to
      have attributes for some of those statistics, for example
      short-term means.



4  Extensions to the Meter MIB


4.1  Implementing distribution- and list- values


Clearly the Meter MIB should be extended so as to standardise the
implementation of distribution- and list-valued attributes.  This would
most probably be done using BER sequences, described using textual
conventions.


4.2  'Flow size' filters


Experience with nifty (NeTraMet's X Window real-time flow analyser) has
shown that it would be useful for a meter reader to retrieve flow data
only for 'large' flows.  nifty works by reading the PDU and Octet counts
for every active flow, sorting the flows by size (number of PDUs or
Octets), then displaying information about the top n flows.  On busy
networks nifty may have to retrieve data for many thousands of flows


Nevil Brownlee                                                  [Page 6]


INTERNET-DRAFT       RTFM: Implementing New Attributes      October 2000


(e.g. 120,000 flows at 2-minute intervals on an OC3 link carrying about
35 Mbps) - this can badly degrade nifty's performance.

A simple solution to this problem is to provide another column in the
Reader Control table, specifying the minimum number of packets a flow
should have if it is to read by a meter reader.  This provides a filter
so that only 'busy' flows are read by that meter.  Tests of such a
filter with nifty reduced the number of flows of interest from 120,000
to about 10,000.

The default value for this filter would be zero, i.e. all flows would
be retrieved.


5  Author's Address


    Nevil Brownlee
    Information Technology Systems & Services
    The University of Auckland

    Phone: +64 9 373 7599 x8941
    E-mail: n.brownlee@auckland.ac.nz



























                                                      Expires April 2001

Nevil Brownlee                                                  [Page 7]