Benchmarking Methodology for Firewall Performance
RFC 3511

• Status
• IESG evaluation record
• IESG writeups
• Email expansions
• History

• Versions
• 08

Network Working Group                                         B. Hickman
Request for Comments: 3511                        Spirent Communications
Category: Informational                                        D. Newman
                                                            Network Test
                                                             S. Tadjudin
                                                  Spirent Communications
                                                               T. Martin
                                                     GVNW Consulting Inc
                                                              April 2003

           Benchmarking Methodology for Firewall Performance

Status of this Memo

   This memo provides information for the Internet community.  It does
   not specify an Internet standard of any kind.  Distribution of this
   memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2003).  All Rights Reserved.

Abstract

   This document discusses and defines a number of tests that may be
   used to describe the performance characteristics of firewalls.  In
   addition to defining the tests, this document also describes specific
   formats for reporting the results of the tests.

   This document is a product of the Benchmarking Methodology Working
   Group (BMWG) of the Internet Engineering Task Force (IETF).

Table of Contents

   1. Introduction . . . . . . . . . . . . . . . . . . . . . . . .  2
   2. Requirements . . . . . . . . . . . . . . . . . . . . . . . .  2
   3. Scope  . . . . . . . . . . . . . . . . . . . . . . . . . . .  3
   4. Test setup . . . . . . . . . . . . . . . . . . . . . . . . .  3
      4.1 Test Considerations. . . . . . . . . . . . . . . . . . .  4
      4.2 Virtual Client/Servers . . . . . . . . . . . . . . . . .  4
      4.3 Test Traffic Requirements. . . . . . . . . . . . . . . .  5
      4.4 DUT/SUT Traffic Flows. . . . . . . . . . . . . . . . . .  5
      4.5 Multiple Client/Server Testing . . . . . . . . . . . . .  5
      4.6 Network Address Translation (NAT). . . . . . . . . . . .  6
      4.7 Rule Sets. . . . . . . . . . . . . . . . . . . . . . . .  6
      4.8 Web Caching. . . . . . . . . . . . . . . . . . . . . . .  6
      4.9 Authentication . . . . . . . . . . . . . . . . . . . . .  7

Hickman, et al.              Informational                      [Page 1]
RFC 3511          Methodology for Firewall Performance        April 2003

      4.10 TCP Stack Considerations. . . . . . . . . . . . . . . .  7
   5. Benchmarking Tests . . . . . . . . . . . . . . . . . . . . .  7
      5.1 IP throughput. . . . . . . . . . . . . . . . . . . . . .  7
      5.2 Concurrent TCP Connection Capacity . . . . . . . . . . .  9
      5.3 Maximum TCP Connection Establishment Rate. . . . . . . . 12
      5.4 Maximum TCP Connection Tear Down Rate. . . . . . . . . . 14
      5.5 Denial Of Service Handling . . . . . . . . . . . . . . . 16
      5.6 HTTP Transfer Rate . . . . . . . . . . . . . . . . . . . 18
      5.7 Maximum HTTP Transaction Rate. . . . . . . . . . . . . . 21
      5.8 Illegal Traffic Handling . . . . . . . . . . . . . . . . 23
      5.9 IP Fragmentation Handling. . . . . . . . . . . . . . . . 24
      5.10 Latency . . . . . . . . . . . . . . . . . . . . . . . . 26
   6. References . . . . . . . . . . . . . . . . . . . . . . . . . 29
      6.1 Normative References . . . . . . . . . . . . . . . . . . 29
      6.2 Informative References . . . . . . . . . . . . . . . . . 30
   7. Security Consideration . . . . . . . . . . . . . . . . . . . 30
   Appendix A - HyperText Transfer Protocol (HTTP) . . . . . . . . 31
   Appendix B - Connection Establishment Time Measurements . . . . 31
   Appendix C - Connection Tear Down Time Measurements . . . . . . 32
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . 33
   Full Copyright Statement  . . . . . . . . . . . . . . . . . . . 34

1. Introduction

   This document provides methodologies for the performance benchmarking
   of firewalls.  It covers four areas: forwarding, connection, latency
   and filtering.  In addition to defining tests, this document also
   describes specific formats for reporting test results.

   A previous document, "Benchmarking Terminology for Firewall
   Performance" [1], defines many of the terms that are used in this
   document.  The terminology document SHOULD be consulted before
   attempting to make use of this document.

2. Requirements

   In this document, the words that are used to define the significance
   of each particular requirement are capitalized.  These words are:

   *  "MUST" This word, or the words "REQUIRED" and "SHALL" mean that
      the item is an absolute requirement of the specification.

   *  "SHOULD" This word or the adjective "RECOMMENDED" means that there
      may exist valid reasons in particular circumstances to ignore this
      item, but the full implications should be understood and the case