Minutes interim-2020-bmwg-01: Fri 15:00
minutes-interim-2020-bmwg-01-202005151500-00

BMWG Interim Meeting (Post IETF-107) May 15, 2020

Agenda

WG Status
WG Drafts:
 - EVPN - status: IESG processing (AD Reviewed - more editing)
   https://tools.ietf.org/html/draft-ietf-bmwg-evpntest-05
 - Next Generation Firewall Benchmarking
   https://tools.ietf.org/html/draft-ietf-bmwg-ngfw-performance-03
 - Back-to-Back Frame (Update to RFC2544)
   https://tools.ietf.org/rfcdiff?url2=draft-ietf-bmwg-b2b-frame-01.txt
Proposals:
 - Multiple Loss Ratio Search
   https://tools.ietf.org/html/draft-vpolak-mkonstan-bmwg-mlrsearch-03
 - Probabilistic Loss Ratio Search
   https://tools.ietf.org/html/draft-vpolak-bmwg-plrsearch-03
 - Network Function Service Density
   draft-mkonstan-nf-service-density (expired),
   revisit the overall problem space, explore tighter collaboration options
 - Benchmarking Methodology for EVPN VPWS
   https://tools.ietf.org/html/draft-kishjac-bmwg-evpnvpwstest-04
 - Benchmarking Methodology for EVPN Multi-casting
   https://tools.ietf.org/html/draft-vikjac-bmwg-evpnmultest-04
 -

AOB:

Attendance: 13 people

BMWG Interim Meeting (Post IETF-107) May 15, 2020

Blue Sheets:
    Al Morton, AT&T
    Warren Kumari, Google
    Timothy Carlin, UNH-IOL  <<< Primary notetaker!!  THANKS TIM !!!
    Maciek Konstantynowicz, Cisco
    Vratko Polak, Cisco
    Brian Monkman, NetSecOPEN
    Stephen Goudreault, TrendMicro
    Sudhin Jacob, Juniper Networks
    Vladimir Vassilev, Lightside Instruments
    Jim Welch, Telestream
    Jurrie Van Den Breekel, Spirent
    Carsten Rossenhoevel, EANTC


Preliminaries:
    Al:
https://datatracker.ietf.org/meeting/interim-2020-bmwg-01/session/bmwg
        using Etherpad for Minutes
        use "q" in chat to enqueue in mic-line
        use "-q" in chat to leave mic-line
        agenda review/bashing (no edits and approved)
    Timothy Carlin, UNH-IOL  <<< Primary notetaker!!  THANKS TIM !!!

WG Status

    Al:
    (slides) -
    https://datatracker.ietf.org/meeting/interim-2020-bmwg-01/materials/slides-interim-2020-bmwg-01-sessa-agenda-status-and-milestones
    Note Well

WG Drafts:
 - EVPN - status: IESG processing (AD Reviewed - more editing)
   https://tools.ietf.org/html/draft-ietf-bmwg-evpntest-05

    Warren: EVPN Doc - Ad Rev+ status - doc still needs reasonable amount of
    editorial work before IETF last call - suggest return to WG to get more
    comments and activity in order to move along.

    Al: Similar observations (e.g. whitespace)

    Sudhin: Will correct the issues

    Agree to send draft back for [short] editorial WG LC

    Warren: (To WG) Please review

    Al: Beneficial for Sudhin to go through entire draft, IESG Review, and RFC
    Editor process before proceeding with other drafts
        Sudhin: agree

 - Next Generation Firewall Benchmarking
   https://tools.ietf.org/html/draft-ietf-bmwg-ngfw-performance-03

    Brian M: Current Draft in process of being updated.  Encouraging feedback
    from NSO participants on WG mailer for more discussion.  Did initial round
    of testing, some suggestions for draft as result.  Looking to "raise bar"
    on security effectiveness portion.  Q: How to best handle updates?

    Al: One option: "RFC updates NNNN".

    Brian: How to handle more frequent (Monthly) updates for
    vulnerabilitys/malware?

    Al: Suggest pointing to "live" page listing of vulnerabilities.  Likely too
    frequent for IETF.

    Brian: Open approach desired via providing guidance within RFC.

    Warren: Page should be authoritative.  e.g. an Individual's page might not
    be desired.

    Brian: Agree.  Want to make sure security industry can respond quickly to
    change.  Proposal: Work with Warren/Al/Others to produce reasonable text.

    Al: Suggest Add section that points to page/paragraph, and where in draft
    to locate, share with WG on list for feedback.

    Brian: Q: Facility in IETF domain for "authoratative" list for these
    purposes?

    Al: Wiki pages are available, maybe not rigid enough access.

    Warren: Not really.  Documents are consensus-based, but listings like this
    wouldn't have "consensus".  IANA doesn't seem right either.

    Brian: Take offline.

    Al: Registry possibility could be examined further. IANA creates new
    registries regularly, new entries are reviewed under policies.  Would need
    to be designed.

    Al: Still have development to go on this doc.  Want to generate more
    traffic [discussion] on mailing list. (Looking at draft side-by-side) e.g.
    Sec 7.4.3.3: Well set up, Performance objects set.  Not bad if not
    Pass/Fail criteria.

    Brian: Goal is target objective. Expectation is that labs/implementers will
    work together to understand.

    Jurrie: Similar to latency test in RFC 2544.  Determine Throughput and
    measure Latency at given throughput.  Rather than run at 100%, run at 50%. 
    FW at peak throughput have high latency, so measure/document.

    Al: Background helps.  Exactly how would have phrased it. Okay to have loss
    requirement ala RFC 3544 Throughput, condition of test.

    Jurrie: Draft states % failed transactions, low percentage.

    Al: Okay to create metrics that use Throughput as basis, looks great.

    Al: Others who have read, questions comments?

    Jurrie: We could add text comparing to RFC 2544.  Value?

    Al: Would have to describe with enough words to explain parallel. 
    Encouraged.  A new section for Test Design in general might be a good place.

    Al: Interesting read, encouraging readership.

 - Back-to-Back Frame (Update to RFC2544)
   https://tools.ietf.org/rfcdiff?url2=draft-ietf-bmwg-b2b-frame-01.txt

    Al (Changing Hats!! Author for this discussion!): Draft in diff-mode. 
    Background: 3 rounds of comments and support from several individuals. 
    Folded into current (draft-01).  Comments have since slowed down, asking
    (as author) for WGLC.  Call for questions.

    Vratko: Sent email (Nov 2019) after -01. Subject of email had -00 possible
    confusion there.

    Al (Author): Possibly overlooked, will look at comments off-line, anything
    for today?

    Vratko: Large comment may be difficult to review in real-time.

    Al (Author): Looking at email from Vratko 2019-11-20 (I-D Action
    draft-ietf-bmwg-b2b-frame-00.txt).

    Vratko: Speaking to email.

    Al (Author): Will clarify and take to list.  Appreciate feedback.

    Vratko: One more idea.  Focus on Max instead of Avg.  Interrupts might
    cause artificially lower Avg.  Will bring to list.

    Maciek: Re: Vratko - Many more situations (over last year) found other
    topologies that this test/benchmark can help.  Thanks for support and work.

    Al (Author): Thanks for support.

        ACTION: reply to Comments, both new and old(DONE)

Proposals:

 - Multiple Loss Ratio Search
   https://tools.ietf.org/html/draft-vpolak-mkonstan-bmwg-mlrsearch-03

    (Maciek now presenting slides)  (Link to slides? - Sent slides to Al for
    publishing.) Published on meeting materials page:
        https://datatracker.ietf.org/meeting/interim-2020-bmwg-01/session/bmwg

    Maciek: Questions?

    Al: Encouraging others to take a look, good approach.

    Maciek: WG Adoption?  Next Steps?

    Al: Asking for readers to volunteer.

    Sudhin will read, as will Al.

 - Probabilistic Loss Ratio Search
   https://tools.ietf.org/html/draft-vpolak-bmwg-plrsearch-03

    (Vratko now presenting slides)  (Link to slides?- Sent slides to Al for
    publishing.)
        https://datatracker.ietf.org/meeting/interim-2020-bmwg-01/session/bmwg

    Vratko: WG Adoption? Next Steps?

    Al: Asking for readers.  Perhaps collaboration with NGFW group re: traffic
    generation.

    Vratko: Not as applicable for stateful traffic.  Currently more volatile
    than repeated MLRsearch.  Indirectly useful for debugging when SUT not
    meeting the assumptions (e.g. when performance deteriorates over time).

    Jurrie: re: Applicable for NGFW Benchmarking?: Since TCP, Known baseline,
    goalseeking increases tx/s, measure % and increase until delta < 1%.  Does
    MLR,PLR authors think this is suitable?

    Vratko: PLRsearch (also MLRsearch) uses pluggable "measurer" (Python). 
    Measurer inputs: Trial duration, offered load (pps).  Measurer outputs:
    "packets" sent, "packets" lost.  Maybe transactions can acts as "packet"s,
    if the "traffic generator" can control transaction submit rate and count
    transaction success/failure reliably.

    Jurrie: Stateful situation, not looking at loss (algorithm ensures
    reception).  Sounds different.

    Maciek: re: Jurrie: Wouldn't mind looking closer at NGFW example to see
    what could be shared between approaches.

 - Network Function Service Density
   draft-mkonstan-nf-service-density (expired),
   revisit the overall problem space, explore tighter collaboration options

    (Maciek now presenting) (Link to slides? - Sent slides to Al for
    publishing.)
        https://datatracker.ietf.org/meeting/interim-2020-bmwg-01/session/bmwg

    Maciek: Does this sound right, or should we re-group?

    Al: Proliferation of cloud networking options makes for complex situation,
    Suggest helping industry coalesce on a few options.

    Maciek: Combinations will continue to increase.  Distill experiences into
    common set for all.

    Breakout meeting possible as next steps, invite other orgs...

 - Benchmarking Methodology for EVPN VPWS
   https://tools.ietf.org/html/draft-kishjac-bmwg-evpnvpwstest-04

    (Sudhin now presenting slides)
    https://datatracker.ietf.org/meeting/interim-2020-bmwg-01/materials/slides-interim-2020-bmwg-01-sessa-benchmarking-methodology-for-evpn-vpws.pdf

    Sudhin: Asking for comments and WG Adoption.
        (further action delayed beyond the current draft,as per agreement above)

 - Benchmarking Methodology for EVPN Multi-casting
   https://tools.ietf.org/html/draft-vikjac-bmwg-evpnmultest-04

    (Sudhin now presenting slides)
    https://datatracker.ietf.org/doc/slides-interim-2020-bmwg-01-sessa-benchmarking-methodology-for-evpn-multihoming-restor-mass-withdrawal/
    (further action delayed beyond the current draft,as per agreement above)

AOB - none: meeting ran over, using additional time as scheduled (20 min)