Last Call Review of draft-ietf-bmwg-sdn-controller-benchmark-term-07

Request Review of draft-ietf-bmwg-sdn-controller-benchmark-term
Requested rev. no specific revision (document currently at 10)
Type Last Call Review
Team Ops Directorate (opsdir)
Deadline 2018-02-02
Requested 2018-01-19
Authors Bhuvaneswaran Vengainathan, Anton Basil, Mark Tassinari, Vishwas Manral, Sarah Banks
Draft last updated 2018-01-31
Completed reviews Rtgdir Last Call review of -07 by Ines Robles (diff)
Opsdir Last Call review of -07 by Ron Bonica (diff)
Genart Last Call review of -07 by Stewart Bryant (diff)
Secdir Last Call review of -07 by Paul Hoffman (diff)
Opsdir Telechat review of -09 by Ron Bonica (diff)
Genart Telechat review of -09 by Stewart Bryant (diff)
Assignment Reviewer Ron Bonica 
State Completed
Review review-ietf-bmwg-sdn-controller-benchmark-term-07-opsdir-lc-bonica-2018-01-31
Reviewed rev. 07 (document currently at 10)
Review result Not Ready
Review completed: 2018-01-31



I have reviewed this document as part of the Operational directorate's ongoing effort to review all IETF documents being processed by the IESG.  These comments were written with the intent of improving the operational aspects of the IETF drafts. Comments that are not addressed in last call may be included in AD reviews during the IESG review.  Document editors and WG chairs should treat these comments just like any other last call comments.

Summary: Not ready for publication

(Maybe I am missing something very fundamental. A conversation with the authors might be appropriate.

Major issues: - If the discovery time begins with the transmission of the first message and ends with some event after transmission of the last message, doesn't this metric rely on the speed at which the network device transmits as well as the speed at which the controller processes messages. Is the controller really the DUT? Or is it a combination of the controller and the network device. - How does the tester know when the discovery process is complete? Ask the controller? Observe some behavior at the network device? - Does discovery time depend on number of nodes and richness of topology? Do we have to standardize these so that we can compare apples to apples? - Do some messages take longer to process than others? Do we need to standardize the messages? - Same questions - Does Reactive Path Provisioning time depend on the number of nodes in the path and the flow control mechanisms implemented (or not implemented) by network devices? How do we compare apples to apples?

These same issues hit other sections. Maybe I am just being dense. I am open to a call with the authors.


Minor issues: