Security Automation and Continuous Monitoring
charter-ietf-sacm-03

[Ballot comment]
I'm glad to see work in this space going forward (so, "Yes" ballot position), but do have some questions that might be worth considering before the charter is approved.

Disclaimer - SACM is far from being something I understand well, and people with more clue may have obvious answers, but since I had questions, I'm asking.

I wasn't sure what would actually be extended in the reference to NEA [https://ietf.org/wg/concluded/nea.html]. Can you point to an NEA RFC that this work is starting from?

I realize that SACM and SUIT are now on the same telechat agenda, but is there any relationship between "information about firmware, operating systems, and software installed on an endpoint" and what you're visualizing for SUIT? ("and for TEEP" is another question, and maybe premature)

I guess I should ask the same thing about "- Define a method of expressing software metadata that is suitable for use by constrained devices including a CBOR-based format derived from the ISO/IEC 19770-2 Software Identification (SWID) tag standard", later in the charter.

For the evaluation criteria language - is there a candidate starting point for this work (or even a potential candidate starting point)?

[Ballot comment]
Thanks for resolving my earlier comments. This version looks better.

The last paragraph of section A. seems to have a cut-and-paste error in the first sentence.

[Ballot comment]
Given the charter has changed substantially and I am not familiar with the current state of work of the group, I'm afraid I can't provide any valuable input about the recharting and will therefore abstain.

[Ballot comment]
Editorial Comments:

I find the charter a little hard to understand in places. This may be entirely due to my lack of familiarity with the subject.

"At its core, posture assessment consists of five basic steps, which the working
group desires to fulfill in an innovative, automated manner capable of avoiding
ad hoc or scheduled assessments:"

I assume the workgroup wants to enable automation of those steps. The language sounds like the workgroup intends to fulfill it's goals in an automated manner.

"A. Collection. The WG will define a standardized way to provide two types of
imperative guidance to collectors over varying collection mechanisms:"

I don't understand what is meant by "imperative guidance". Is that a term of art? Is it different than "information"?

"When classified, a set of instructions (such as vulnerability description data,
YANG filter expressions, Windows Management Instrumentation classes, etc.) can
be brokered to the appropriate collectors using the control plane functions
defined by "C. Orchestration and Communication" (below)."

This is the first mention of classification. It would help to mention what sort of classification is envisioned.

"Detecting and
classifying desired attributes beforehand may require orchestrating functions
that go beyond the set of capabilities a collector can provide, and will inform
the requirements and characteristics for "C. Orchestration and Communication"."

I don't understand the sentence.

"C. Orchestration and Communication. The working group will define a set of
control plane functions to enable the discovery and orchestration between
devices"

Discovery and orchestration of what?

Several of the "specific work items" seem like repetitions of the "lettered" goals.

(Note that much of the language that I found confusing is repeated in the "Specific work item" section.

[Ballot comment]
The charter is very generic and broad: model, collection, evaluation, orchestration and communication, control plane, a criteria language.
I provided feedback to SACM a few times. I trust the responsible AD to do the right thing.

I wonder why you impose CBOR in the charter.

Can we get the milestones please.

Regards, Benoit.

[Ballot comment]
This seems fine to me. Some editorial nits below

> Securing information and the systems that store, process, and transmit
> that information is a challenging task for enterprises of all sizes, and many
> security practitioners spend much of their time on manual processes.
> Standardized protocols and models aiding collection and evaluation of endpoint
> attributes enables automation, thus freeing practitioners to focus on high

Nit: models .... enable


> priority tasks. Due to the breadth of this work, the working group will address
> enterprise use cases pertaining to the assessment of endpoint posture (using
> the definitions of Endpoint and Posture from RFC 5209). In alignment with RFC
> 5209, a network device is an endpoint.
>
> At its core, posture assessment consists of five basic steps, which the working
> group desires to fulfill in an innovative, automated manner capable of avoiding

You're rechartering, so maybe it's less innovative than it was last time :)


> ad hoc or scheduled assessments:
>
> 1. Identify and characterize target endpoints
> 2. Determine specific endpoint elements to assess
> 3. Collect and make available specified elements' actual values
> 4. Compare actual element values to policy compliant element values
> 5. Make results available
>
> This working group will focus on collection, evaluation, and orchestration and
> communication, as described herein.
>
> A. Collection. The WG will define a standardized way to provide two types of
> imperative guidance to collectors over varying collection mechanisms:

I'm not sure what "imperative guidance" means in this context.

[Ballot comment]
I hold no particular position concerning the use of an expiration date, but I share Alissa's question about whether a recharter that just changes a date really needs external review.

[Ballot comment]
Agree with Alvaro. Setting a deadline for a re-charter and then having the only change to the charter be a change to the date seems pointless. Also, if the only thing changing is the date, does the charter really need to go for external review?

[Ballot comment]
Agree with Alvaro. Setting a deadline for a re-charter and then having the only change to the charter be a change to the date seems pointless.

[Ballot comment]
I am not opposed to the work for which the sacm WG is chartered.  I think it is important and look forward to the output.

However, this specific update to the Charter only refreshes the expiration date for the WG — which is being used as a tool to manage its progress.  I don't see the threat of expiration as an effective tool and would not want the practice to spread to other WGs.  As a result I am ABSTAINing.

[Ballot block]
It looks like the only update from the previous charter is changing the expiration date of the WG in the last sentence: 

OLD:

This charter will expire in July 2016. If the charter is not updated
before that time, the WG will be closed and any remaining
documents revert back to individual Internet-Drafts.

NEW:

This charter will expire in January 2017. If the charter is not updated
before that time, the WG will be closed and any remaining documents
revert back to individual Internet-Drafts.

But July 2016 already passed, the WG was not closed and the documents didn't revert back to individual I-Ds.  Not explicitly following this WG I can't tell how realistic the new date is, or if we're going to be changing it again soon.  In other words, what is the value of putting an expiration date in a charter if we already have a precedent of not honoring it?

I may obviously be missing something in the history of this work/WG, so this BLOCK has the intent of taking the expiration date out; or at least clarifying why one is necessary.

[Ballot block]
It looks like the only update from the previous charter is changing the expiration date of the WG in the last sentence: 

OLD:

This charter will expire in July 2016. If the charter is not updated
before that time, the WG will be closed and any remaining
documents revert back to individual Internet-Drafts.

NEW:

This charter will expire in January 2017. If the charter is not updated
before that time, the WG will be closed and any remaining documents
revert back to individual Internet-Drafts.

But July 2016 already passed, the WG was not closed and the documents didn't revert back to individual I-Ds.  Not explicitly following this WG I can't tell how realistic the new date is, or if we're going to be changing the date again soon.  In other words, what is the value of putting an expiration date in a charter if we already have a precedent of not honoring it?

I may obviously be missing something in the history of this work/WG, so this BLOCK has the intent of taking the expiration date out; or at least clarifying why one is necessary.

[Ballot block]
It looks like the only update from the previous charter is changing the expiration date of the WG in the last sentence: 

OLD:

This charter will expire in July 2016. If the charter is not updated
before that time, the WG will be closed and any remaining
documents revert back to individual Internet-Drafts.

NEW:

This charter will expire in January 2017. If the charter is not updated
before that time, the WG will be closed and any remaining documents
revert back to individual Internet-Drafts.

But July 2016 already passed, the WG was not closed and the documents didn't revert back to individual I-Ds.  Not explicitly following this WG I can't tell how realistic the new date is, or if we're going to be changing the date again soon.  In other words, what is the value of putting an expiration date in a charter if we already have a precedent of not honoring it?

I may obviously be missing something in the history of this work/WG, so this BLOCK has the intent of taking the expiration date out; or at least clarifying why one is necessary.

[Ballot block]
It looks like the only update from the previous charter is changing the expiration date of the WG in the last sentence: 

OLD:

This charter will expire in July 2016. If the charter is not updated before
that time, the WG will be closed and any remaining documents revert back to
individual Internet-Drafts.

NEW:

This charter will expire in January 2017. If the charter is not updated before
that time, the WG will be closed and any remaining documents revert back to
individual Internet-Drafts.

But July 2016 already passed, the WG was not closed and the documents didn't revert back to individual I-Ds.  Not explicitly following this WG I can't tell how realistic the new date is, or if we're going to be changing the date again soon.  In other words, what is the value of putting an expiration date in a charter if we already have a precedent of not honoring it?

I may obviously be missing something in the history of this work/WG, so this BLOCK has the intent of taking the expiration date out; or at least clarifying why one is necessary.

[Ballot block]
OLD:
- A standards-track document specifying the informational model for endpoints
data posture and its mapping into the protocol and data format for collecting
actual endpoint posture.

NEW:
- A standards-track document specifying the informational model for endpoints
data posture

[Ballot comment]
Joel how about:

OLD:

The working group will work in close coordination with other WGs in the IETF (including, but not limited to MILE and NEA) in order to create solutions that do not overlap and can be used or re-used to meet the goals of more than one working group.

NEW:

The working group will communicate with non-IETF organizations working on related specifications and will encourage industry participation in the development of the WG's documents.  Other organizations involved in the initial sacm space include ISO/IEC and TCG as well as government agencies such as NIST.

spt
--------------
I'm ok with this.


What other SDO's or existing external work does this integrate?

----

Changing my position until we discuss this on the call.

Subject: Re: [sacm] sacm charter review

Hi Sean,

A list of organizations that are involved in the area, as identified in this
discussion includes:

- TCG
- DMTF
- FIRST
- The Open Group
- ISO/IEC
- W3C
- OASIS
- OMG
- NIST
- MITRE
- 3GPP

It's up to the IESG to decide if we should list these (or some of them)
explicitly, or we should leave to the WG after its formation is approved to
initiate communication and invite participation.

Regards,

Dan

[Ballot comment]
  In accordance with existing IETF processes, the group will communicate
  with and invite participation from other relevant standards bodies and regulatory
  organizations

Is there any sense of what bodies and organizations might be relevant?  Surely we know some now, and can mention them.  Suppose the WG decided that none were relevant; would that be acceptable?

Added charter milestone "Submit protocol and data format for retrieving configuration and policy information for driving data collection and analysis Internet-Draft to the IESG for consideration as Proposed Standard", due September 2014

Added charter milestone "Initial submission of protocol and data format for retrieving configuration and policy information for driving data collection and analysis Internet-Draft", due January 2014

[Ballot block]
"The working group will, whenever reasonable and possible, reuse existing
protocols, mechanisms, information and data models."

Fine. Now, I really would like to see an information model document (see RFC 3444) as a deliverable.
And then the mapping to protocol/data model.

I had to read the charter multiple times to get an idea of the "security information" might mean... Software version, patches, vulnerabilities.
Maybe it's defined by "The initial focus of this work is to address enterprise use cases pertaining to the
assessment of endpoint posture (using the definitions of Endpoint and Posture from RFC 5209).".
And I'm still not quite sure. Hence the importance of the information model as a milestone.

"An example of such an endpoint posture assessment could include, but is not
limited to, the following general steps:
1. Identify endpoints
2. Determine specific endpoint elements to assess
3. Collect actual value of elements
4. Compare actual element values to expected element values
5. Report results"
Then I see in the next sentence of the charter: "policy management"

The WG scope is (too) huge: inventory management + monitoring + fault management + policy management?
Please don't re-invent monitoring and fault management.
Is policy management in scope or not? I guess no.
And inventory: I understand that it means device inventory, but I see later also "vulnerability identifiers".
Clearly mentions what's in scope and what is not.

[Ballot comment]
"Repository protocols are needed to store, update, and retrieve configuration
checks and other types of content required for posture assessment (see step 2
above)."
I don't know what a repository protocol is.

[Ballot comment]
I support work in this area. I have to say, however, not having had the opportunity to attend the meetings on this matter or read the proposals, that I found it hard to read the draft charter. It was not crystal clear to me what the working group will do, even after having read it.

[Ballot comment]
  - An Informational document on Use Cases
  - An Informational document on Requirements
  - An Informational document on SACM Architecture

Unlike the standards track documents, which are well-specified, describing exactly what is going to be produced, these are not. I'd like to understand why there are three documents. It seems to me that Use Cases would simply be examples for the Requirements, and the Requirements would simply define the SACM Architecture. Is that right, or is the Requirements document really a "rules for writing the protocol document" document? It seems to me these could and should all be folded into one document, namely the Architecture document. But I'd also like to hear what the purpose of the Architecture document is. If it's a higher-level layout of how the protocol documents fit together and how a SACM system can be built using the protocol documents, that should probably be a standards track document instead of Informational. But if it's simply an overview of the system, then again I don't understand why it is different than the Requirements or Use Cases document.

So far, these items look like make-work items. Without better explanation, they could cause the WG to spin on them for quite some time.

I agree with Barry that the last paragraph on re-chartering should go away.

All that said, while I'd prefer these things to be addressed before it goes for IETF Review, I don't object to it going forward. If this were going for approval, I might feel differently.

[Ballot block]
What other SDO's or existing external work does this integrate?

----

Changing my position until we discuss this on the call.

Subject: Re: [sacm] sacm charter review

Hi Sean,

A list of organizations that are involved in the area, as identified in this
discussion includes:

- TCG
- DMTF
- FIRST
- The Open Group
- ISO/IEC
- W3C
- OASIS
- OMG
- NIST
- MITRE
- 3GPP

It's up to the IESG to decide if we should list these (or some of them)
explicitly, or we should leave to the WG after its formation is approved to
initiate communication and invite participation.

Regards,

Dan

[Ballot comment]
If you could identify the other SDOs you expect to work closely with, that would be helpful, especially to the IAB.

It would be great if this text wasn't at the bottom of the proposed charter when it's sent out:

This message and attachments may contain confidential information. If it appears
that this message was sent to you by mistake, any retention, dissemination,
distribution or copying of this message and attachments is strictly prohibited.
Please notify the sender immediately and permanently delete the message and any
attachments.

[Ballot comment]
In the first paragraph, the sentences before and after the one that begins "This working group will define" are really repetitious.  I suggest merging them both into the first, like this:
--
  Automating these routine tasks would allow security practitioners to work
  more effectively, focusing on more advanced and high priority tasks, and
  should improve operators' ability to prioritize risk based on timely
  information about threats and vulnerabilities.  To that end, this working
  group will define security automation protocols and data format standards
  in support of information security processes and practices.
--

  An example of such an endpoint posture assessment could include, but
  is not limited to, the following general steps:

The list of five steps after that seems at a level of detail that's out of place in a charter.  Is there a good reason for putting that in the charter?

  In accordance with existing IETF processes, the group will communicate
  with and invite participation from other relevant standards bodies and regulatory
  organizations

Is there any sense of what bodies and organizations might be relevant?  Surely we know some now, and can mention them.  Suppose the WG decided that none were relevant; would that be acceptable?

  After the work items in the current charter have been submitted to and approved
  by the IESG, the WG will recharter or close.

We put this sort of thing in a lot of charters.  Why?  What value does it add?

The last paragraph appears to have been pasted in by accident.