JSONPath IETF January 2022 Interim Meeting Notes

Date: 2022-01-18
Time: 10:00 - 12:00 UTC

Chair: Tim Bray
Chair: James Gruessing
Area Director: Francesca Palombini

• Datatracker
• Jabber
• Minutes

Agenda

• Draft Status
• Issues
• Regular Expressions - #143, #70
• Absent equal to absent - #123
• Updates on other issues
• AOB
• Scheduling of next meeting & IETF 113

Administrivia

• Note Takers
• Jabber Scribe
• Agenda Bashing

Discussion

Status

Carsten: We had 37 issues, down to 12

Stefan: I consider this a big success

Greg: My feelings is that we shouldn't close them if they aren't done, and many aren't done because we aren't including them in the base document, put labels on but leave them open

Carsten: I have to disagree, and need to focus on open issues

Greg: You can focus on issues with filters, how do I know which ones are done or not inclue?

Carsten: That is what the label is for?

Carsten: Let's take 124. We've made a solid decision not to include it.

Greg: If we have made a decision, then why have the label?

Carsten: Because we may have an extension document in future

Tim: The things left open are what is left to do until WGLC

Greg: My disagreement isn't base document inclusion, it's management of issues, I think they should stay open

Greg: Maybe a label for the base?

Glyn: Suggestion: Why don't we reopen the issues after the base is done?

Carsten: Let's do that.

Glyn: Does this satisfy your concern?

Greg: If it happens immediately after, yes.

Tim: This is already done, and we've agreed to reopen them afterwards, so let's move on.

Action: Issues with "revist-after-base-done" label to be re-opened after WGLC and submission of base draft to IESG

Absent == Absent (#123)

Carsten: Not much discussion happened on the list, but when we compare empty node lists in comparison, are these equal or not? Do we convert to undefined, or NaN?

Tim: I think this is well understood, but the idea was we'd get some use cases

Carsten: Assuming we say that equals returns true, someone who wants "yes" has to use ==, but for "no" has to do an existence test first before comparing. If we decide "NaN" someone wants "yes" has to do a negated existence check, and "no" only a == comparison.

Tim: Just to be clear, I don't want them to equal if they aren't there. I should only have to make an existence test on one, right?

Carsten: You have to check both because the other might not exist

Greg: If they don't both exist, you still want it to return true [in the "yes" NaN example]

Stefan: JSON doesn't define undefined, I think I would preference on undefined conversion

Greg: I agree with the undefined, .NET would evaluate those values and return null, and "undefined" is more intuitive

Carsten: The emphasis should not be on the implementor but on the person who writes the queries

Greg: What's a case for having #2?

Stefan: We chose this because of Javascript where NaN == NaN? is false

Tim: In IEEE, NaN == NaN

Greg: This is talking about property presence not numbers

Tim: I've never done actually like this before, this is why we're looking for use cases

Stefan: What about spare arrays? An index with non-existing value, what results? An Exception?

Greg: An Exception, but depends on implementation?

Tim: Does anyone have a use case?

Stefan: I think case 1 is appropriate

Glyn: I'm on the fence, for NaN, what happens if you compare !=?

Tim: Also false

Glyn: Predicate logic holds up a bit better with undefined

[Poll Started]

Stefan: Javascript uses NaN == NaN is true

[Poll Results: 5 in support for undefined, 1 for NaN]

Tim: It appears the group is learning towards with undefined

Carsten: We should make a decision and confirm on the list

Action: Take to list as undefined as the proposed approach

Action: Examples of this behaviour to be included

Regular Expressions (#70, #143)

Carsten: When we tried to select one regular expression engine, we don't have a consensus so the working group gets to choose, but this doesn't mean we shouldn't follow the principle of least surprise

Carsten: The prominent example for parsing is ECMA, and for matching W3C XSD, but both have their complexities as ECMAScript is extremely complex, XSD is simpler but is hard to map to other types of Regex engines. My proposal is to build a modest subset that fits with both camps.

Greg: JSONSchema uses ECMAScript

Carsten: It says you should use a subset of it

Stefan: I think the value for implementors is agreement to a certain subset, so the point of a modest subset is the best way to go

Tim: I've changed my opinion, I'm also with Stefan and that there's a 80/20 point about the advanced features. Our chief priority should be a maximum amount of interoperability, so we should develop the iregexp document.

Carsten: Comparing to iregexp, it has /p so you can use unicode support

Stefan: I see no problem with certain implementors using ECMAScript for as long as they take care of the subset that's included

Greg: I can tell you as an implementor I won't implement iregexp, I'll use whatever .NET provides and other provides will likely do the same

Tim: We need to select the common subset, so it works the same. Are people comfortable with what the slide proposes?

Greg: If it doesn't make our spec too verbose, then sure

Tim: Are we comfortable with having a separate document eventually to become an RFC as opposed to wiring it directly into JSONPath?

Glyn: Will we get the right constituency of people? How do we socialise it?

Carsten: We can shop this around with regexp scars on their face

James: If we do it this way, we can't have an RFC for JSONPath until iregexp is done

Greg: The JSON Schema folks would be happy to include such a document

Stefan: The examples should also include regular expressions, we need that iregexp subset

Consensus: Build an separate regex specification

Action: Take the consensus call to the list

Tim: Are there any outstanding issues with iregexp that deserve some attention?

Carsten: What about /p and do we keep it in? If yes, do we make any further restrictions?

Tim: We have a meta-work item to figure out what to do iregexp to finalise it? It needs a whole lot of examples

Carsten: One issue is the syntax around those regular expressions? Do we use slashes or just use string syntax.

Action: Work on the embedding of regular expressions in JSONPath

Examples (#69)

Carsten: This is still open as a task, and if would be great if people were contributing. We still need to find a structure. A message sent to the mailing list (or PRs) would be appreciated.

Tim: In my experience JSON Schema doesn't provide examples, we should look around existing examples

Glyn: JSONPath comparison is a set of examples

Tim: I appeal to working group members should put some work into this, as it'll reveal disagreements where we thought were agreement

Glyn: Should each example have a schema, document and JSONPath?

Tim: Or a single document with examples

Greg: JSON Pointer has some examples, as well as Stefan's original blog post

Stefan: The examples go to the mailing list or GH issue?

Greg: I say post them on Github, otherwise they'll get lost

Differentiation from JSON Pointer (#67)

Greg: In my experience people get the two confused all the time, and explaining the intents is quite important

W3C WoT discovery, geolocation, etc (#55)

Carsten: I think this is mostly done, we should go back to the WoT people and ask if they are now happy with what we have

Greg: What specifically is in #55?

Carsten: A statement of interest from the W3c and a few requirements they came up with

Tim: I don't understand the request

Carsten: They want to use JSONPath, and we should talk about it

Greg: Does it mean we have to use a URI encoding like JSON Pointer does? Or just say "URI encode the string"?

Stefan: This would be implementation specific and not baked into the standard

Greg: This is specific to them

Carsten: Do we want to do this earlier than WGLC?

Tim: I think not

Security Considerations (#25)

Carsten: Apart from regular expressions, what do people have in mind for this section?

Tim: Denial of Service

Greg: Nested expressions as well

James: JSON Pointer references null unicode characters, and dead-end pointers

Carsten: What's the last point?

James: "implementations should anticipate references to an actual value is not guaranteed"

Tim: Which is all of them? I'm trying to figure out the scenario.

James: I think we've already covered these but worth mentioning

Carsten: We can just point to considerations in other documents

Greg: Another one to look at is GraphQL, what they've done is added details on max depth and their security considerations are robust

Glyn: Do we specify minimum values for interop?

Carsten: That would be hard

James: It's very implementation dependant

Glyn: Someone could turn the knobs to 0 and claim interop?

Carsten: In some spaces we ask implementations to document these limitations

Tim: This is another TODO

Glyn: If an implementation hits a constraint, do they have to fail in a standard way?

Tim: We don't have runtime failures yet, do we? If we're going to allow implementations express these limits.

Stefan: So it would be a syntax error?

Carsten: You can always have a '500', but silently failing is not a good thing.

Duplicates in selector output (#23)

Greg: My issue is that if I'm procesing a value with the same values as the keys, once I've processed it I shouldn't need to process it again

Tim: From someone writing the query, at what point is their life made better..

Greg: Do they just want the resulting values? Do they want nodes, duplicate nodes won't help but values do, or the raw selection set of the query? Seems like three levels of possible output.

Tim: We don't say anything, if you execute with JSONPath with a lot of duplicates you wouldn't be surprised...

Greg: I would ask what we would do in this case?

Glyn: Good use case for examples

Carsten: I don't think we have a good structure for duplicate detection, unlike e.g. "SELECT DISTINCT". We could add that, but I would be against adding it into the base document.

Stefan: If I put two identical selectors into a union, I get the same value twice. Do I expect that? Or should I only get a single value?

Tim: I know if my selector has two things, my code expects two things. I'm just worried about writing the rules that people expect.

Stefan: We need examples. The ".." selector will be surprising here, but not allowed in unions. The proposal is don't remove duplicates, right?

Carsten: Yes.

Glyn: The only positive thing I've heard is performance

Carsten: Except you have to do duplicate detection.

Greg: Can we put into spec that implementors are required to have an option?

Tim: Do we have any other options? This is a big step and one we should be nervous about taking

Stefan: It's the responsibility of post-processing

Greg: In contrast to #123 I can see a lot of use cases for the removal of duplicates

Tim: So instead of an option, rather a syntax feature?

Greg: I don't know of that

Stefan: The most intuitive thing is to remove nothing, and the least amount of effort

Tim: The WG is leaning towards minimalism, is that right? Greg if you think there's a simple and straightforward way, could you please write it down?

Carsten: It's not that easy, so we need examples

Union could have more description (#21)

Stefan: I don't know why we need more description?

Greg: Union is a comma delimited list of selectors, but I was reading as selector and was confusing to me

Stefan: Do you suggest a better name from Union to something else?

Tim: This awakens the set theory guy inside of me

Greg: Collection maybe?

Carsten: List selector?

Tim: Selector list?

Carsten: The slice-index ABNF doesn't exist

Tim: Sounds like "List" is preferable to "Union"

Action: Carsten to make a PR with a proposal and get feedback

CI Hook for commit checking of ABNF (#2)

Glyn: I'll close the ticket

Next:

Carsten: After processing the issues, do a final check and then WGLC. The WGLC would go to a few places e.g. JSON working group mailing list.

Tim: We're not talking about meetings here, but have we closed enough stuff? I agree.

Carsten: We could do a poll on the list, and then WGLC after. And we should do the sanity check ahead of WGLC in February.

Glyn: Is there any constraint to iregexp?

James: We need to do an adoption call on iregexp first

Action: Call for Adoption for iregexp

James: Is a new version needed?

Carsten: Its current state is fine.

Next Meetings

James: IETF 113 is in March, about 6-8 weeks away and unsure if another interim makes sense before then. Should we have a meeting at 113? It's unlikely chairs can attend in person and will be online.

Tim: We should try to have one given it'll be largely virtual

Carsten: There are WG's that never go and only interims, this is not unheard of. The advantage of attendance is getting other people pulled in, at the risk of conflict.

Glyn: Would anyone propose to attend in person? I'm not.

Carsten: I'm thinking but it's unlikely.

Tim: Sounds like the WG isn't enthusiastic?

James: Shall we book it as online only for IETF 113? Or book an interim around the same time, before or after?

Carsten: I would say before.

James: Will draft submission cutoff interfere?

Tim: Let's try and do one 4 weeks from now?

Carsten: We can't do an interim the week before an IETF meeting.

James: IETF 113 is week starting March 21st, latest is week starting Monday 7th.

Action James to put a poll out

AOB