Proposed solution to make grammar changes less disruptive

[tobie]

Goal

Allow faster and easier changes to the grammar while being less disruptive to specs and tools relying on WebIDL.

Issue

Grammar changes are particularly problematic as removed syntax causes parsers to fail. This has the following consequence:

• Previously passing idlharness tests start to fail or error.
• ReSpec specs rendered in the client suddenly start completely failing.
• New builds with Bikeshed require updating the WebIDL.

Working around those issues creates yet another level of problems:

• Avoiding to update the idlharness parser to avoid breaking previous tests incentives editors to keep to the old syntax.
• Parsers may decide to keep supporting older syntax, thus creating situations where certain syntax is supported by Bikeshed, but not by ReSpec, or by ReSpec but not by idlharness, which is really confusing for editors.

Proposed solution

Don't remove grammar right away. Instead, mark grammar as deprecated for a given period of time (6 months?). Have all the tools (parsers, idlharness, ReSpec, Bikeshed) implement the same version of the grammar (with the same deprecation warnings) and sync dropping off support.

Add information in the spec as to when and what tools may/should emit as warning for deprecated syntax.

Immediate targets

• I think serializer has been mostly fixed, so it's probably not worth adding it back to the spec and marking it as deprecated (also because it seems like a lot of work). @gsnedders, @foolip: can you confirm?
• implements is a great candidate, however.

Would love to hear @plinss's, @marcoscaceres's, @tabatkins's, @bzbarsky's, @annevk's and @domenic's thoughts on the topic.

[marcoscaceres]

ReSpec specs rendered in the client suddenly start completely failing.

FWIW, if parsing fails, I just place the IDL back into the pre element. However, the linking doesn't work and an error is shown to the editor... so, not exactly "completely failing", but not great.

Don't remove grammar right away. Instead, mark grammar as deprecated for a given period of time (6 months?). Have all the tools (parsers, idlharness, ReSpec, Bikeshed) implement the same version of the grammar (with the same deprecation warnings) and sync dropping off support.

I've be supportive of this, but maybe with a shorter cycle (3 months max). Editors don't usually take action until things actually break - and I've seen technical debt accumulate to breaking point (e.g., the Web Audio spec).

Breaking things lights (controlled) fires, and we are pretty good at putting them out.

[domenic]

I don't think this is a good idea. We need to be able to evolve Web IDL without the burden of old syntax, and we need to be able to push the ecosystem forward as fast as we did with serializers.

If some specs are having trouble, then I think this should be solved at a tooling level, not a spec level. E.g. a deprecated features mode could be added to the tools, or we could make it easier to build specs using last-known-good revisions of tools instead of always-master.

[marcoscaceres]

I don't think this is a good idea. We need to be able to evolve Web IDL without the burden of old syntax, and we need to be able to push the ecosystem forward as fast as we did with serializers.

I agree. Just need to keep communicating breaking changes upstream by filing bugs on the various tools (as has been done quite effectively lately, so huge thanks to everyone doing that! ❤️).

If some specs are having trouble, then I think this should be solved at a tooling level, not a spec level.

Agree. We can definitely work on supporting a compat mode on the webidl2.js side.

[annevk]

I think what would help is not landing the IDL PR until the necessary downstream bugs are filed.

[foolip]

A part of the solution could be web-platform-tests/wpt#7784, which should mean that web-platform-tests IDL is very rarely out of sync with the spec. So, the process for replacing some bit of syntax would be:

1. Add support for the new syntax in webidl2.js and roll that into web-platform-tests
2. Update the WebIDL spec and all specs using the old syntax
3. Wait for all of those IDL changes to roll into web-platform-tests
4. Remove support from the old syntax in webidl2.js
5. It doesn't matter when that reaches web-platform-tests, but it eventually will

Not exactly streamlined, but more fun that updating IDL in wpt manually as I was doing this time.

[tobie]

@annevk wrote:

I think what would help is not landing the IDL PR until the necessary downstream bugs are filed.

Unless they're acted on immediately, this won't prevent either a period where the spec is updated and the tools aren't or the risk of important merge conflicts if we wait until the tools are updated to merge the spec. No?

[annevk]

We could additionally wait for at least one such tool to also have a PR (preferably the Bikeshed dependency). If we additionally fix web-platform-tests I think we'd be mostly covered. It'll take a little longer, but should be smoother. And the specification PR will get the benefits of multiple reviews.

[littledan]

I like the idea of having a deprecation period. It's not reasonable to expect all spec authors to be experts in the latest changes in specification infrastructure; that will be a burden on them, and in mitigating the burden, there will just be more conservatism in changes to WebIDL. Especially if we make drastic changes, such as the ones I'm suggesting in #485 , we may want a long period, such as two years, for all the specs and tools to update, together with releasing tooling to translate from one version to another (not sure if this already exists).

[annevk]

We have no translation tooling available. That would probably be good, even if we manage to batch all changes we want to make.

[tobie]

Translation tooling should be relatively easy to build on top of Bikeshed and ReSpec's parser, or as a standalone tool to use in post processing.

[ExE-Boss]

This would also help with #843 (#1030).

[annevk]

Perhaps we can consider this more-or-less solved thanks to @saschanaz's @autokagami + filing various bugs against implementations? We've used that tactic a few times now with relatively little fallout (though it's still somewhat intensive of course).