Should a formal spec be required for phase 3?

[bvisness]

The current requirements for phase 2 are:

• Precise and complete overview document is available in a forked repo around which a reasonably high level of consensus exists.
• Updates to the actual spec document, test suite, and reference interpreter are NOT yet required.

The current requirements for phase 3 are:

• Test suite has been updated to cover the feature in its forked repo.
• The test suite should run against some implementation, though it need not be the reference interpreter.
• Updates on the actual spec document and reference interpreter are NOT yet required (but can happen earlier).

The purpose of phase 2 is for at least one engine to implement the feature, enough to produce a reasonable test suite for phase 3. On the other hand, the purpose of phase 3 is for all engines to implement the feature - but they are forced to do so without a formal spec, and must make do with a "precise and complete overview document". Personally, having gone through much of the Wasm GC implementation with nothing to guide me but a linked list of Google Docs, I would have MUCH PREFERRED a formal spec to follow.

I presume the lack of spec requirement was because, in the past, the formal spec was very burdensome to update. But these days much of the tedium is handled by SpecTec and we have AIs that can catch bugs and slog through macro minutia. I find it much easier these days to keep the formal spec up to date.

Should we amend the process to require a complete formal spec by phase 3?

[rossberg]

Historical note: the process doc as originally written required prose spec text (though not the formalism) for phase 3 entry. But turned out we didn't follow our own process for various proposal and advanced them with only informal "overview" documents. Eventually we relaxed the doc to match reality.

But yeah, that was before SpecTec, so it may be worth considering a reversal.

[kmiller68]

I definitely agree with @bvisness that implementing spec, especially the larger, cross cutting ones without a formal spec is quite daunting. Is anyone particularly opposed to requiring the formal spec for phase 3?

[bvisness]

I would also ask what the process is for amending our phase process. I've not been around for any previous changes to the process. I'm happy to bring it up for discussion in a future CG meeting at the very least, but I imagine some kind of vote would also be required?

[tlively]

The overview is already supposed to be precise enough to enable implementation going into phase 2. If it's not precise enough by phase 3, then something is going wrong. I think the first-order fix here would be to push back more strongly on proposals trying to go to phase 3 with unclear overviews, not to change the spec process.

Also, even with SpecTec and AI, it can still be significant work to update the formal spec (for some proposals more than others), so I'm worried that moving it earlier in the process will lengthen the critical path to get a proposal to phase 4 and shipped. For example, if a champion currently spends phase 2 implementing the proposal themselves and phase 3 working on the spec while other browsers implement in parallel, then moving the spec work to phase 2 would delay things.

My other concern is the review bottleneck for SpecTec changes. If we move the spec work up and folks still depend on @rossberg for reviews even as the spec might be undergoing changes, then @rossberg would end up with a higher review load. It probably makes sense to spread the review load around for PRs to proposal repos, but as a matter of practicality, there just aren't that many people who can review SpecTec. Maybe this is fine because there will still be a thorough review as the repo is merged in phase 5.

I would also ask what the process is for amending our phase process.

In the past we've used a normal CG consensus poll to amend the process.

[rossberg]

@tlively, these are all good points.