Use flexible serialization for swc wasm plugin

[quininer]

Motivation

Currently the swc plugin abi is quite unstable, which is mentioned in #9439 .

There are two reasons for this

1. The swc plugin currently uses rkyv, which is a fairly static serialization scheme. It is quite sensitive to struct changes, and field changes can easily break the ABI.
2. The swc plugin is always passed a whole ast, which needs to be serialized and deserialized, even the parts that plugin doesn't care about.

This RFC hopes to solve the first problem, so that most plugins can handle the ast produced by future versions of swc without modification.

Detailed design

Flexible serialization scheme

Efficient serialization schemes such as rkyv typically do not record struct layout information in serialized result. This makes it more efficient and compact, but also makes it less flexible.

Flexible serialization schemes are usually self-describing, similar to JSON, and describe their own object boundaries (close token or object length) in the serialized result.

Using a flexible serialization scheme will allow us to decode structs with additional fields without failure.

I used cbor4ii in prototype, which I developed. As can be seen from the benchmark, the performance is not an order of magnitude behind rkyv. see #11100

• that I modified some serde attrs in this benchmark because features like untagged can significantly degrade performance.

Unknown variant

Using a flexible serialization scheme solves the problem of plugin decode new AST, but return plugin AST to swc host will cause information loss.

We need to use unknown field to record information that cannot be processed during deserialization to avoid loss.

#[ast_node]
enum Expr {
    #[cfg(feature = "unknown")]
    Unknown(u32, Unknown), // auto insert
    Call(CallExpr),
    Bin(BinExpr),
    // ...
}

• Use the ast_node macro to automatically insert unknown variant

impl Encode for Expr {
    fn encode(&self) {
        match self {
            Expr::Unknownn(tag, value) => {
                tag.encode();
                value.encode();
            },
            // ...
        }
    }
}

• Keep the original value

To ensure the performance of swc host, unknown variants will only be generated in swc plugin.
This means that the performance of swc host will not be affected at all when there is no any plugin.

Compatibility constraints

Due to performance and complexity considerations, it is planned to use unknown fields only in enums.
So there are some things to pay attention to when make changes to the ast node

1. Exist fields cannot be modified;
2. The order of exist fields cannot be change;
3. New fields must be nullable without affect semantics;
4. For necessary modifications, the old node type need be deprecated and replaced with a new one.

Drawbacks

Not fully compatible

The forward compatibility provided by unknown variant is only for encode and decode.
We cannot perform operations such as transform and emit for unknown nodes and can only panic.

Compatibility constraints may be frequently broken

If we really need to modify fields frequently, we will need to frequently deprecate nodes, which is troublesome for users.

Performance degradation

I think this is inevitable under current rchitecture.