Skip to content

Eliminate fixed-content tokens from serialized AST data 🚀 #10755

@kazcw

Description

@kazcw

Many AST fields always have the same corresponding text, e.g. the import keyword in an import statement, or the = operator in an assignment. We should not serialize these fields, and in turn not create Token objects for them in Java and JS.

In JS we will need to codegen type-specific printing logic to insert the known strings when concretizing. We will also need spacing information for the elided tokens. We can represent this with a SMI-sized bitfield. The bitfield too can be elided as long as its value is the default for its tree type. In case the user does something unusual like "x = 23", we can fall back to a full representation.

This change will probably not affect backend code at all, as the backend has no reason to look at these fields, and uses span info for any printing.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    ❓New

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions