Tact source code formatter

[anton-trunov]

Currently, tact-vscode does source code formatting using a plugin for Prettier.
I believe a formatter should be a part of the language package. Also, we can have better control over how code is formatted.

Btw, since we now have a Tree-sitter grammar for Tact, formatting can be done using Topiary.

The formatter should probably be aware of tactdoc comments: #232. Or at least keep the formatting of comments intact

[novusnota]

Good suggestion. Alternatively, we could just re-use the aforementioned prettier plugin made in tact-lang/tact-vscode.

Also, we can have better control over how code is formatted.

Yes, but if anything, it would be nice to make the formatter "opinionated" and idiomatic, like the psf/black formatter for Python, or the default formatter for Golang, gofmt. That is, to figure out the optimal defaults and allow no change — this would significantly simplify all the formatting means, as all code would be in either of two states: unformatted, or formatted according to the one and only set of official rules.

I believe a formatter should be a part of the language package.

I do agree. Besides, the possible static analyzer (linter) or other similar tooling too.
Perhaps, it's best to make them standalone binaries/scripts (to make use of on their own), then bundle them with the compiler and make a unified interface in the CLI (#136).

[anton-trunov]

it's best to make them standalone binaries/scripts (to make use of on their own), then bundle them with the compiler and make a unified interface in the CLI

Good idea!

The first step could be to indeed move the code from tact-vscode to tact itself and refactor tact-vscode to use it throw the Tact API. And then we can employ our API for the CLI tool. And at some point we can switch the formatter implementation to the one we want (if there is an actual need)

[byakuren-hijiri]

The initial implementation of the formatter is available in the 162-formatter branch: main...162-formatter.

@heli0dus could you please take a look? You could simply fork the 162-formatter branch and create a new PR from it.

Before merging this, we should consider the following:

• Test and fix the existing logic: The current version of the formatter might contain implementation flaws. It makes sense to look at the following cases in particular:
  • Parentheses in complex binary expressions
  • Indentation, including nested code blocks
  • New lines
• Add unit tests: To avoid regressions in future development, we should cover AST constructions with regular unit tests. The idea is to parse a contract, format it, and compare the result with the formatted code. Code coverage might be a good metric to ensure we cover all the formatter constructions at least once.
• Additional features needed: If any additional features need to be implemented, we should discuss them and create issues as needed. Here are a few examples:
  • Configuration of the formatter: aspects like indentation level and column length should be configurable
  • Should we force new lines when formatting long expressions?

[novusnota]

Does the formatter handle comments? Can't find mentions of them in its sources.

Configuration of the formatter: aspects like indentation level and column length should be configurable

Also, regarding to configuration, I think that it's best to go into the direction of Go (gofmt), Dart (dart fmt) and Black (Python code formatter), which do not support any configuration and provide "good enough" defaults that form a standard across all code bases. So we can omit this point, I think.

Should we force new lines when formatting long expressions?

We should definitely do so for trailing commas in arguments of initOf (...), static and extension function calls and Struct/Message instantiations. And, probably, chained dot-call sequences have to be split by lines, with one extension function on each.

Those (or alternative/additional) rules should probably also go under style guide

[byakuren-hijiri]

Does the formatter handle comments? Can't find mentions of them in its sources.

No, since it requires changes in the frontend that should be additionally discussed.

It would be great to implement this later in a separate PR, while the goal of #335 is to introduce an API for third parties and internal use that will help test the compiler itself. Our next step would be make a tool for end-users.

Also, regarding to configuration, I think that it's best to go into the direction of Go (gofmt), Dart (dart fmt) and Black (Python code formatter), which do not support any configuration and provide "good enough" defaults that form a standard across all code bases. So we can omit this point, I think.

That's a good point. I agree that the tool could have only the default configuration. However, it might be beneficial to have configuration options in the API internals, as it could be useful for internal use to have, for example, a more concise format of the formatted code.

We should definitely do so for trailing commas in arguments of initOf (...), static and extension function calls and Struct/Message instantiations. And, probably, chained dot-call sequences have to be split by lines, with one extension function on each.

Agreed. I think we should also add some newlines in long binary expressions, e.g., between big numbers, as I see them a lot in the fuzzer.

[novusnota]

May be insightful: https://journal.stuffwithstuff.com/2015/09/08/the-hardest-program-ive-ever-written/

[anton-trunov]

#335 implemented a pretty-printer, but we still don't have a formatter

[jubnzv]

implemented a pretty-printer, but we still don't have a formatter

What do we need to prepare a production version of the formatter?

From my perspective, it would be nice to:

• support comments in the syntax tree and the formatter (see Tact frontend API: Improvements for tooling #314)
• split long expressions to multiple lines
• leverage the AST equivalence API (AST Comparison API #686) in the test suite instead of the JSON hack

Creating a binary and integrating it to text editors is a quite trivial task (at least for neovim).

[anton-trunov]

Sounds like a great plan!

[novusnota]

Good points, @jubnzv!

Creating a binary and integrating it to text editors is a quite trivial task (at least for neovim).

We'll only need to add it to the public API, then use that in the LSP — and all the clients that will be using the language server will also get formatting right away!

As a bonus, we may also expose the formatter in Tact's CLI. Something like --fmt or even a separate subcommand will do.