Merge master and support upstream's parser enhancements #41

st0012 · 2025-04-25T08:19:06Z

ruby/rbs recently received a few enhancements for new keywords and inline parsing:

So the major change of this PR is to reimplement them under the c-api structure.

A sub buffer is a buffer that contains slices of a buffer.

* Position APIs returns the absolute position of the buffer * Use `#local_location` to access the local positions of a sub-buffer

Bumps [csv](https://github.com/ruby/csv) from 3.3.3 to 3.3.4. - [Release notes](https://github.com/ruby/csv/releases) - [Changelog](https://github.com/ruby/csv/blob/main/NEWS.md) - [Commits](ruby/csv@v3.3.3...v3.3.4) --- updated-dependencies: - dependency-name: csv dependency-version: 3.3.4 dependency-type: indirect update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>

Bumps [strscan](https://github.com/ruby/strscan) from 3.1.2 to 3.1.3. - [Release notes](https://github.com/ruby/strscan/releases) - [Changelog](https://github.com/ruby/strscan/blob/master/NEWS.md) - [Commits](ruby/strscan@v3.1.2...v3.1.3) --- updated-dependencies: - dependency-name: strscan dependency-version: 3.1.3 dependency-type: indirect update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>

….3.4 Bump csv from 3.3.3 to 3.3.4 in /steep

…an-3.1.3 Bump strscan from 3.1.2 to 3.1.3 in /steep

Bumps [rubocop-ast](https://github.com/rubocop/rubocop-ast) from 1.44.0 to 1.44.1. - [Release notes](https://github.com/rubocop/rubocop-ast/releases) - [Changelog](https://github.com/rubocop/rubocop-ast/blob/master/CHANGELOG.md) - [Commits](rubocop/rubocop-ast@v1.44.0...v1.44.1) --- updated-dependencies: - dependency-name: rubocop-ast dependency-version: 1.44.1 dependency-type: indirect update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>

Bumps [parser](https://github.com/whitequark/parser) from 3.3.7.4 to 3.3.8.0. - [Changelog](https://github.com/whitequark/parser/blob/master/CHANGELOG.md) - [Commits](whitequark/parser@v3.3.7.4...v3.3.8.0) --- updated-dependencies: - dependency-name: parser dependency-version: 3.3.8.0 dependency-type: indirect update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>

…-1.44.1 Bump rubocop-ast from 1.44.0 to 1.44.1

Bump parser from 3.3.7.4 to 3.3.8.0

Inline annotations

Add `@rbs skip` annotation

…t-runtime-0.5.12024 Bump sorbet-runtime from 0.5.12020 to 0.5.12024 in /steep

Bumps [sorbet-runtime](https://github.com/sorbet/sorbet) from 0.5.12024 to 0.5.12026. - [Release notes](https://github.com/sorbet/sorbet/releases) - [Commits](https://github.com/sorbet/sorbet/commits) --- updated-dependencies: - dependency-name: sorbet-runtime dependency-version: 0.5.12026 dependency-type: indirect update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>

…t-runtime-0.5.12026 Bump sorbet-runtime from 0.5.12024 to 0.5.12026 in /steep

Bumps [stringio](https://github.com/ruby/stringio) from 3.1.6 to 3.1.7. - [Release notes](https://github.com/ruby/stringio/releases) - [Changelog](https://github.com/ruby/stringio/blob/master/NEWS.md) - [Commits](ruby/stringio@v3.1.6...v3.1.7) --- updated-dependencies: - dependency-name: stringio dependency-version: 3.1.7 dependency-type: indirect update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>

Bump stringio from 3.1.6 to 3.1.7

Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.18.7 to 1.18.8. - [Release notes](https://github.com/sparklemotion/nokogiri/releases) - [Changelog](https://github.com/sparklemotion/nokogiri/blob/main/CHANGELOG.md) - [Commits](sparklemotion/nokogiri@v1.18.7...v1.18.8) --- updated-dependencies: - dependency-name: nokogiri dependency-version: 1.18.8 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]>

…18.8 Bump nokogiri from 1.18.7 to 1.18.8

Bumps [sorbet-runtime](https://github.com/sorbet/sorbet) from 0.5.12026 to 0.5.12028. - [Release notes](https://github.com/sorbet/sorbet/releases) - [Commits](https://github.com/sorbet/sorbet/commits) --- updated-dependencies: - dependency-name: sorbet-runtime dependency-version: 0.5.12028 dependency-type: indirect update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>

…t-runtime-0.5.12028 Bump sorbet-runtime from 0.5.12026 to 0.5.12028 in /steep

Bumps [sorbet-runtime](https://github.com/sorbet/sorbet) from 0.5.12028 to 0.5.12032. - [Release notes](https://github.com/sorbet/sorbet/releases) - [Commits](https://github.com/sorbet/sorbet/commits) --- updated-dependencies: - dependency-name: sorbet-runtime dependency-version: 0.5.12032 dependency-type: indirect update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>

…t-runtime-0.5.12032 Bump sorbet-runtime from 0.5.12028 to 0.5.12032 in /steep

Bumps [rubocop](https://github.com/rubocop/rubocop) from 1.75.2 to 1.75.3. - [Release notes](https://github.com/rubocop/rubocop/releases) - [Changelog](https://github.com/rubocop/rubocop/blob/master/CHANGELOG.md) - [Commits](rubocop/rubocop@v1.75.2...v1.75.3) --- updated-dependencies: - dependency-name: rubocop dependency-version: 1.75.3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>

Bump rubocop from 1.75.2 to 1.75.3

st0012 · 2025-04-25T08:21:09Z

include/rbs/location.h

+  rbs_location_list_node_t *head;
+  rbs_location_list_node_t *tail;
+  size_t length;
+} rbs_location_list_t;


The new MethodTypesAnnotation type node has an attribute called vertical_bar_locations that holds a list of locations. Since we currently don't have a data structure for that, I added this by mimicking the rbs_node_list struct.

st0012 · 2025-04-25T08:31:00Z

src/parser.c

-      rbs_parser_set_error(parser, parser->current_token, false, "Unexpected error");
-      return false;
-    }
+    switch (parser->current_token.type) {


Following upstream to use switch

st0012 · 2025-04-25T08:32:11Z

templates/ext/rbs_extension/ast_translation.c.erb

 }

 VALUE rbs_loc_to_ruby_location(rbs_translation_context_t ctx, rbs_location_t *source_loc) {
+  if (source_loc == NULL) {


This is for the new ReturnTypeAnnotation as its comment_location attribute is a nilable Location.
The alternatives would be either:

Add a new nilable_rbs_location C type to the template and config.yml

Or, have a case in templates specifically for comment_location

templates/src/ast.c.erb

`rbs_node_destroy`, `rbs_hash_free`, `rbs_node_list_free` are only calling each other recursively without any real freeing logic. This is the result of previous efforts to allocate all nodes on the arena. So we don't need these functions anymore. Discovered while working on #41

Morriar · 2025-04-25T13:53:29Z

lib/rbs/ast/ruby/annotations.rb

+            self.class.new(
+              location:, prefix_location:,
+              type: type.map_type_name { yield _1 }
+            ) #: self


amomchilov

Looks good overall! I suggest some small changes, but none are blockers.

Great work, Stan!

ext/rbs_extension/main.c

src/parser.c

amomchilov · 2025-04-29T15:57:09Z

src/parser.c

+
+      rbs_parser_advance(parser);
+
+      rbs_location_list_append(bar_locations, bar_location);


From what I can tell, this is the only place where rbs_location_list_append is called. We have access to the allocator here.

Would it be preferable to pass is as the first param, so that the rbs_location_list_t doesn't have to store it? I'm not sure either way, cc @Morriar

Let's focus on getting this merged to avoid more rebasing. The change can be done later.

Ideally we should refactor this so we don't need a location list.

Agree

I suggest some small changes, but none are blockers.

Co-authored-by: Alexander Momchilov <[email protected]>

src/parser.c

soutaro and others added 30 commits April 9, 2025 16:13

Introduce sub buffer

5668949

A sub buffer is a buffer that contains slices of a buffer.

Let Location support sub buffers

cbff9ec

* Position APIs returns the absolute position of the buffer * Use `#local_location` to access the local positions of a sub-buffer

Add inline annotation AST

4a9d257

Add @rbs token

7211c18

Setup bridge between parser and AST

d0b2e36

Parse inline trailing : TYPE annotation

41da42e

Parser inline leading : METHOD-TYPE annotation

7ac54d4

Parser inline leading @rbs METHOD-TYPEs annotation

94facd8

Add parsing error test

13478cf

Add CommentBlock

15d2614

Add CommentAssociation

0f4fd2c

Merge pull request ruby#2399 from ruby/dependabot/bundler/steep/csv-3…

816a5a0

….3.4 Bump csv from 3.3.3 to 3.3.4 in /steep

Merge pull request ruby#2400 from ruby/dependabot/bundler/steep/strsc…

de95753

…an-3.1.3 Bump strscan from 3.1.2 to 3.1.3 in /steep

Merge pull request ruby#2401 from ruby/dependabot/bundler/rubocop-ast…

9100dc6

…-1.44.1 Bump rubocop-ast from 1.44.0 to 1.44.1

Merge pull request ruby#2402 from ruby/dependabot/bundler/parser-3.3.8.0

6f9725e

Bump parser from 3.3.7.4 to 3.3.8.0

Add MethodTypeAnnotation class

594ee7d

Implement inline parser with method type annotations

3178cfc

Use Struct instead of Data

b0173c2

Add frozen_string_literal comment

de06b01

Merge pull request ruby#2403 from ruby/inline-annotations

674dbdf

Inline annotations

Add SkipAnnotation AST

c8b7fce

Add skip keyword

1ceb7af

Implement parser

0795a23

Update parser to support skip annotation

76e1b46

Merge pull request ruby#2405 from ruby/inline-skip-annotation

351b6cd

Add `@rbs skip` annotation

Add annotation AST

30af9cb

soutaro and others added 13 commits April 18, 2025 06:44

Merge pull request ruby#2418 from ruby/dependabot/bundler/steep/sorbe…

33804aa

…t-runtime-0.5.12024 Bump sorbet-runtime from 0.5.12020 to 0.5.12024 in /steep

Merge pull request ruby#2420 from ruby/dependabot/bundler/steep/sorbe…

e43b3b6

…t-runtime-0.5.12026 Bump sorbet-runtime from 0.5.12024 to 0.5.12026 in /steep

Merge pull request ruby#2421 from ruby/dependabot/bundler/stringio-3.1.7

bdefcc4

Bump stringio from 3.1.6 to 3.1.7

Merge pull request ruby#2422 from ruby/dependabot/bundler/nokogiri-1.…

65141c9

…18.8 Bump nokogiri from 1.18.7 to 1.18.8

Merge pull request ruby#2423 from ruby/dependabot/bundler/steep/sorbe…

8d6a164

…t-runtime-0.5.12028 Bump sorbet-runtime from 0.5.12026 to 0.5.12028 in /steep

Merge pull request ruby#2424 from ruby/dependabot/bundler/steep/sorbe…

b25d268

…t-runtime-0.5.12032 Bump sorbet-runtime from 0.5.12028 to 0.5.12032 in /steep

Merge pull request ruby#2425 from ruby/dependabot/bundler/rubocop-1.75.3

e73a2ec

Bump rubocop from 1.75.2 to 1.75.3

st0012 commented Apr 25, 2025

View reviewed changes

Merge branch 'master' into merge-master

afc0f98

st0012 force-pushed the merge-master branch from 791e0bc to afc0f98 Compare April 25, 2025 08:35

st0012 mentioned this pull request Apr 25, 2025

Remove unnecessary freeing functions #40

Merged

st0012 requested review from Morriar and amomchilov April 25, 2025 12:21

Merge branch 'c-api' into merge-master

b5e5032

Morriar reviewed Apr 25, 2025

View reviewed changes

Morriar approved these changes Apr 25, 2025

View reviewed changes

amomchilov approved these changes Apr 29, 2025

View reviewed changes

st0012 and others added 2 commits April 30, 2025 21:23

Apply suggestions from code review

0a8a4c5

Co-authored-by: Alexander Momchilov <[email protected]>

Update src/parser.c

a9e7702

Co-authored-by: Alexander Momchilov <[email protected]>

st0012 commented Apr 30, 2025

View reviewed changes

src/parser.c Outdated Show resolved Hide resolved

Update src/parser.c

514745b

st0012 merged commit 21752fb into c-api Apr 30, 2025
19 of 20 checks passed

st0012 deleted the merge-master branch April 30, 2025 13:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Merge master and support upstream's parser enhancements #41

Merge master and support upstream's parser enhancements #41

Uh oh!

st0012 commented Apr 25, 2025

Uh oh!

st0012 Apr 25, 2025

Uh oh!

st0012 Apr 25, 2025

Uh oh!

st0012 Apr 25, 2025

Uh oh!

Uh oh!

Morriar Apr 25, 2025

Uh oh!

amomchilov left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

amomchilov Apr 29, 2025

Uh oh!

Morriar Apr 29, 2025

Uh oh!

amomchilov Apr 29, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!


		rbs_parser_advance(parser);

		rbs_location_list_append(bar_locations, bar_location);

Merge master and support upstream's parser enhancements #41

Merge master and support upstream's parser enhancements #41

Uh oh!

Conversation

st0012 commented Apr 25, 2025

Uh oh!

st0012 Apr 25, 2025

Choose a reason for hiding this comment

Uh oh!

st0012 Apr 25, 2025

Choose a reason for hiding this comment

Uh oh!

st0012 Apr 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Morriar Apr 25, 2025

Choose a reason for hiding this comment

Uh oh!

amomchilov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

amomchilov Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

Morriar Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

amomchilov Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!