Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[.NET] Removed dependency on System.Text.Json for improved startup time #338

Merged
merged 1 commit into from
Dec 24, 2024

Conversation

obligaron
Copy link
Contributor

🤔 What's changed?

This PR replaces reading reading gherkin-languages.json on runtime with a precompiled step (roslyn source generator). This reduces the runtime overhead for startup/coldstart and removes dependencies (and with them jit time).

Results before:

Method Runtime FeatureFile Mean Error StdDev Gen0 Gen1 Gen2 Allocated
Parser .NET 8.0 tags.feature 410.61 μs 8.183 μs 10.349 μs 35.1563 35.1563 35.1563 490.98 KB
ParserReuse .NET 8.0 tags.feature 15.34 μs 0.182 μs 0.162 μs 2.5177 0.0916 - 41.18 KB
Parser .NET Framework 4.8.1 tags.feature 876.67 μs 15.740 μs 14.724 μs 71.2891 35.1563 35.1563 512.97 KB
ParserReuse .NET Framework 4.8.1 tags.feature 23.80 μs 0.108 μs 0.084 μs 7.3853 0.2747 - 45.45 KB
Parser .NET 8.0 very_long.feature 989.34 μs 7.098 μs 6.292 μs 140.6250 70.3125 35.1563 2012.58 KB
ParserReuse .NET 8.0 very_long.feature 565.29 μs 5.020 μs 4.695 μs 94.7266 47.8516 - 1558.77 KB
Parser .NET Framework 4.8.1 very_long.feature 1,901.60 μs 11.795 μs 10.456 μs 320.3125 142.5781 35.1563 2071.23 KB
ParserReuse .NET Framework 4.8.1 very_long.feature 905.75 μs 2.283 μs 1.906 μs 259.7656 100.5859 - 1596.55 KB

Results after:

Method Runtime FeatureFile Mean Error StdDev Gen0 Gen1 Allocated
Parser .NET 8.0 tags.feature 23.01 μs 0.440 μs 0.556 μs 2.9297 0.1221 49.04 KB
ParserReuse .NET 8.0 tags.feature 15.49 μs 0.215 μs 0.201 μs 2.5024 0.0916 41.18 KB
Parser .NET Framework 4.8.1 tags.feature 31.99 μs 0.613 μs 0.656 μs 9.2773 0.4272 57.32 KB
ParserReuse .NET Framework 4.8.1 tags.feature 23.45 μs 0.105 μs 0.088 μs 7.3853 0.2747 45.45 KB
Parser .NET 8.0 very_long.feature 565.50 μs 5.492 μs 5.138 μs 95.7031 54.6875 1568.74 KB
ParserReuse .NET 8.0 very_long.feature 546.43 μs 10.785 μs 10.088 μs 94.7266 47.8516 1558.77 KB
Parser .NET Framework 4.8.1 very_long.feature 931.86 μs 5.545 μs 5.187 μs 261.7188 96.6797 1610.54 KB
ParserReuse .NET Framework 4.8.1 very_long.feature 896.85 μs 2.761 μs 2.447 μs 259.7656 100.5859 1596.55 KB
Bonus Comparions with RunStrategy.ColdStart

Note: RunStrategy.ColdStart is not very stable, so take this with a little salt.

Results before:

Method Runtime FeatureFile Mean Error StdDev Median Allocated
Parser .NET 8.0 tags.feature 1,560.7 μs 1,093.0 μs 3,222.6 μs 1,211.90 μs 508.6 KB
ParserReuse .NET 8.0 tags.feature 434.4 μs 1,113.6 μs 3,283.5 μs 103.95 μs 41.9 KB
Parser .NET Framework 4.8.1 tags.feature 1,825.7 μs 2,641.5 μs 7,788.6 μs 1,026.30 μs 533.16 KB
ParserReuse .NET Framework 4.8.1 tags.feature 700.4 μs 2,219.5 μs 6,544.3 μs 45.35 μs 48 KB
Parser .NET 8.0 very_long.feature 4,129.8 μs 1,017.0 μs 2,998.7 μs 3,820.30 μs 2028.3 KB
ParserReuse .NET 8.0 very_long.feature 2,767.4 μs 1,072.6 μs 3,162.7 μs 2,268.20 μs 1559.48 KB
Parser .NET Framework 4.8.1 very_long.feature 2,317.5 μs 1,800.2 μs 5,308.1 μs 1,753.50 μs 2085.16 KB
ParserReuse .NET Framework 4.8.1 very_long.feature 1,339.9 μs 1,793.1 μs 5,287.0 μs 777.25 μs 1600 KB

Results after:

Method Runtime FeatureFile Mean Error StdDev Median Allocated
Parser .NET 8.0 tags.feature 315.1 μs 599.9 μs 1,768.9 μs 142.60 μs 49.76 KB
ParserReuse .NET 8.0 tags.feature 279.9 μs 586.4 μs 1,729.1 μs 102.45 μs 41.9 KB
Parser .NET Framework 4.8.1 tags.feature 430.8 μs 1,263.7 μs 3,726.0 μs 57.60 μs 64 KB
ParserReuse .NET Framework 4.8.1 tags.feature 419.8 μs 1,270.1 μs 3,744.8 μs 43.55 μs 48 KB
Parser .NET 8.0 very_long.feature 2,703.4 μs 527.6 μs 1,555.5 μs 2,572.95 μs 1569.45 KB
ParserReuse .NET 8.0 very_long.feature 2,487.9 μs 526.4 μs 1,552.1 μs 2,180.60 μs 1559.48 KB
Parser .NET Framework 4.8.1 very_long.feature 1,190.0 μs 1,037.8 μs 3,060.0 μs 832.15 μs 1616 KB
ParserReuse .NET Framework 4.8.1 very_long.feature 1,141.2 μs 1,137.8 μs 3,354.9 μs 783.45 μs 1601.52 KB

⚡️ What's your motivation?

Speedup startup time when gherkin parser is used in visual studio plugin or or while compiling.
Also reducing the public needed dependencies of the project, so hopefully less needed (security) updates.

🏷️ What kind of change is this?

  • 🏦 Refactoring/debt/DX (improvement to code design, tooling, etc. without changing behaviour)
  • 💥 Breaking change (incompatible changes to the API)

♻️ Anything particular you want feedback on?

gherkin-languages.json is referenced directly and doesn't get copied in the makefile. I thought this is clearer and avoid duplicate files in the repro, but I'm not 100% sure if there are other reasons for it.

📋 Checklist:

Copy link
Member

@gasparnagy gasparnagy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are great improvements of the performance. I'm fine with it.

@gasparnagy gasparnagy merged commit 04da830 into cucumber:main Dec 24, 2024
2 checks passed
@obligaron obligaron deleted the jsonserializer branch December 24, 2024 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants