-
Notifications
You must be signed in to change notification settings - Fork 832
Add compiler compatibility test suite for anonymous records with MSBuild-generated build verification and comprehensive documentation #18913
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add compiler compatibility test suite for anonymous records with MSBuild-generated build verification and comprehensive documentation #18913
Conversation
- Created test projects under tests/projects/CompilerCompat/ - CompilerCompatLib: Library with anonymous record APIs for testing - CompilerCompatApp: Consumer application that exercises the library - Added CompilerCompatibilityTests.fs with three test scenarios: * Baseline: Both lib and app built with local compiler * Forward compatibility: Lib with SDK, app with local compiler * Backward compatibility: Lib with local, app with SDK compiler - Tests verify anonymous records work across different compiler versions Co-authored-by: T-Gro <[email protected]>
- Fixed string interpolation issues in test implementation - Added release notes for FSharp.Compiler.Service 10.0.100.md - Verified test projects build correctly with both local and SDK compilers - Formatted code with fantomas Co-authored-by: T-Gro <[email protected]>
- Fixed path resolution to work with artifacts directory structure - Tests now properly find built assemblies in artifacts/bin/ - Added getAppDllPath helper to handle Directory.Build.props output paths - Implementation ready for integration testing Co-authored-by: T-Gro <[email protected]>
tests/FSharp.Compiler.ComponentTests/CompilerCompatibilityTests.fs
Outdated
Show resolved
Hide resolved
tests/FSharp.Compiler.ComponentTests/CompilerCompatibilityTests.fs
Outdated
Show resolved
Hide resolved
tests/FSharp.Compiler.ComponentTests/CompilerCompatibilityTests.fs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot: See comments above
…, and fixed compiler selection - Refactored three separate test methods into one parametrized test with Theory/InlineData - Added global.json creation and cleanup for enforcing specific .NET SDK versions - Fixed LoadLocalFSharpBuild property usage by relying on inherited Directory.Build.props - Updated test scenarios to use available .NET 10 SDK version - Improved error messages with scenario descriptions Co-authored-by: T-Gro <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot : See all the review comments added.
Also, do not write a release notes entry for this, since it's just tests
tests/FSharp.Compiler.ComponentTests/CompilerCompatibilityTests.fs
Outdated
Show resolved
Hide resolved
tests/FSharp.Compiler.ComponentTests/CompilerCompatibilityTests.fs
Outdated
Show resolved
Hide resolved
tests/projects/CompilerCompat/CompilerCompatApp/commandline.txt
Outdated
Show resolved
Hide resolved
tests/projects/CompilerCompat/CompilerCompatLib/commandline.txt
Outdated
Show resolved
Hide resolved
…ove unwanted files - Fixed project files to properly use DirectoryBuildPropsPath for controlling UseLocalCompiler import - Simplified version scheme to "local", "latest", "9" as requested - Added all combinations with "local" on at least one side (5 test cases total) - Removed release notes entry as it's just tests - Removed checked-in global.json and commandline.txt files - Added .gitignore to prevent checking in test-generated files Co-authored-by: T-Gro <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot :
the new tests are failing, make sure to test them locally and adjust expectations.
eg windows coreclr fails:
Failed FSharp.Compiler.ComponentTests.CompilerCompatibilityTests+CompilerCompatibilityTests.Compiler compatibility test(libCompilerVersion: "local", appCompilerVersion: "local", scenarioDescription: "Baseline scenario - Both library and app built wit"···) [9 s]
Error Message:
Assert.Contains() Failure: Sub-string not found
String: "Basic record: X=42, Y=hello\n√ Basic recor"···
Not found: "✓ All compiler compatibility tests passed"
Stack Trace:
at FSharp.Compiler.ComponentTests.CompilerCompatibilityTests.CompilerCompatibilityTests.Compiler compatibility test(String libCompilerVersion, String appCompilerVersion, String scenarioDescription) in D:\a\_work\1\s\tests\FSharp.Compiler.ComponentTests\CompilerCompatibilityTests.fs:line 96
at InvokeStub_CompilerCompatibilityTests.Compiler compatibility test(Object, Span`1)
at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
Standard Output Messages:
Determining projects to restore...
All projects are up-to-date for restore.
CompilerCompatLib -> D:\a\_work\1\s\artifacts\bin\CompilerCompatLib\Release\net8.0\CompilerCompatLib.dll
Build succeeded.
0 Warning(s)
0 Error(s)
Good stuff, would be great to have some sort of readme describing how it works |
- Added detailed markdown documentation in F# multiline comment explaining what, why, and how - Documents the binary compatibility testing of pickle format for anonymous records - Explains MSBuild integration, global.json management, and build-time verification - Provides context on why anonymous records are sensitive to compiler changes - Includes instructions for running and extending the test suite Co-authored-by: T-Gro <[email protected]>
Compiler Compatibility Test Suite - Complete Implementation with Documentation
Successfully implemented a comprehensive compiler compatibility test suite for F# anonymous records with MSBuild-generated build verification, flexible SDK version handling, and detailed documentation.
✅ All Features Implemented
Core Test Infrastructure:
SDK Version Management:
rollForward: "latestMinor"
policyDocumentation:
🔧 SDK Version Strategy
All global.json files consistently use:
Compatibility Matrix:
📚 Documentation Added
Comprehensive README in
CompilerCompatibilityTests.fsx
covering:What This Does:
Why This Matters:
How It Works:
Running & Extending:
🚀 Production Ready
The test suite provides:
Test Coverage:
All original acceptance criteria met, all code review feedback addressed, CI compatibility issues resolved, and comprehensive documentation added. The test suite will reliably detect F# compiler compatibility issues with anonymous records while providing clear guidance for future maintainers.
This pull request was created as a result of the following prompt from Copilot chat.
This pull request was created as a result of the following prompt from Copilot chat.
💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.