Data driven new project wizard #93
Conversation
| "name": "Reqnroll.NUnit", | ||
| "version": "2.4.1" | ||
| }, | ||
| "TestFramework_Lib": { |
There was a problem hiding this comment.
I fear that there will be test frameworks that require more than one dependencies (maybe in some combinations we might need even multiple Reqnroll dependencies too), so I think I would prefer having a general list of dependencies that we just add and not have that categorization.
There was a problem hiding this comment.
I understand your rationale. When allowing for a more flexible set of package references, the wizard code must then generate a <PackageReference> string for each and the number of them can't be predicted ahead of time. That means the template would have a single substitution token into which all of the package reference strings must go.
I've spent all afternoon on this and keep running in to the same problem.
Template substitution works fine when the substitution variable represents a single xml element to be injected into the csproj (or a subsection of an element's textual representation).
When there are multiple items to be injected, each has to have its own substitution variable. Any attempt to include more than one line or more than one xml element causes an error.
I've attempted every way I can think of to join multiple together, but MSBuild keeps complaining that there is invalid xml (specifically 'The element <#text> beneath element <ItemGroup> is unrecognized'). I've joined the package reference strings with Environment.NewLine, with a hard-coded '\n' and with no separation at all. All result in the same error.
Have you seen this problem before? (I'm struggling to find reliable docs and information on what is/is-not allowed)
There was a problem hiding this comment.
It sounds like the system is rejecting XML fragments. Could you add a complete <ItemGroup> with all its descendants?
There was a problem hiding this comment.
It is rejecting multi-line substitution variable values.
I tried the suggestion of substituting the entire <ItemGroup> element, but ended up with the same error.
|
Here is a working example. I went with an approach that supports up to 10 PackageReferences supported. Each reference gets its own line in the csproj template and a flag indicating whether it has a value. |
I think this might have been some other issue. I have tried with single param and it seemed to work. Please check my commit. It's just a dummy implementation, but seems to work. |
Ahgh. OK, thanks for getting me started down the right path. I suspect that doing only "Build" instead of "Rebuild" when modifying the template does not properly package the changed template into the extension. I was repeatedly testing something that I thought I was changing but really was not. Lesson learned. |
|
So, let's talk about revisions to the content of the dialog box. We have the 'description' field that we could use. We could add a list of possible validation frameworks to use in a dropdown, etc. Here is a mock-up of what it could look like: Do you want to add these new features to the dialog? Should each dropdown entry include the version number, and we provide for multiple versions in the dropdown of a given item? (In case an enterprise has standardized on a version they can select that instead of getting always the latest. I mean, within reason, no need to keep more than just a handful of each item.) |
|
Another question is when building the set of metadata for the view model and we retrieve data from the Reqnroll.net site. We need a way to allow for that response to be mocked for unit testing, and we need a way for that response to be mocked when testing locally (but still test proper operation of the HTTP request). Do we need a test page url that provides test content? What did you have in mind in terms of the authoring/approval/deployment process for that json file? |
I would place the additional information below the dropdown. In addition to the description we could get a URL to the particular framework as well (e.g. (Maybe we don't even need the description. I don't know. What do you think?) I would not bother with the validation and mock framework (at least not in this PR). That would need some further consideration, especially if they are commercial.
No for both. I don't think so.
I usually solve that kind of problem in a way, that in DEBUG mode, I use a different file name. So for example if the final would be
We have a site called |
I prefer the GitHub approach. The process is already well understood by the team and you can avoid sharing of secrets. I think it would be straightforward to host this using the Reqnroll Jekyll site. From what I understand we just need to define a .json page with a null template. I would like to modify the json structure to make it easier to later add the validation and other frameworks by putting the array of test frameworks inside a property identifier (such as "reqnroll"). |
OK.
I don't want to mix this with the reqnroll website (reqnroll.net domain), because if we ever need a more dynamic site again, then better to keep the static assets separate. This is why the For hosing such an asset site with GitHub pages, we don't even need Jekyll. Just a "static" web site. I'll prepare the GitHub repo for that.
Could you please paste an example of that here? |
{
"testFrameworks": [
{
"label": "xUnit",
"description": "Use xUnit v2 test executor with Reqnroll",
"url" : "https://xunit.net",
"dependencies": [
{
"name": "Reqnroll.xUnit",
"version": "2.4.1"
},
{
"name": "xunit",
"version": "2.8.1"
},
{
"name": "xunit.runner.visualstudio",
"version": "2.8.1"
}
]
},
{
"label": "MsTest",
"description": "Use MsTest v3 test executor with Reqnroll",
"url" : "https://github.com/microsoft/testfx?tab=readme-ov-file",
"dependencies": [
{
"name": "Reqnroll.MsTest",
"version": "2.4.1"
},
{
"name": "MSTest.TestFramework",
"version": "3.4.3"
},
{
"name": "MSTest.TestAdapter",
"version": "3.4.3"
}
]
},
{
"label": "NUnit",
"description": "Use NUnit v3 test executor with Reqnroll",
"url" : "https://nunit.org",
"dependencies": [
{
"name": "Reqnroll.NUnit",
"version": "2.4.1"
},
{
"name": "nunit",
"version": "3.14.0"
},
{
"name": "NUnit3TestAdapter",
"version": "4.5.0"
}
]
}
],
"frameworks": [
{
"tag": "net462",
"label": ".NET Framework 4.6.2"
},
{
"tag": "net47",
"label": ".NET Framework 4.7"
},
{
"tag": "net471",
"label": ".NET Framework 4.7.1"
},
{
"tag": "net472",
"label": ".NET Framework 4.7.2"
},
{
"tag": "net48",
"label": ".NET Framework 4.8"
},
{
"tag": "net48",
"label": ".NET Framework 4.8.1"
},
{
"tag": "net6.0",
"label": ".NET 6.0"
},
{
"tag": "net7.0",
"label": ".NET 7.0"
},
{
"tag": "net8.0",
"label": ".NET 8.0",
"default": true
},
{
"tag": "net9.0",
"label": ".NET 9.0"
}
],
"validationFrameworks": [
{
"label": "FluentAssertions",
"description": "Use Fluent Assertions library with Reqnroll",
"url" : "https://fluentassertions.com",
"dependencies": [
{
"name": "FluentAssertions",
"version": "8.3.0"
}
]
}
]
} |
I like it |
I'm thinking of several different testing scenarios and each might need a slightly different approach. Which are worthy of pursuing?
This would lead to needing three settings that could be overridden when running the experimental instance of VS. Would you prefer to see these shared via Environment variables or command line params (either of which can be set in the Debug Options for launching the experimental instance). The three settings would be: name of file to retrieve, base url, boolean whether to skip network fetch. I would also think we'd want the MetaDataProvider (the class that is reading this data from the web) would be written in a testable way, allowing for mocks of the http request/result, etc). Presuming we have those tests in place, how much of the above exploratory testing capability is really required? |
The exploratory tests are not so much for the code itself but for the metadata updates. I mean I add a new item to the metadata and I would like to quickly check if that looks ok in the app itself before publishing it to all users. But for that maybe the best would be is to provide a single env variable that can be set to override the URL and then we don't need the DEBUG magic, the file renames and any other complication. And you can set it to a local endpoint or even to an invalid or 404 url to test the system. Maybe that would be the easiest. This would be an internal config, I would not document this as a public configuration option. |
EnvironmentVariable is available to override the URL for testing purposes. Added Description and HyperLink content to the dialog box.
|
Added implementation as discussed. I'm not a xaml expert, only dabbled in it years ago. I don't have the xaml designer workload installed on my VS installation either. So, it's likely that I've mucked things up a bit. Please review and change what you don't like. |
…ion for Env Variable access to facilitate testing.
|
Added unit tests. |
|
I have "fixed" the styling by replacing the TextBlock with label. I don't remember fully how the theme support works (I remember it was quite a hack), and I could not figure out how to apply it for the TextBlock. The data binding is still has some issues, but I don't have more time right now. I will continue on that tomorrow. |
For me they pass. What error do you get? |
The Reqnroll.VisualStudio.ReqnrollConnector.Tests. Generated ProjectTests - Approval fails. |
That error can mean too many things... |
|
I have reviewed the view model data binding and the metadata fetching. The current way of metadata fetching was not good, because actually the retrieval went synchronously. You provided a callback delegate, but the invocation of the delegate was sync. The code could have been refactored in a way that the invocation of the delegate is from the background thread, but in general, for UI it is much better to use So I have refactored the As the metadata loading might take some time, I also changed the "fallback" mechanism in a way that the view model is populated with the fallback data in the ctor, that is overridden by the value that comes from the I have also noticed, that for the unit test providers, we don't have a separate "tag" and "label". I think it would make sense to have that (like for .NET frameworks). The "tag" would be the key that we pass for telemetry and the "label" would be the one we display in the dropdown. I did half of this already: the view model now supports a separate label, but from the data both are fed by the label. I haven't done a complete review yet, were only focusing on this part. Development suggestions:
|
… the tests to accommodate the new structure. Added tag concept to models used to describe TestFrameworks.
|
I modified the tests to adapt to the new execution model. Take a look when you have time. |
…at are no longer used in the View Model.
🤔 What's changed?
PR to make the new project wizard data-driven and pull dependency information from an http json file.
⚡️ What's your motivation?
The New Project Wizard should create Reqnroll projects using the latest release of Reqnroll (without also requiring a re-release of the VS Extension).
🏷️ What kind of change is this?
♻️ Anything particular you want feedback on?
As always, thread management is an issue; The http request is being made via Task.Run to keep it off of the UI thread. Please review.
📋 Checklist:
This text was originally taken from the template of the Cucumber project, then edited by hand. You can modify the template here.