Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nexus has deprecated some NuGet v2 optimizations that Chocolatey CLI uses #3506

Open
pauby opened this issue Sep 2, 2024 · 15 comments
Open

Comments

@pauby
Copy link
Member

pauby commented Sep 2, 2024

NOTE: This issue is a work in progress and will be updated as we have additional information and experiences from customers and community members.

In version 3.71.0 of Sonatype Nexus Repository, Sonatype deprecated some NuGet v2 optimizations that Chocolatey CLI uses:

NuGet v2 Client Compatibility

The supported subset of legacy Nuget v2 protocol in H2 and PostgreSQL environments is the same as that supported by Microsoft's NuGet Gallery, nuget.org. Use cases that rely on deprecated parts of the NuGet v2 API are not supported, including many common Chocolatey use cases and some custom OData queries.

As part of the deprecation, Nexus still advertises that it supports NuGet v2 search for it's NuGet v3 repository, which Chocolatey CLI will fall back to using and tries to optimize. This may lead to issues.

Sonatype Nexus Repository NuGet v3 Search Issues

If you are having issues during search of a NuGet v3 repository in Sonatype Nexus Repository, please disable usePackageRepositoryOptimizations by running choco feature disable --name='usePackageRepositoryOptimizations'. This will stop Chocolatey CLI from optimizing queries to the NuGet v2 search endpoint, but may lead to more queries being made and therefore longer search times.

Further Help

See the Sonatype Nexus Repository documentation:

Customer Support

If you are a customer with Support and Maintenance, and require further help, please reach out to the Chocolatey Support team. You will find the options available to you by running choco support from the command line.

Community Support

If you are using the open-source edition of Chocolatey CLI, please reach out on our Community Hub.

Related Issues and Discussions

@pauby pauby changed the title Nexus has deprecated some NuGet v2 functionality Nexus has deprecated some NuGet v2 optimizations that Chocolatey CLI uses Sep 2, 2024
@pauby pauby pinned this issue Sep 2, 2024
@ViToRiO92
Copy link

I can confirm this issue. We are also running in to problems to install software with Nexus as a repository, since we updated to the new version of the repository.

chocolatey cli version: 2.3.0
Nexus Repository OSS version: 3.72.0-04

the workaroud metioned above is working

choco feature disable --name='usePackageRepositoryOptimizations'

A fix for the chocolatey cli would be awesome.

@pauby
Copy link
Member Author

pauby commented Oct 1, 2024

A fix for the chocolatey cli would be awesome.

Chocolatey CLI is not 'broken', so there is nothing to fix. Nexus deprecated some features that Chocolatey CLI used. Disabling package optimizations works around the changes made.

@liukonen
Copy link

Recently ported my Nexus from 3.70.3 with an OrientDB, to an H2 DB, then the "latest" docker image.
I had to rebuild my proxy which works great again, but my "self-hosted" project/repo I have is now no longer returning values, AND causing all of my other software items I have installed to also error out when attempting to do a choco outdated, even having this repo link in a lower priority AND having the usePackageRepositoryOptimizations disabled
Repo returns a NuGet.V2.Deprecated on the self hosted option, with no way, that I can tell, to switch my self hosted software over to v2 again

@daemenseth
Copy link

@pauby

Is there any information on whether a new release of Chocolatey is planned that no longer relies on the deprecated parts of the NuGet v2 API?

We are facing issues with our Nexus repository, which follows Microsoft's guidance on this matter. More information can be found in this link.

According to Sonatype's documentation:

The supported subset of the legacy NuGet v2 protocol in H2 and PostgreSQL environments is the same as that supported by Microsoft's NuGet Gallery, nuget.org. Use cases that rely on deprecated parts of the NuGet v2 API are not supported, including many common Chocolatey use cases and some custom OData queries.

It seems likely that more NuGet repositories will adopt this approach, making the reliance on deprecated API parts increasingly problematic.

While the workaround choco feature disable --name='usePackageRepositoryOptimizations' can mitigate the issue, it’s not ideal as it requires specific client-side settings.

Would it be possible to get an update on this?

@pauby
Copy link
Member Author

pauby commented Nov 13, 2024

Is there any information on whether a new release of Chocolatey is planned that no longer relies on the deprecated parts of the NuGet v2 API?

Chocolatey CLI version 2.0.0 introduced NuGet v3 API support 18 months ago. Can you be more specific?

It seems likely that
While the workaround choco feature disable --name='usePackageRepositoryOptimizations' can mitigate the issue, it’s not ideal as it requires specific client-side settings.

To be fair, it requires a single command to be run to disable the feature. If you are a Chocolatey for Business user you can make that change through a single Deployment Step. Open-source users can add the command to the method they used for deployment.

Would it be possible to get an update on this?

To be very clear, Chocolatey CLI does not rely on NuGet v2. It supports it. It will happily work with NuGet v3.

Chocolatey CLI will continue to support NuGet v2 for some time as many community members and customers still rely on it, despite repository managers removing support.

@daemenseth
Copy link

daemenseth commented Nov 13, 2024


Issue Summary

When using a Nexus-hosted repository as a NuGet source, it should serve packages via either v2 or v3 protocols, depending on the URL configuration. However, inconsistencies are observed when performing different operations (choco install vs. choco upgrade).

Steps to Reproduce

  1. Configure Nexus to host a NuGet repository.
  2. Specify the URL with /index.json to ensure NuGet v3 compatibility.
  3. Run the following commands:
    • choco install <package-name> (works without issues using the v3 URL).
    • choco upgrade <package-name> (unexpectedly falls back to v2, causing an error).

Observed Behavior

choco install

The installation process works correctly, and the logs indicate that the v3 URL is being used:

[NuGet]   GET http://localhost:8081/repository/nuget-hosted/v3/content/inp-central-7zip/24.8.0/inp-central-7zip.24.8.0.nupkg
[NuGet]   OK http://localhost:8081/repository/nuget-hosted/v3/content/inp-central-7zip/24.8.0/inp-central-7zip.24.8.0.nupkg 2067ms

choco upgrade

During an upgrade, the request unexpectedly reverts to NuGet v2, resulting in a failure:

[NuGet]   CACHE http://localhost:8081/repository/nuget-hosted/$metadata
[NuGet]   GET http://localhost:8081/repository/nuget-hosted/Packages()?$filter=(tolower(Id) eq 'inp-central-7zip') and IsLatestVersion&semVerLevel=2.0.0
[NuGet]   BadRequest http://localhost:8081/repository/nuget-hosted/Packages()?$filter=(tolower(Id) eq 'inp-central-7zip') and IsLatestVersion&semVerLevel=2.0.0 2ms
Unable to connect to source 'http://localhost:8081/repository/nuget-hosted/index.json':
Failed to fetch results from V2 feed at 'http://localhost:8081/repository/nuget-hosted/Packages()?$filter=(tolower(Id)%20eq%20'inp-central-7zip')%20and%20IsLatestVersion&semVerLevel=2.0.0' with following message: Response status code does not indicate success: 400 (Bad Request).

nuget group

Using a NuGet group repository in Nexus resolves this issue; the upgrade correctly identifies and uses the v3 repository:

[NuGet]   CACHE http://localhost:8081/repository/nuget-group/v3/registration/5/inp-central-7zip/index.json
inp-central-7zip v24.8.0 is the latest version available based on your source(s).
[NuGet]   CACHE http://localhost:8081/repository/nuget-group/v3/registration/5/inp-central-7zip/index.json
[NuGet] Resolving dependency information took 0 ms

issues

In previous versions (2.0), NuGet groups had dependency resolution issues, which led us to avoid using them. I will test further to verify if these dependency issues persist in the current version.

we had this issue with search and dependencies here

Expected Behavior

For choco upgrade to consistently use the v3 URL when specified, as choco install does.


@martinisoft1
Copy link

@pauby
In your response to @daemenseth you state that we can use V3 with chocolatey as the way forward.
Do you happen to know if V3 is already used in production with chocolatey cli?
The thing is that when we conduct tests with Nexus V3 repo's and Choco pretty basic stuff stops to function.
As an example, when I disconnect Chocolatey CLI from V2 and add it to a V3 repo on the same Nexus the Dependency chain is broken.
When installing a Chocolatey package that has Dependencies declared in the .nuspec they are completely ignored.
I am not in the position right now to conduct the test with an other backend then Nexus so in theory this could be a Nexus specific Issue. But the fact that most whitepapers regarding the setup of a Chocolatey Repo are steering to V2 and the community Repo is still only serving V2 made me thinking that Chocolatey and V3 repo's are not that common yet.
Any thoughts on this?

thanks in advance,

Martijn.

@pauby
Copy link
Member Author

pauby commented Nov 29, 2024

When installing a Chocolatey package that has Dependencies declared in the .nuspec they are completely ignored.

Do the dependencies have a version number declared? If they do not, then this is a bug in Nexus that we identified and which they are aware of. Unfortunately, I can't point you to any public facing issue, as Sonatype no longer has that facility.

You can look at the requests going back and forth between Chocolatey CLI and Nexus (use something like Fiddler). You will see Nexus not returning the dependency chain when no version number is present.

@pauby
Copy link
Member Author

pauby commented Nov 29, 2024

But the fact that most whitepapers regarding the setup of a Chocolatey Repo are steering to V2

NuGet v3 support for Chocolatey CLI was added in version 2.x, released last year. That's a small blip in the lifespan of the product (which was original released in 2011). So it is going to take time for the guides to be updated, blog posts written, etc.

the community Repo is still only serving V2

This is simply because of historical reasons and nothing more. There is a plan in place to add v3 support to the Chocolatey Community Repository. As there are no plans to remove NuGet v2 support from Chocolatey CLI, and the Chocolatey Community Repository supports Chocolatey CLI, there is no rush on this.

made me thinking that Chocolatey and V3 repo's are not that common yet.
Do you happen to know if V3 is already used in production with chocolatey cli?

Chocolatey CLI doesn't phone home, so we have no data that I can point to that tells me how widely used NuGet v3 repositories are. I can tell you that we have customers who are using it. How many, how widely and in what way I don't have information on.

@martinisoft1
Copy link

@pauby thank you for the response. Regarding the dependencies, no I am not declaring version numbers at the moment. Since there is in fact no version dependency within my dependency chain this would be undesirable.
This in fact means that V3 is not a valid option for us to overcome the original issue namely that Nexus has deprecated some NuGet v2 optimizations until this issue is resolved. V2 in combination with disabling Package Optimizations would then be the only workable option as far as I can see. Wouldn't it be a good Idea to start adding some disclaimers in the Chocolatey Docs that the Chocolatey and Nexus Sonatype Combo is not at its best at the moment? This because I presume that Nexus would be the go to choice for many since It is well known, widely used and mentioned as the first option in most Chocolatey docs. Apart from who needs to fix which issue, bottom line is that the end users are likely to have issue's when going this route without prior knowledge of this (as we did). Now regarding the workable Combo, what side affects can I expect from setting "usePackageRepositoryOptimizations" to disabled?
The docu state the following:

Turn on optimizations for reducing bandwidth with repository queries during package install/upgrade/outdated operations. Should generally be left enabled, unless a repository needs to support older methods of query. When disabled, this makes queries similar to the way they were done in Chocolatey v0.10.11 and before.

Could you elaborate on this? What is exactly changed in the behavior when having it Disabled v/s Enabled?
Will it increase the load on my Nexus and will it effect bandwidth usage for instance?

thanks again,

Martijn.

@pauby
Copy link
Member Author

pauby commented Dec 6, 2024

Wouldn't it be a good Idea to start adding some disclaimers in the Chocolatey Docs that the Chocolatey and Nexus Sonatype Combo is not at its best at the moment?

That is one of the purposes of this issue. But to be clear, this is a bug in Sonatype Nexus with a fix / workaround and I want to be careful that we are not simply documenting issues in, and providing support for, a third-party product. There is a balance to be struck for open-source products.

Now regarding the workable Combo, what side affects can I expect from setting "usePackageRepositoryOptimizations" to disabled?

Disabling the feature will likely see an increase in the number of queries being made to the Sonatype Nexus repository to gather the same information. I say likely as every environment is unique.

@daemenseth
Copy link

That is one of the purposes of this issue. But to be clear, this is a bug in Sonatype Nexus with a fix / workaround and I want to be careful that we are not simply documenting issues in, and providing support for, a third-party product. There is a balance to be struck for open-source products.

@pauby can you point to a github issue or ticketnr inside Sonatype about this issue. So we can ask there for more information. I have a issue open there for this issue so I can point to this.

@pauby
Copy link
Member Author

pauby commented Dec 6, 2024

can you point to a github issue or ticketnr inside Sonatype about this issue. So we can ask there for more information. I have a issue open there for this issue so I can point to this.

Unfortunately not. Sonatype removed public access to their bug tracker.

@TheCakeIsNaOH
Copy link
Member

Sonatype does have a public issue tracker for Nexus here: https://github.com/sonatype/nexus-public/issues

@pauby
Copy link
Member Author

pauby commented Dec 6, 2024

Sonatype had an internal bug tracker that had public read access. My understanding is that it still exists but public access for non-customers was removed. I was aware the GitHub repository existed for open-source.

My understanding may be outdated and perhaps they exclusively use GitHub now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants