Choco returns inconsistent version info for packages

[o-l-a-v]

Checklist

• I have verified this is the correct repository for opening this issue.
• I have verified no other issues exist related to my problem.
• I have verified this is not an issue for a specific package.
• I have verified this issue is not security related.
• I confirm I am using official, and not unofficial, or modified, Chocolatey products.

What You Are Seeing?

Choco 2.2.2 just got released, I have v2.2.0. It's also updated in the repositories:

• https://community.chocolatey.org/packages/chocolatey/2.2.2

I first check for outdated packages with choco outdated, says no packages are outdated.

I then try choco search chocolatey --exact, says newest version of Chocolatey is v2.2.0.

I then do choco search chocolatey which returns a lot of packages, also Chocolatey. Says latest version of Chocolatey is v2.2.2.

I then check the chocolatey.log, the API request when searching exact is:

Invoke-RestMethod -Method 'Get' -Uri "https://community.chocolatey.org/api/v2/Packages()?`$filter=(tolower(Id)%20eq%20'chocolatey')%20and%20IsLatestVersion&semVerLevel=2.0.0"

It says v2.2.0 is the latest version. But if you add "not IsPrerelease" to the query, like the search without exact does, voila:

Invoke-RestMethod -Method 'Get' -Uri "https://community.chocolatey.org/api/v2/Packages()?`$filter=(tolower(Id)%20eq%20'chocolatey')%20and%20not%20IsPrerelease%20and%20IsLatestVersion&semVerLevel=2.0.0"

What is Expected?

I expect a consistent experience when searching and updating a package, with or without --exact. choco should use the API consistently to not get inconsistent results.

I also expect the API to return the latest non-prerelease version of a package, even if not specifying not IsPrerelease.

How Did You Get This To Happen?

See first section.

System Details

• Operating System: Windows 10 22H2 19045.3208
• Windows PowerShell version: 5.1
• Chocolatey CLI Version: 2.2.0
• Chocolatey Licensed Extension version: None
• Chocolatey License type: None
• Terminal/Emulator: None

Installed Packages

Not relevant

Output Log

Not relevant

Additional Context

No response

[TheCakeIsNaOH]

@o-l-a-v I can reproduce this, where choco search chocolatey --exact --source=https://community.chocolatey.org/api/v2 returns v2.2.0, not v2.2.2. However, this appears to be an issue with Chocolatey Community Repository, not with Chocolatey CLI.

[o-l-a-v]

@TheCakeIsNaOH

I agree, the fault seems to be with the API of the community repository. Where do I create an issue for that?

That being said, the CLI passes different filters to the API when using --exact vs. not, ref not IsPrerelease. So could be mitigated by the CLI too it seems.

[o-l-a-v]

It seems to return v2.2.2 with or without not IsPrerelease today, strange.

[pauby]

I can't reproduce this.

When there is a mismatch about what is in the Chocolatey Community Repository and what is reported by Chocolatey CLI, it's generally down to caching. It's something we are working on and mostly is not an issue, but does rear it's head every so often.

[o-l-a-v]

@pauby

Cache where, in the API?

This is still a bug IMO, we just don't know what caused it, and we can't reproduce it today.

Why not include not IsPrerelease in --exact search (and coho outdated and choco upgrade), as it seems to make a difference in edge cases?

[cobrabr]

I'm having the exact same problem with another package, k-litecodecpackfull:

Funnily enough, Chocolatey GUI doesn't have this problem — I used the "Check for outdated packages (forced)" button and it successfully picked up this package as outdated:

I was trying to find a way to run the same "forced" check for outdated packages on the CLI, but choco outdated --force doesn't do anything.

[gep13]

@o-l-a-v said...

Cache where, in the API?

There are several layers of caching in play with Chocolatey and Chocolatey GUI. Let me quickly recap them. NOTE: The following assumes that you are using 2.x.x of both Chocolatey CLI and GUI.

1. One of the first things that Chocolatey GUI does on startup is to figure out which packages are currently outdated. This happens each time to open the application, although it is possible to control this action through a feature. So that this information doesn't have to be re-calculated each time, Chocolatey GUI caches the results in an xml file, which is read when the application loads. This information is stored for a configurable period of time, and it can be forced to be recalculated using the Check for outdated packages (forced) button. Additionally, you can manually purge this file using the purge outdated packages button.
2. On top of the built in caching that Chocolatey GUI provides, Chocolatey CLI itself also caches the results of HTTP queries. By default, the results of these queries are stored within the C:\ProgramData\ChocolateyHttpCache folder. To clear this cache, you can use the choco cache command, or you can make use of the --ignore-http-cache option which is available on all Chocolatey CLI commands.
3. Then, at the website level (the API), there are a number of different caches in play. For example, we have a shared key/value cache, as well as search index caching (which currently is web host dependent). These caches are used when Chocolatey GUI, or Chocolatey CLI, make an HTTP request. Part of the problem here, which @pauby alluded to above, is that the search index caching is done on a per web host basis. As a result, it is possible to get one response from web host, and a different one from another web host. This is something that we are actively working on correcting, but we don't have anything to share on this at the moment. Typically, caching issues at this level tend to "fix themselves" after a period of time, once the cache expires, and the information is repopulated.

@o-l-a-v said...

I agree, the fault seems to be with the API of the community repository. Where do I create an issue for that?

Issues for the Chocolatey Community Repository would be raised here: https://github.com/chocolatey/home/issues

However, the issue that you are seeing it likely covered by this issue: chocolatey/home#8 Feel free to add additional information to there.

@o-l-a-v said...

Why not include not IsPrerelease in --exact search (and coho outdated and choco upgrade), as it seems to make a difference in edge cases?

I honestly do not believe that this is required. Almost everywhere in Chocolatey, everything defaults to the stable version numbers. You have to opt-in to get prerelease versions. As such, explicitly stating not IsPrerelease should not be required on queries. The differences that you are seeing is likely due to the cached values that are stored for the "normal" queries, compared to the lesser executed queries. Eventually, both of these would result in getting the same result.

Hopefully the above helps to understand what is going on, and what the different caching solutions are in place for Chocolatey GUI and CLI. Let me know if you have any questions about any of the above.

[github-actions]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?
This issue will be closed in 14 days if it continues to be inactive.

[github-actions]

Dear contributor,

As this issue seems to have been inactive for quite some time now, it has been automatically closed.
If you feel this is a valid issue, please feel free to re-open the issue if / when a pull request
has been added.
Thank you for your contribution.

[o-l-a-v]

AFAIK this is not fixed, so please open again.

Going stale != Fixed.

[pauby]

@o-l-a-v As pointed out in the comments above, this is a caching issue which is related to chocolatey/home#8.

Going stale != Fixed.

After no response to the previous two comments on 10 August and 10 September, the issue is closed automatically. This is the process we have in place for this repository.