Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade existing operations to Milvus API v2 #10

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

wiljanslofstra
Copy link

The v1 endpoints are to be deprecated in the near future, and the v2 endpoints are the recommended ones to use.

Source: https://milvus.io/api-reference/restful/v2.5.x/About.md

I've updated the operations and resources to support the Milvus V2 API (https://milvus.io/api-reference/restful/v2.5.x/About.md).

A few notable changes:

  • v2.3.3 does not support the V2 API. 2.3.14 does, and I can't find when the actually added it (the release notes do not mention it).
    • However, in version 2.4 there are also changes in the return codes (0 instead of 200). So the test are currently using version 2.4. But it should be usable with something like version 2.3.14 and up.
  • Vector delete now works with a filter, instead of an id.
  • Vector query requires the filter field.
  • Collection list and describe changed to POST methods.
  • Vector search changed vector to data. params has become searchParams and annsField is required (for example "vector").
  • Collection create changed parameter primaryField to primaryFieldName and vectorField to vectorFieldName.
  • Vector inserts require an id (added in tests).

This is definitely a breaking change.

When the above changes are fine to be merged. I might add a few new endpoints, most notable the database endpoints, since I see a few issues mentioning it.

}

public function __construct(
protected ?string $dbName = null,
) {
$this->body()->setJsonFlags(JSON_FORCE_OBJECT);
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to set this flag, because when there isn't any parameter given, the defaultBody array is empty. And when Saloon sends it to Milvus, the default json encoded request body is []. Milvus returns errors because it requires an object.

expect($response->json('code'))->toEqual(200);
// Status 0 and empty data is a success response
expect($response->status())->toEqual(200);
expect($response->json('code'))->toEqual(0);
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that code 0 is now a success response, per the documentation.
E.g. https://milvus.io/api-reference/restful/v2.5.x/v2/Collection%20(v2)/List.md

@@ -113,30 +120,33 @@
$this->milvus->collections()->create(
collectionName: 'collection_test',
dimension: 128,
// L2 sets the distance between 0 and ∞ (0 is the closest)
metricType: 'L2',
Copy link
Author

@wiljanslofstra wiljanslofstra Dec 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find any reference that the metric type has changed between versions. But when I ommit this parameter, the distance in the assertion is 1. My assumption is that the default metric type changed to something like cosine.

Different metric types and their distance value ranges: https://milvus.io/docs/metric.md

@wiljanslofstra
Copy link
Author

Added Github actions to test (PHP 8.2 and 8.3) and analyse.

https://github.com/wiljanslofstra/milvus/actions/runs/12536214078

@HelgeSverre
Copy link
Owner

Great work! ⭐

I see some of the tests are failing locally (might be because it does not "reset" the database after each test or runs in parallel by default on my machine), will look closer into that later.

Mind adding php 8.1 to the test workflow file as well?, I think it (8.1) should work without any code modification, but have not tried it myself, if it causes issues just ignore it for now.

@HelgeSverre HelgeSverre self-assigned this Jan 3, 2025
@wiljanslofstra
Copy link
Author

@HelgeSverre Thanks!

I've added PHP 8.1 and 8.4 to the Github actions matrix for testing. I had to downgrade the PHP version in the composer.json, but that doesn't cause any issues with the other dependencies.

https://github.com/wiljanslofstra/milvus/actions/runs/12620996795

@HelgeSverre
Copy link
Owner

Was this good to go @wiljanslofstra ?

@wiljanslofstra
Copy link
Author

@HelgeSverre It is 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants