Vector Index support in azcosmos

[ntttorris]

Description

I try to use the following code to create a container and the index. But I cannot find the vector index setting and field.

resp, err := db.CreateContainer(
		ctx,
		azcosmos.ContainerProperties{
			ID: "vector-index-test",
			PartitionKeyDefinition: azcosmos.PartitionKeyDefinition{
				Paths: []string{"/partition"},
			},
			IndexingPolicy: &azcosmos.IndexingPolicy{
				IndexingMode: azcosmos.IndexingModeConsistent,
				Automatic:    true,
				IncludedPaths: []azcosmos.IncludedPath{
					{
						Path: "/numericId/?",
					},
					{
						Path: "/tags/*",
					},
					{
						Path: "/external/?",
					},
				},
				ExcludedPaths: []azcosmos.ExcludedPath{
					{
						Path: "/embedding/*",
					},
				},
			},
			UniqueKeyPolicy: &azcosmos.UniqueKeyPolicy{
				UniqueKeys: []azcosmos.UniqueKey{
					{
						Paths: []string{"/numericId"},
					},
				},
			},
		},

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @MehaKaushik @Pilchie @wmengmsft.

[ntttorris]

/assign @jliusan

[Pilchie]

Thanks - also tagging @analogrelay and @TheovanKraay. Note that while it's relatively easy to add the create options here, that doesn't solve the fact that the go sdk doesn't support cross partition query aggregation, which is often required for vector queries.

Can you tell us more about your planned use case? Would you be able to make use of vector search without any client side aggregation of results from multiple partitions?

[ntttorris]

Thanks - also tagging @analogrelay and @TheovanKraay. Note that while it's relatively easy to add the create options here, that doesn't solve the fact that the go sdk doesn't support cross partition query aggregation, which is often required for vector queries.

Can you tell us more about your planned use case? Would you be able to make use of vector search without any client side aggregation of results from multiple partitions?

Thanks for replying.
In my use case, doing vector search without client side aggregation is OK. I'm thinking about searching from several different partitions then do a reranking procedure with local reranking model with python. Could you just support vector search in single partition in the azcosmos sdk? which will be really helpful.

[Pilchie]

We'll take a look at it (probably next week, when more folks are back from holidays), and see what the effort looks like.

Note - if you're already re-ranking in Python, you could use the Python SDK, which does have this support already.