GetItemLinqQueryable ... ToFeedIterator invalid response from queryDocuments

[Socolin]

Hi,

The route "/dbs/:dbId/colls/:collId/docs": queryDocuments, is not responding the expected body when using the following example with dotnet sdk

            var iterator = container.GetItemLinqQueryable<SomeObject>()
                .Where(x => ids.Contains(x.Id))
                .ToFeedIterator();

The responses received is (https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CosmosQueryExecutionContextFactory.cs#L217)

{"Documents":[{"id":"1","someData":["a"],"_etag":"93f16095-8d53-4dc8-b8b7-11b9c29d0d60","_rid":"gAAGAIAAAAEBAAAAAAAAAA==","_self":"/dbs/gAAGAA==/colls/gAAGAIAAAAE=/docs/gAAGAIAAAAEBAAAAAAAAAA==/","_ts":1608932852},{"id":"2","someData":["b"],"_etag":"842c8917-e84b-4a92-99e8-4237daa2ff85","_rid":"gAAGAIAAAAECAAAAAAAAAA==","_self":"/dbs/gAAGAA==/colls/gAAGAIAAAAE=/docs/gAAGAIAAAAECAAAAAAAAAA==/","_ts":1608932852},{"id":"3","someData":["c"],"_etag":"e8033da5-1761-48a7-b0dd-365fe1548a0b","_rid":"gAAGAIAAAAEDAAAAAAAAAA==","_self":"/dbs/gAAGAA==/colls/gAAGAIAAAAE=/docs/gAAGAIAAAAEDAAAAAAAAAA==/","_ts":1608932852}],"_count":3}

And here the response I get from azure

{"partitionedQueryExecutionInfoVersion":2,"queryInfo":{"distinctType":"None","top":null,"offset":null,"limit":null,"orderBy":[],"orderByExpressions":[],"groupByExpressions":[],"groupByAliases":[],"aggregates":[],"groupByAliasToAggregateType":{},"rewrittenQuery":"","hasSelectValue":true},"queryRanges":[{"min":"05C1A3C5D33F20083200","max":"05C1A3C5D33F20083200","isMinInclusive":true,"isMaxInclusive":true},{"min":"05C1D12BA14B20083300","max":"05C1D12BA14B20083300","isMinInclusive":true,"isMaxInclusive":true},{"min":"05C1D3D3BB9304083400","max":"05C1D3D3BB9304083400","isMinInclusive":true,"isMaxInclusive":true}]}

Repro

Note: I had to generate my own CA then generate a certificate with CN=localhost then add the CA to /usr/local/share/ca-certificates/ and run sudo update-ca-certificates to pass the https check

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Threading.Tasks;
using Microsoft.Azure.Cosmos;
using Microsoft.Azure.Cosmos.Fluent;
using Microsoft.Azure.Cosmos.Linq;

namespace ReproBugCosmosDb
{
    class Program
    {
        public class SomeObject
        {
            public string Id { get; set; }
            public IList<string> SomeData { get; set; } = new List<string>();
        }

        private static CosmosClient _cosmosClient;
        public const string DevConnectionString = @"AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";
        
        public const string DatabaseName = "SomeReproDatabase2";
        public const string ContainerId = "SomeContainerName";

        static async Task Main(string[] args)
        {
            _cosmosClient = new CosmosClientBuilder(DevConnectionString)
                .WithSerializerOptions(new CosmosSerializationOptions {PropertyNamingPolicy = CosmosPropertyNamingPolicy.CamelCase})
                .WithConnectionModeGateway()
                .Build();
            await ResetDatabaseAsync(DatabaseName);
            var database = _cosmosClient.GetDatabase(DatabaseName);

            await database.CreateContainerAsync(ContainerId, "/id");
            var container = _cosmosClient.GetContainer(DatabaseName, ContainerId);
            await container.UpsertItemAsync(new SomeObject {Id = "1", SomeData = new List<string> {"a"}}, new PartitionKey("1"));
            await container.UpsertItemAsync(new SomeObject {Id = "2", SomeData = new List<string> {"b"}}, new PartitionKey("2"));
            await container.UpsertItemAsync(new SomeObject {Id = "3", SomeData = new List<string> {"c"}}, new PartitionKey("3"));
            await container.UpsertItemAsync(new SomeObject {Id = "4", SomeData = new List<string> {"d"}}, new PartitionKey("4"));

            try
            {
                var result = await GetElementsByIds(new List<string> {"1", "2", "3"});
                if (!result.OrderBy(x => x).SequenceEqual(new[] {"a", "b", "c"}))
                    throw new Exception("Invalid result");
                Console.WriteLine("Success");
            }
            catch (CosmosException ex)
            {
                Console.WriteLine(ex.Diagnostics.ToString());
                throw;
            }
        }

        private static async Task ResetDatabaseAsync(string databaseName)
        {
            await _cosmosClient.CreateDatabaseIfNotExistsAsync(databaseName);
            var database = _cosmosClient.GetDatabase(databaseName);
            await database.DeleteAsync();
            await _cosmosClient.CreateDatabaseIfNotExistsAsync(databaseName);
        }

        private static async Task<List<string>> GetElementsByIds(IList<string> ids)
        {
            var container = _cosmosClient.GetContainer(DatabaseName, ContainerId);
            var iterator = container.GetItemLinqQueryable<SomeObject>()
                .Where(x => ids.Contains(x.Id))
                .ToFeedIterator();
            var fetchedPhotoDocuments = new List<string>();

            while (iterator.HasMoreResults)
            {
                var item = await iterator.ReadNextAsync().ConfigureAwait(false);
                fetchedPhotoDocuments.AddRange(item.Resource.SelectMany(x => x.SomeData).ToList());
            }

            return fetchedPhotoDocuments.Distinct().ToList();
        }
    }
}

It throws here https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Query/v3Query/CosmosQueryClientCore.cs#L255

And data are loaded here
https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CosmosQueryExecutionContextFactory.cs#L217

Exception

Unhandled exception. Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: InternalServerError (500); Substatus: 0; ActivityId: 00000000-0000-0000-0000-000000000000; Reason: (Value cannot be null. (Parameter 'providedRanges'));
 ---> System.ArgumentNullException: Value cannot be null. (Parameter 'providedRanges')
   at Microsoft.Azure.Cosmos.CosmosQueryClientCore.GetTargetPartitionKeyRangesAsync(String resourceLink, String collectionResourceId, List`1 providedRanges, Boolean forceRefresh)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.GetTargetPartitionKeyRangesAsync(CosmosQueryClient queryClient, String resourceLink, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, IReadOnlyDictionary`2 properties, FeedRangeInternal feedRangeInternal)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateFromPartitionedQuerExecutionInfoAsync(DocumentContainer documentContainer, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateCoreContextAsync(DocumentContainer documentContainer, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.AsyncLazy`1.GetValueAsync(CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.LazyQueryPipelineStage.MoveNextAsync()
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.NameCacheStaleRetryQueryPipelineStage.MoveNextAsync()
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.CatchAllQueryPipelineStage.MoveNextAsync()
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Cosmos.CosmosQueryClientCore.GetTargetPartitionKeyRangesAsync(String resourceLink, String collectionResourceId, List`1 providedRanges, Boolean forceRefresh)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.GetTargetPartitionKeyRangesAsync(CosmosQueryClient queryClient, String resourceLink, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, IReadOnlyDictionary`2 properties, FeedRangeInternal feedRangeInternal)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateFromPartitionedQuerExecutionInfoAsync(DocumentContainer documentContainer, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateCoreContextAsync(DocumentContainer documentContainer, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.AsyncLazy`1.GetValueAsync(CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.LazyQueryPipelineStage.MoveNextAsync()
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.NameCacheStaleRetryQueryPipelineStage.MoveNextAsync()
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.CatchAllQueryPipelineStage.MoveNextAsync()
--- Cosmos Diagnostics ---{"DiagnosticVersion":"2","Summary":{"StartUtc":"2020-12-25T21:59:18.9399687Z","RunningElapsedTimeInMs":28.6454,"UserAgent":"cosmos-netstandard-sdk/3.15.1|3.15.2|03|X64|Linux 5.4.0-58-generic 64-Ubu|.NET 5.0.1-servicing.20575.16|","TotalRequestCount":0,"FailedRequestCount":0,"Operation":"CosmosDiagnosticsContextCore"},"Context":[]}

[nkzawa]

It looks like the method is for getting the query plan result but this server does not support it yet.

Maybe x-ms-documentdb-query-metrics: "true" https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-query-metrics

[jacobwaller]

Hey @Socolin did you ever get an update to this? We're trying to leverage this server on our current project.

[Socolin]

Hey @Socolin did you ever get an update to this? We're trying to leverage this server on our current project.

No, so far I switched to an cosmos db serverless instance in azure, since the official emulator is buggy on linux (and MS is not responding to the bugs report)
I would love this to be implemented so I could switch to this version of cosmos.

And in the near future I'll not have time to look into it and suggest a PR.