How to work with services where the metadata is inaccurate?

[JBoman32768]

Hi there,
FIrst off - great work with this library - and this may be a bit of a noob question, and not a bug in Simple.OData.Client - but I am trying to work with an ODATA service (SharePoint Online) where the $metadata has declared the type of the value as:

and the ODATA returned for the property looks like this:

<d:AssignedToId m:null="true" />

... the error surfaced from the ODATA client is quite correct:

"A null value was found for the property named 'AssignedToId', which has the expected type 'Collection(Edm.Int32)[Nullable=False]'. The expected type 'Collection(Edm.Int32)[Nullable=False]' does not allow null values."

but I need to consume this data anyway, without being able to modify the source service (SharePoint Online)

I have tried tactics like modifying the metadata document and recreating the client, which have not worked:

var metadata = await myODataClient.GetMetadataAsStringAsync();
metadata = metadata.Replace("Type="Collection(Edm.Int32)" Nullable="false" />", "Type="Collection(Edm.Int32)" Nullable="true"");

var newSettings = new ODataClientSettings();
newSettings.MetadataDocument = metadata;
var newMyODataClient = new ODataClient(newSettings);

.... perform the query and the same thing happens! :( its a mystery where the metadata comes from that tells it nullable=false.

Any advice on how to use the Simple.Odata.Client library to consume this data would be helpful as I really don't want to write my own parser for this.

Cheers,
James.

[object]

Hi,
I believe rewriting metadata won't help you if the error occurs on a server side (does it?) Can you instead of rewriting metadata assign AssignedToId an empty collection?

[JBoman32768]

Would it be possible for you to expand on the idea of assigning AssignedToId an empty collection?

I am writing the consumer of the ODATA feed only. The provider of the ODATA is Microsoft SharePoint online, so I can't change anything on the providing server-side. I am using the Simple.Odata.Client in an attempt to query the data, but the queries always fail because the result cannot be parsed as the AssignedToId contains a null (when according to the metadata it shouldn't).

Downloading the metadata document, altering the nullable setting and then using the MetadataDocument property of the Simple.Odata.Client settings object (creating a new client) appears to have no effect. It still manages to get the nullable='false' metadata from somewhere - perhaps the in-memory cache.

The only workaround I have found so far is to remove the null values from the http response before the parser executes, like this:

        newSettings.AfterResponse = async (System.Net.Http.HttpResponseMessage response) => 
        {
            var content = await response.Content.ReadAsStringAsync();
            response.Content = new StringContent(content.Replace(",\"AssignedToId\":null", ""));
        };

This allows the result of the query to be read with the Simple.Odata.Client without throwing an exception - but is far from optimal :(

Any suggestions you have here would be really appreciated.

Cheers,
James.

[object]

If this happens on the client side and even if you change MetadataDocument, then this must be something I should be able to fix. Do you have a chance to give a temporary access to your service to me? If not, can you send me the original metadata document at vagif.abilov at gmail?

[JBoman32768]

Thanks for looking at this - I will send the metadata document and the service response.

Just for reference - the workaround above had a few problems, the following works better to remove null values before parsing - but again, far from optimal:

        newSettings.AfterResponse = async (System.Net.Http.HttpResponseMessage response) => 
        {
            var content = await response.Content.ReadAsStringAsync();
            content = new Regex("\"[^\"]*\"\\s*:\\s*null\\s*,|,\"[^\"]*\"\\s*:\\s*null\\s*").Replace(content, "");
            var contentStream = new MemoryStream();
            var contentWriter = new StreamWriter(contentStream);
            contentWriter.Write(content);
            contentWriter.Flush();
            contentStream.Position = 0;
            var newContentStream = new System.Net.Http.StreamContent(contentStream);
            newContentStream.Headers.ContentType = response.Content.Headers.ContentType;
            response.Content = newContentStream;
            //we were never here.
        };

[JBoman32768]

I have sent you a sample project to reproduce this issue - and also the Async cancellation issue.

Thanks again for your time looking into this - we are big fans of your library here at IPMO.

[johnwc]

@object @robertmclaws Can this be looked at again. We are getting the same error, and we actually see this often. Where metadata states a property is a collection, and the server replies with the property as null. Which causes the error to be thrown, because according to oData specs, collections cannot be null in server response. It would be helpful if we can apply a setting that tells the oData client to ignore that specific error for collections.

I believe it stems from using Microsoft's JsonLight.
https://github.com/OData/odata.net/blob/main/src/Microsoft.OData.Core/ReaderValidationUtils.cs#L1171