RavenDB 6.1 breaking changes #1890

reebhub · 2024-08-20T11:49:10Z

Related issues:
RDoc-2951 - Document breaking changes for 6.1
RDoc-2968 - Add 6.1 breaking changes to related 6.1 articles

…urable" section

reebhub · 2024-08-20T19:54:21Z

To reviewers:
there are two pages dedicated to breaking changes, one for client issues and a second for server issues, please enter them to find the sections you're related to.

The server page is at: migration/server/server-breaking-changes.markdown
it includes:
- Tracking operations using the Changes API now requires Node Tag (@ppekrol)
- Custom identity parts-separator is operative (@ppekrol)
- MsSql connection string requires an Encrypt property (@Lwiel)
- Corax handling of complex JSON objects in static indexes is configurable (@maciejaszyk)
the client page is at: migration/client-api/client-breaking-changes.markdown
it includes:
- Consistency in null handling (Pawel)
- CounterBatchOperation default increment Delta is 1 (@ppekrol)
- CmpXchg item can only be created with its index set to 0 (@shaharhikri)
- Dynamic Linq query cannot apply .Any with logical AND (&&) (@Lwiel)
- LoadDocument must be provided with a collection name string (@Lwiel)

Besides these:
@shaharhikri - please take a look at put-compare-exchange-value.dotnet.markdown (the additional point you requested at the top and the third example you requested at the bottom)
@ppekrol - please check what-is-changes-api.dotnet.markdown and how-to-subscribe-to-operation-changes.dotnet.markdown (just added the new overload and a few notes there)
@maciejaszyk - please also look at search-engine/corax.markdown (just look for ComplexFieldIndexingBehavior and you'll find the locations in it) and configuration/indexing-configuration.markdown (same here, look for ComplexFieldIndexingBehavior - it's just a minimal explanation of the configuration option)

thanks (o:

…rver configuration options page

arekpalinski · 2024-08-21T06:46:53Z

Documentation/6.1/Raven.Documentation.Pages/indexes/search-engine/corax.markdown

+     1. Open the index definition's **Fields** tab.  
+     2. Click **Add Field** to specify what field Corax shouldn't index.  
+     3. Enter the name of the field Corax should not index.  
+     4. Set to **Yes** when Corax is used since Corax will not index the value.  


Let's mention that storing the field will make that when querying and using this field in projection then it will be retrieved directly from the index. I know we have it mentioned in different part of the docs but this is important so let's repeat it again here.

-->

{NOTE: }
Content we retrieve from the database and store in indexes becomes available for projection and will be henceforth retrieved directly from the indexes, accelerating its retrieval at the expense of indexes storage space.
{NOTE/}

arekpalinski · 2024-08-21T07:04:34Z

Documentation/6.1/Raven.Documentation.Pages/indexes/search-engine/corax.markdown

+
+---
+
+#### 3. Disable the Indexing of the Complex Field


I think we need to rephrase this section. Saying that you can disable the indexing on a specific field isn't the "solution". The whole point is that a user needs to revise the index definition and the usage of fields that they intended for indexing.

Can we emphasize the need of revising the definition? I'd like to force a users make him/her more aware of what they are doing. I'm afraid that just saying "disable the indexing on that field and store it" might sound like a solution. While maybe they don't event need that field at all.

So maybe we can say (similar to what you have here but from slightly different angle):

searching over a complex field isn't typically indented, so it's likely you don't actually need that field at all

although if you needed this complex field because of projecting it during queries then you might still keep it but then disable the indexing and explicitly mark that it's stored

@reebhub if you have any doubts please ping me directly

->

3. Revise index definition and fields usage

As shown above, indexing a whole complex field is rarely needed, and users would typically index and search only the simple properties such a field contains.
Queries may sometimes need, however, to project the content of an entire complex field.
When this is the case, you can revise the index definition (see below) to disable the indexing of the complex field but store its content so projection queries would be able to project it.

arekpalinski · 2024-08-21T07:06:43Z

Documentation/6.1/Raven.Documentation.Pages/indexes/search-engine/corax.markdown

+     4. Set to **Yes** when Corax is used since Corax will not index the value.  
+     5. Select **No** to disable indexing for the specified field.  
+
+* To disable indexing for a specified field **using Code**:  


Please note that this sample code does 2 things, marks the field as non indexed and marks it for storing inside index

-->

To store a field's content and disable its indexing via Studio:

Open the index definition's Fields tab.

Click Add Field to specify what field Corax shouldn't index.

Enter the name of the field Corax should not index.

Select Yes to Store the field's content

Select No to disable the field's indexing

To store a field's content and disable its indexing using Code:
{CODE:csharp index-definition_disable-indexing-for-specified-field@Indexes/SearchEngines.cs /}

arekpalinski · 2024-08-21T07:10:00Z

Documentation/6.1/Raven.Documentation.Pages/indexes/search-engine/corax.markdown

+
+#### 4. Turn the complex property into a string
+
+You can handle the complex property as a string.  


Let's underline that this is an option only if you really need to index it as a JSON, and search over it. Note it will keep JSON structure so all { } and propetry names like { "Latitude":... , "Longitude": ... } will be processed as well (by the specified analyzer etc)

Oh, I see we talk about that below. So it's fine. Maybe just underline that this is last resort option, if you really need that.

yeah the warning is kind of sharp as it is. added "as a last resort" in any case.

-->

{NOTE: }
Serializing all the properties of a complex property into a single string, including names, values, brackets, and so on, can be used as a last resort to produce a string that doesn't make a good feed for analyzers and is not commonly used for searches.
It does, however, make sense in some cases to project such a string.
{NOTE/}

arekpalinski · 2024-08-21T07:16:58Z

Documentation/6.1/Raven.Documentation.Pages/indexes/search-engine/corax.markdown

+* If a **static index** is used and it doesn't explicitly relate 
+  to the complex field, Corax will automatically exempt the field 
+  from indexing (by defining **Indexing: No** for this field as shown 
+  [above](../../indexes/search-engine/corax#disable-the-indexing-of-the-complex-field)).  
+


This is not longer the case. We no longer have any distinction between setting explicitly Indexing: Yes, and having the default (which is also Indexing: Yes).

The behavior is the same now. We simply relay on Indexing.Corax.Static.ComplexFieldIndexingBehavior option. I think it's worth mentioning here that this is only for new deployed indexes to version 6.1 (or if they will be reset on 6.1, since it's effectively the same as deploying a new index). Old indexes will keep the old behavior.

changed it to a description of the three scenarios -

auto index (same description as before)

old static index (same description as before)

new static index (behavior set by ComplexFieldIndexingBehavior )

arekpalinski · 2024-08-21T07:22:48Z

Documentation/6.1/Raven.Documentation.Pages/indexes/search-engine/corax.markdown

+         Corax will throw a `NotSupportedInCoraxException ` exception with this message:  
+         **The value of `{fieldName}` is a complex object. Indexing it 
+         as a text isn't supported. You should consider querying on individual 
+         fields of that object.**


The message will be different:

https://github.com/ravendb/ravendb/pull/19035/files#diff-32e55cbc24468e2b2d3ad17fca814f4ac0922689c702c519c6a21c456f532185R541

The value of '{fieldName}' field is a complex object. Typically a complex field is not intended to be indexed as as whole hence indexing it as a text isn't supported in Corax. The field is supposed to have 'Indexing' option set to 'No' (note that you can still store it and use it in projections). Alternatively you can switch 'Indexing.Corax.Static.ComplexFieldIndexingBehavior' configuration option from 'Throw' to 'Skip' to disable the indexing of all complex fields in the index or globally for all indexes (index reset is required). If you really need to use this field for searching purposes, you have to call ToString() on the field value in the index definition. Although it's recommended to index individual fields of this complex object. Read more at: https://ravendb.net/l/OB9XW4/6.1

arekpalinski · 2024-08-21T07:23:23Z

Documentation/6.1/Raven.Documentation.Pages/indexes/search-engine/corax.markdown

+         fields of that object.**
+
+       * If `ComplexFieldIndexingBehavior` is set to **`Skip`** -  
+         Corax will skip the complex field without throwing an exception.  


Suggested change

Corax will skip the complex field without throwing an exception.

Corax will skip indexing the complex field without throwing an exception.

arekpalinski · 2024-08-21T07:26:32Z

...entation/6.1/Raven.Documentation.Pages/migration/client-api/client-breaking-changes.markdown

+        .Where(x => x.Suitcase.Any(p => p.Size == "Small" && p.Color == "Silver"))
+        .ToList();
+}
+{CODE-BLOCK/}


Maybe we can add more explanation here? Basically in order to handle a query like that we need a fanout index. That is no supported by auto indexes, hence the solution is to use a static fanout index.

Yes, we should mention this

added:

{INFO: }
While auto indexes do not currently support this type of query, static indexes do.
You can define a static fanout index that creates separate index entries for different document sub-objects, and run your query over these entries.
Read about the indexing of nested data here,
and find explanations and samples for fanout indexes here.
{INFO/}

arekpalinski · 2024-08-21T07:27:43Z

Documentation/6.1/Raven.Documentation.Pages/migration/server/server-breaking-changes.markdown

+
+---
+
+{PANEL: Tracking operations using the Changes API now requires Node Tag}


This is client side change

moved to the client-breaking-changes page

arekpalinski · 2024-08-21T07:30:32Z

...mentation/6.1/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown

+  {CODE-BLOCK:csharp}
+  public enum CoraxComplexFieldIndexingBehavior
+  {
+     None,


I would rather not mention None option here

Lwiel · 2024-08-21T09:42:44Z

Documentation/6.1/Raven.Documentation.Pages/migration/server/server-breaking-changes.markdown

+{PANEL: MsSql connection string requires an Encrypt property}
+
+To establish a connection with an MsSql server via ETL, RavenDB is required 
+by the `Microsoft.Data.SqlClient` package it utilizes to include in its 


Maybe we could mention the reason of this change, e.g.

by the Microsoft.Data.SqlClient package (that replaced deprecated System.Data.SqlClient in version 6.0.105)

-->

To establish a connection with an MsSql server via ETL, RavenDB is required by the Microsoft.Data.SqlClient package it utilizes (which replaces the deprecated System.Data.SqlClient package we've been using in previous versions) to include..

Lwiel · 2024-08-21T09:44:06Z

Documentation/6.1/Raven.Documentation.Pages/migration/server/server-breaking-changes.markdown

+connection string an `Encrypt` property that would determine whether to 
+encrypt the connection or not.  
+
+Recent RavenDB versions added this property to their connection strings 


It only applies to versions 6.0.105 up to 6.1.0 (excluded)

-->

Some RavenDB versions preceding 6.1 (down to 6.0.105) added this..

Just RavenDB versions preceding 6.1 (down to 6.0.105) added this... will be better

Lwiel · 2024-08-21T09:50:20Z

Documentation/6.1/Raven.Documentation.Pages/migration/server/server-breaking-changes.markdown

+string and users are required to explicitly choose whether to encrypt the 
+connection or not.  
+
+To encrypt the connection include an `Encrypt=true` in your connection 


Encrypt is an enum, so maybe we should just mention they should either set it to Strict or Mandatory

Or Optional, right @Lwiel ?

Yes, but not if they want to encrypt the connection. Optional will make it unencrypted

Yeah, but this is up to user. We just say what are the options

Lwiel · 2024-08-21T09:52:20Z

...entation/6.1/Raven.Documentation.Pages/migration/client-api/client-breaking-changes.markdown

+        .Where(x => x.Suitcase.Any(p => p.Size == "Small" && p.Color == "Silver"))
+        .ToList();
+}
+{CODE-BLOCK/}


Yes, we should mention this

Lwiel · 2024-08-21T09:55:10Z

...entation/6.1/Raven.Documentation.Pages/migration/client-api/client-breaking-changes.markdown

+from doc in docs.Orders
+select new {
+    // This attempt to construct a collection name will fail with an exception
+    ComapnyName = LoadDocument(doc.Company, doc.Company.Split('/')[0]).Name


CompanyName

ppekrol · 2024-08-21T12:41:21Z

Documentation/6.1/Raven.Documentation.Pages/migration/server/server-breaking-changes.markdown

+
+* In this page:  
+   * [Tracking operations using the Changes API now requires Node Tag](../../migration/server/server-breaking-changes#tracking-operations-using-the-changes-api-now-requires-node-tag)  
+   * [MsSql connection string requires an Encrypt property](../../migration/server/server-breaking-changes#mssql-connection-string-requires-an-encrypt-property)  


ppekrol · 2024-08-21T12:42:14Z

Documentation/6.1/Raven.Documentation.Pages/migration/server/server-breaking-changes.markdown

+
+{PANEL/}
+
+{PANEL: MsSql connection string requires an Encrypt property}


ppekrol · 2024-08-21T12:44:36Z

Documentation/6.1/Raven.Documentation.Pages/migration/server/server-breaking-changes.markdown

+An **identity parts separator** can now be defined using the server-wide client configuration, 
+to replace the default separator with a user-defined one.  
+
+The configuration option was available in Studio's **Client Configuration** for a while, but 


I would phrase that differently. Because previously that value could be set in server-wide client configuration, but wasn't taken into account for Database IdentityPartsSeparator. This means that if user set that value prior to 6.1 it was not active, but migrating to 6.1 will immediately activate it.

ppekrol · 2024-08-21T12:51:11Z

...entation/6.1/Raven.Documentation.Pages/migration/client-api/client-breaking-changes.markdown

+When [CounterBatchOperation](../../client-api/operations/counters/counter-batch) is 
+called to `Increment` a batch of counters, and `Delta` is not specified to indicate 
+what value should be added to the counters, the operation will increment the counters 
+by a default `Delta` of `1`.  


write that it was 0 before and that 1 is consistent with rest of the API

maciejaszyk · 2024-08-22T07:31:05Z

Documentation/6.1/Raven.Documentation.Pages/migration/server/server-breaking-changes.markdown

+The behavior of RavenDB's [Corax](../../indexes/search-engine/corax) search engine while 
+handling [complex JSON objects](../../indexes/search-engine/corax#handling-of-complex-json-objects) 
+in **static indexes** is now configurable using the `Indexing.Corax.Static.ComplexFieldIndexingBehavior` 
+configuration option.  
+(The [handling of auto indexes](../../indexes/search-engine/corax#if-corax-encounters-a-complex-property-while-indexing) 
+remains unchanged.)  
+
+By default, `ComplexFieldIndexingBehavior` is set to **`Throw`**, instructing the search 
+engine to throw a `NotSupportedInCoraxException` exception when it encounters a complex 
+field in a static index.  
+The exception includes this message: **The value of '`{fieldName}`' field is a complex object. 
+Indexing it as a text isn't supported. You should consider querying on individual fields of that object.**
+
+If you prefer, you can set `ComplexFieldIndexingBehavior` to **`Skip`** to disable the 
+indexing of complex fields without throwing an exception or raising a notification.  
+
+* The configuration option will apply only to **new** static indexes, created after the release 
+  of RavenDB 6.1. It will not affect older indexes.  
+* `ComplexFieldIndexingBehavior` can be set for a particular index as well as for all indexes.  
+* Though complex fields cannot be indexed, they **can** still be stored and projected.  
+* To search by the contents of a static index's complex field, you can convert it 
+  to a string (using `ToString` on the field value in the index definition).  
+  It is recommended, though, to index individual properties of the complex field.  


LGTM
cc @arekpalinski

thx
changed it a bit and added a few links to the revised Corax section

arekpalinski · 2024-08-23T05:50:04Z

Documentation/6.1/Raven.Documentation.Pages/indexes/search-engine/corax.markdown

+  **disable indexing** for this field by defining **Indexing: No** for it as shown 
+  [above](../../indexes/search-engine/corax#disable-the-indexing-of-the-complex-field).  
+   * If the Indexing flag is set to anything but "no" -  
+     Corax will throw a `NotSupportedInCoraxException ` exception.  


But it will do it just once, since we also disable the indexing of that field. So it won't attempt to index next values from that field

-->

If the Indexing flag is set to anything but "no" -
Corax will throw a NotSupportedInCoraxException exception.
As disabling indexing for this field will prevent additional attempts to index its values,
the exception will be thrown just once.

arekpalinski · 2024-08-23T05:51:57Z

Please check one, new comment from. Other than that it LGTM

…es` page, fixed by review comments

shaharhikri · 2024-08-25T05:49:06Z

...tion.Pages/client-api/operations/compare-exchange/put-compare-exchange-value.dotnet.markdown

+
+* The _Value_ is saved only if the passed _index_ is equal to the _index_ currently stored in the server for the specified _Key_.  
+
+* Creating a new compare exchange item is possible only when the passed _index_ is `0`.  


Add the "non-zero index" in cpmxg creation and "Example III" also to the java and nodejs pages

will be able to do that only when a 6.1 client is available for these languages..

…erverOptions.Licensing.ThrowOnInvalidOrMissingLicense`

reebhub added 6 commits August 19, 2024 23:44

RDoc-2951 - RavenDB 6.1 breaking changes

c259fe9

Added a "Corax handling of complex fields in a static index is config…

7b152e1

…urable" section

added related documents

3d880ca

updated Changes API articles with the relevant 6.1 change

5e49216

Compare exchange example added

58c60c0

cosmetics

b69c4c4

reebhub requested review from ppekrol, Lwiel, shaharhikri, maciejaszyk and arekpalinski August 20, 2024 19:55

explaining ComplexFieldIndexingBehavior in the Corax article and se…

fba3197

…rver configuration options page

arekpalinski reviewed Aug 21, 2024

View reviewed changes

Lwiel suggested changes Aug 21, 2024

View reviewed changes

ppekrol reviewed Aug 21, 2024

View reviewed changes

maciejaszyk reviewed Aug 22, 2024

View reviewed changes

reebhub added 2 commits August 23, 2024 01:33

fixes by review comments (mostly in the corax article)

ab9a50c

fixed by review comments

df45413

arekpalinski reviewed Aug 23, 2024

View reviewed changes

added a Migration\Embedded folder with a `TestDriver breaking chang…

be41c93

…es` page, fixed by review comments

arekpalinski approved these changes Aug 26, 2024

View reviewed changes

shaharhikri reviewed Aug 26, 2024

View reviewed changes

Lwiel approved these changes Aug 26, 2024

View reviewed changes

maciejaszyk approved these changes Aug 26, 2024

View reviewed changes

shaharhikri approved these changes Aug 26, 2024

View reviewed changes

TestServerOptions.License.ThrowOnInvalidOrMissingLicense --> `TestS…

1b66984

…erverOptions.Licensing.ThrowOnInvalidOrMissingLicense`

ppekrol merged commit 86804a2 into ravendb:master Aug 27, 2024
1 of 2 checks passed


		#### 4. Turn the complex property into a string

		You can handle the complex property as a string.

	Corax will skip the complex field without throwing an exception.
	Corax will skip indexing the complex field without throwing an exception.


		---

		{PANEL: Tracking operations using the Changes API now requires Node Tag}


		{PANEL/}

		{PANEL: MsSql connection string requires an Encrypt property}


		* The _Value_ is saved only if the passed _index_ is equal to the _index_ currently stored in the server for the specified _Key_.

		* Creating a new compare exchange item is possible only when the passed _index_ is `0`.

RavenDB 6.1 breaking changes #1890

RavenDB 6.1 breaking changes #1890

Conversation

reebhub commented Aug 20, 2024

reebhub commented Aug 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reebhub Aug 22, 2024 • edited Loading

Choose a reason for hiding this comment

3. Revise index definition and fields usage

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reebhub Aug 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arekpalinski commented Aug 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reebhub commented Aug 20, 2024 •

edited

Loading

reebhub Aug 22, 2024 •

edited

Loading

reebhub Aug 23, 2024 •

edited

Loading