HTML character encoding issue for “ when upgrading

[portedison]

Describe the bug

Similar issue : #115

When upgrading from Vizy 1.0.23 and Craft CMS 3.9.10, to Vizy 2.1.23 and Craft CMS 4.13.1.1

Character were previously stored in the database as “ any newly created content saves in the database as “ though on an upgrade none of these characters have been migrated. When inspecting the code in the editors html view they appear as “ - though in the database they are still stored as “. A manual update fixes these.

Are there any recommended steps to migrate these?

Steps to reproduce

1. Create a project with Vizy 1.0.23 and Craft CMS 3.9.10
2. Create some content with a “ in a Vizy Field.
3. Migrate to Vizy 2.1.23 and Craft CMS 4.13.1.1

Craft CMS version

3.9.10 -> 4.13.1.1

Plugin version

1.0.23 -> 2.1.23

Multi-site?

No

Additional context

We have written a migration to update each entry in the database, though we've come across an issue where SuperTable content inside Vizy field, is difficult to access/update. (I'll mention that in another ticket and attach).

[engram-design]

Sounds like this'll be encoding changes. Just to be clear, re-saving the entry doesn't change anything, but re-typing the encoded values (swapping & for & for example) works correctly?

I might have to provide a manual console command to run on field content to take care of this.

In addition, there's actually an internal helper function to help with updating content, which handles global, Matrix and Super Table content. We use this in Hyper migrations.

[portedison]

Sounds like this'll be encoding changes. Just to be clear, re-saving the entry doesn't change anything, but re-typing the encoded values (swapping & for & for example) works correctly?

Yes that is correct.

That command could be helpful, the other issue we had was where we have the structure -
pageBlocks(Neo) > richText(Neo : Common Block) > blockRichText (Vizy) > vizy2ColumnsTable (Super Table)

But the last one vizy2ColumnsTable 'disappears', though it still appears to be in the database in the old craft format. I can make a ticket explaining more about that if it's useful? There is a comment relating to this already #292 (comment)

[Kethatril]

This is the change that surfaced the issue: 819a013

In 2.1.22 the characters showed up correctly in the editor and if you modified the field and saved it the database values would no longer have the escaped characters and that field would show up correctly in 2.1.23 in the editor. But because it only gets fixed by code in the editor resaving all entries using the cli doesn't fix the issue.

[engram-design]

Thanks for pointing that out @Kethatril that's a good start.