Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RCORE-2175 RCORE-2176 RCORE-2182 RCORE-2083 Fix several problems with object removal and links #7830

Merged
merged 8 commits into from
Jun 29, 2024

Conversation

ironage
Copy link
Contributor

@ironage ironage commented Jun 21, 2024

I discovered these when inspecting the code while investigating #7594

The first issue appears to have been a mistake, and was introduced by #6766
I don't know of any reports of the wrong objects being deleted so hopefully this is niche enough that nobody has been affected, but on the other hand maybe it is related to some of the Key 'Number' not found in '<Class Name>' exceptions we have seen because unexpected objects are being deleted.

The second issue is more benign, as fewer objects than expected may be deleted. The main thing there is that the state shouldn't have been ignored, and we already had a method for handling this.

The third issue was found due to CI running the new tests in the BPNODE=4 configuration. This uncovered a few other places in the code that had the same mistake: using the cluster local key instead of the "real" key. I renamed the variables there to hopefully make it clearer, and to avoid this mistake in the future.

The randomized testing that I added to cover the changes for the nested list/dictionary cases happened to trigger yet another bug which is the collapse/rejoin of clusters with nested collections and links.

Fixes #7828, #7829, #7839, #7594

☑️ ToDos

  • 📝 Changelog update
  • 🚦 Tests (or not relevant)
  • C-API, if public C++ API changed
  • bindgen/spec.yml, if public C++ API changed

@ironage ironage self-assigned this Jun 21, 2024
@cla-bot cla-bot bot added the cla: yes label Jun 21, 2024
@ironage ironage requested a review from tgoyne June 21, 2024 00:07
Copy link

coveralls-official bot commented Jun 21, 2024

Pull Request Test Coverage Report for Build james.stone_561

Details

  • 82 of 82 (100.0%) changed or added relevant lines in 2 files are covered.
  • 55 unchanged lines in 14 files lost coverage.
  • Overall coverage decreased (-0.003%) to 90.974%

Files with Coverage Reduction New Missed Lines %
src/realm/mixed.cpp 1 86.75%
src/realm/sort_descriptor.cpp 1 94.06%
src/realm/sync/noinst/server/server_history.cpp 1 63.7%
src/realm/util/file.cpp 1 84.84%
src/realm/util/serializer.cpp 1 90.43%
test/test_index_string.cpp 1 93.48%
src/realm/table_view.cpp 2 92.99%
src/realm/sync/noinst/protocol_codec.hpp 3 73.54%
src/realm/sync/noinst/server/server.cpp 5 73.19%
test/fuzz_group.cpp 5 48.64%
Totals Coverage Status
Change from base Build 2434: -0.003%
Covered Lines: 214814
Relevant Lines: 236128

💛 - Coveralls

@ironage
Copy link
Contributor Author

ironage commented Jun 22, 2024

CI caught another bug in the BPNODE = 4 configuration where the wrong backlinks were being removed. I think I fixed it as well, but I plan to add more tests next week.

Copy link

coveralls-official bot commented Jun 22, 2024

Pull Request Test Coverage Report for Build james.stone_563

Details

  • 84 of 84 (100.0%) changed or added relevant lines in 2 files are covered.
  • 104 unchanged lines in 18 files lost coverage.
  • Overall coverage decreased (-0.01%) to 90.963%

Files with Coverage Reduction New Missed Lines %
src/realm/array_string.cpp 1 87.23%
src/realm/util/file.cpp 1 84.84%
src/realm/util/serializer.cpp 1 90.43%
test/fuzz_tester.hpp 1 57.73%
test/test_index_string.cpp 1 93.48%
test/test_query2.cpp 1 98.73%
test/test_all.cpp 2 75.82%
src/realm/mixed.cpp 3 86.46%
src/realm/query_expression.hpp 3 93.81%
src/realm/sync/noinst/server/server.cpp 3 73.71%
Totals Coverage Status
Change from base Build 2434: -0.01%
Covered Lines: 214789
Relevant Lines: 236128

💛 - Coveralls

@ironage ironage changed the title RCORE-2175 RCORE-2176 Fix several problems with recursive object removal RCORE-2175 RCORE-2176 RCORE-2182 RCORE-2083 Fix several problems with recursive object removal Jun 25, 2024
@ironage ironage changed the title RCORE-2175 RCORE-2176 RCORE-2182 RCORE-2083 Fix several problems with recursive object removal RCORE-2175 RCORE-2176 RCORE-2182 RCORE-2083 Fix several problems with object removal and links Jun 25, 2024
Copy link

coveralls-official bot commented Jun 26, 2024

Pull Request Test Coverage Report for Build james.stone_564

Details

  • 258 of 265 (97.36%) changed or added relevant lines in 3 files are covered.
  • 94 unchanged lines in 24 files lost coverage.
  • Overall coverage decreased (-0.005%) to 90.972%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/cluster.cpp 13 14 92.86%
test/test_mixed_null_assertions.cpp 243 249 97.59%
Files with Coverage Reduction New Missed Lines %
src/realm/array_string.cpp 1 88.03%
src/realm/sort_descriptor.cpp 1 94.06%
src/realm/util/file.cpp 1 84.84%
src/realm/util/serializer.cpp 1 90.43%
test/test_dictionary.cpp 1 99.83%
test/test_index_string.cpp 1 93.48%
test/test_query2.cpp 1 98.73%
test/test_util_network.cpp 1 95.56%
src/realm/array_blobs_big.cpp 2 98.58%
src/realm/table_view.cpp 2 92.99%
Totals Coverage Status
Change from base Build 2434: -0.005%
Covered Lines: 214958
Relevant Lines: 236291

💛 - Coveralls

@ironage
Copy link
Contributor Author

ironage commented Jun 26, 2024

Asking for re-review as there have been a couple more changes, cc: @tgoyne @jedelbo. Thanks!

Copy link

coveralls-official bot commented Jun 26, 2024

Pull Request Test Coverage Report for Build james.stone_566

Details

  • 319 of 328 (97.26%) changed or added relevant lines in 3 files are covered.
  • 84 unchanged lines in 22 files lost coverage.
  • Overall coverage increased (+0.02%) to 90.998%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/cluster.cpp 12 13 92.31%
test/test_mixed_null_assertions.cpp 305 313 97.44%
Files with Coverage Reduction New Missed Lines %
src/realm/array_string.cpp 1 88.03%
src/realm/query_expression.hpp 1 93.88%
src/realm/util/file.cpp 1 84.84%
src/realm/util/serializer.cpp 1 90.43%
test/fuzz_tester.hpp 1 57.73%
test/test_dictionary.cpp 1 99.83%
test/test_query2.cpp 1 98.73%
src/realm/array_blobs_big.cpp 2 98.58%
src/realm/db.cpp 2 92.05%
src/realm/object-store/shared_realm.cpp 2 91.89%
Totals Coverage Status
Change from base Build 2434: 0.02%
Covered Lines: 215088
Relevant Lines: 236365

💛 - Coveralls

CHANGELOG.md Outdated
@@ -6,7 +6,10 @@

### Fixed
* <How do the end-user experience this issue? what was the impact?> ([#????](https://github.com/realm/realm-core/issues/????), since v?.?.?)
* None.
* Fixed a change of mode from Strong to All when removing links from an embedded object that links to a tombstone. This affects sync apps that use embedded objects which have a `Lst<Mixed>` that contains a link to another top level object which has been deleted by another sync client (creating a tombstone locally). In this particular case, the switch would cause any remaining link removals to recursively delete the destination object if there were no other links to it. ([#7828](https://github.com/realm/realm-core/issues/7828), since 14.0.0-beta.0)
* Fixed `Table::remove_object_recursive` which wouldn't recursively follow links through a single `Mixed` property. ([#7829](https://github.com/realm/realm-core/issues/7829), likely since the introduction of Mixed)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This maybe goes under Internal since we don't actually expose remove_object_recursive().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid point. It is part of core's public API, so I wasn't sure if this feature was exposed by any SDK or not.

Copy link
Contributor

@jedelbo jedelbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice. I realize that using ObjKey parameters to hold something that does not actually identify an Obj was wrong. I have made #7852 as a PR to this PR. It introduces a new type - RowKey that can hold those internal keys, You are welcome to merge it if you like it.

Copy link

coveralls-official bot commented Jun 29, 2024

Pull Request Test Coverage Report for Build james.stone_571

Details

  • 371 of 382 (97.12%) changed or added relevant lines in 8 files are covered.
  • 34 unchanged lines in 8 files lost coverage.
  • Overall coverage increased (+0.04%) to 90.991%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/cluster_tree.hpp 1 2 50.0%
src/realm/cluster.cpp 31 33 93.94%
test/test_mixed_null_assertions.cpp 305 313 97.44%
Files with Coverage Reduction New Missed Lines %
src/realm/index_string.hpp 1 93.48%
src/realm/sync/noinst/server/server_history.cpp 1 63.7%
test/test_query2.cpp 1 98.73%
src/realm/sync/transform.cpp 2 61.05%
test/fuzz_group.cpp 3 51.67%
src/realm/bplustree.cpp 7 72.39%
src/realm/sync/noinst/server/server.cpp 9 73.74%
src/realm/sync/noinst/client_impl_base.cpp 10 82.06%
Totals Coverage Status
Change from base Build 2449: 0.04%
Covered Lines: 215227
Relevant Lines: 236536

💛 - Coveralls

@ironage ironage merged commit 3551a42 into master Jun 29, 2024
39 checks passed
@ironage ironage deleted the js/removal-fixes branch June 29, 2024 00:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Removing an object which links to embedded objects can sometimes delete non-embedded objects too
3 participants