Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RCORE-2064 String EQ/NEQ optimisations for compressed strings #7820

Merged

Conversation

nicola-cab
Copy link
Member

What, How & Why?

Optimize query/search with EQ/NE

String translation to StringID (interned string ID) is currently done for each array search. This PR is to move the translation up the call chain, such that it is done only once for a column.

Note: Even if a string has not been interned, it may still be present in leaf arrays which have not been interned (interned and non-interned leafs may coexist within a column)

Copy link

coveralls-official bot commented Jun 18, 2024

Pull Request Test Coverage Report for Build nicola.cabiddu_1798

Details

  • 41 of 41 (100.0%) changed or added relevant lines in 4 files are covered.
  • 115 unchanged lines in 11 files lost coverage.
  • Overall coverage decreased (-0.04%) to 90.818%

Files with Coverage Reduction New Missed Lines %
src/realm/array_with_find.hpp 2 98.15%
src/realm/sync/network/http.hpp 2 82.27%
src/realm/query_expression.hpp 3 93.84%
src/realm/sync/client.cpp 3 91.78%
test/fuzz_tester.hpp 4 57.32%
test/object-store/util/sync/baas_admin_api.cpp 5 84.93%
src/realm/sync/noinst/protocol_codec.hpp 6 73.5%
src/realm/sync/noinst/server/server.cpp 8 74.11%
src/realm/sync/instruction_applier.cpp 12 68.01%
src/realm/sync/noinst/client_impl_base.cpp 18 82.02%
Totals Coverage Status
Change from base Build nicola.cabiddu_1796: -0.04%
Covered Lines: 218024
Relevant Lines: 240067

💛 - Coveralls

@nicola-cab nicola-cab force-pushed the nc/eq_neq_string_compression_query branch from 34b3fa7 to 71b34ed Compare June 19, 2024 14:37
@nicola-cab nicola-cab marked this pull request as ready for review June 19, 2024 14:43
@nicola-cab nicola-cab force-pushed the nc/eq_neq_string_compression_query branch from b760185 to 050fb75 Compare June 19, 2024 15:22
Copy link

coveralls-official bot commented Jun 19, 2024

Pull Request Test Coverage Report for Build nicola.cabiddu_1804

Details

  • 48 of 48 (100.0%) changed or added relevant lines in 4 files are covered.
  • 101 unchanged lines in 18 files lost coverage.
  • Overall coverage decreased (-0.01%) to 90.846%

Files with Coverage Reduction New Missed Lines %
src/realm/array_string.cpp 1 88.13%
src/realm/sync/noinst/server/server_history.cpp 1 63.7%
src/realm/array_blobs_big.cpp 2 98.58%
src/realm/array_with_find.hpp 2 98.15%
src/realm/db.cpp 2 92.06%
src/realm/sync/network/http.hpp 2 82.76%
src/realm/sync/noinst/server/server.cpp 2 74.26%
test/object-store/sync/flx_sync.cpp 2 98.22%
test/object-store/util/test_file.cpp 2 86.29%
src/realm/unicode.cpp 3 83.83%
Totals Coverage Status
Change from base Build nicola.cabiddu_1801: -0.01%
Covered Lines: 218304
Relevant Lines: 240302

💛 - Coveralls

src/realm/array_string.cpp Show resolved Hide resolved
src/realm/query_engine.cpp Outdated Show resolved Hide resolved
src/realm/table.cpp Outdated Show resolved Hide resolved
src/realm/table.cpp Outdated Show resolved Hide resolved
src/realm/table.cpp Outdated Show resolved Hide resolved
@nicola-cab nicola-cab force-pushed the nc/string_interner_tests branch 4 times, most recently from 26fe7f2 to 27b9c94 Compare July 4, 2024 15:26
@nicola-cab nicola-cab force-pushed the nc/string_interner_tests branch 3 times, most recently from 2600251 to 5660d1f Compare July 4, 2024 15:39
@nicola-cab nicola-cab requested a review from jedelbo July 5, 2024 11:03
@nicola-cab
Copy link
Member Author

@jedelbo should be ready for a second pass... Much smaller :)

Copy link

coveralls-official bot commented Jul 5, 2024

Pull Request Test Coverage Report for Build nicola.cabiddu_1822

Details

  • 16 of 17 (94.12%) changed or added relevant lines in 2 files are covered.
  • 1693 unchanged lines in 70 files lost coverage.
  • Overall coverage decreased (-0.003%) to 90.859%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/array_string.cpp 7 8 87.5%
Files with Coverage Reduction New Missed Lines %
src/realm/chunked_binary.cpp 1 98.36%
src/realm/sync/noinst/server/server_history.cpp 1 63.7%
src/realm/table.cpp 1 90.1%
test/object-store/sync/session/connection_change_notifications.cpp 1 93.88%
test/util/crypt_key.cpp 1 75.0%
src/realm/array_blobs_big.cpp 2 98.58%
src/realm/array_key.cpp 2 96.55%
src/realm/db.hpp 2 87.63%
src/realm/query_expression.cpp 2 86.62%
test/test_lang_bind_helper.cpp 2 93.2%
Totals Coverage Status
Change from base Build nicola.cabiddu_1821: -0.003%
Covered Lines: 218312
Relevant Lines: 240275

💛 - Coveralls

Copy link
Contributor

@jedelbo jedelbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the fixes took the wrong turn. The PR is about looking up the search string once and use that.

src/realm/query_engine.cpp Outdated Show resolved Hide resolved
src/realm/table.cpp Show resolved Hide resolved
src/realm/query_engine.hpp Outdated Show resolved Hide resolved
Base automatically changed from nc/string_interner_tests to feature/string-compression July 8, 2024 17:56
@nicola-cab nicola-cab merged commit 1afc39e into feature/string-compression Jul 10, 2024
13 of 29 checks passed
@nicola-cab nicola-cab deleted the nc/eq_neq_string_compression_query branch July 10, 2024 10:51
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 9, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants