Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reindexer context support #20

Merged
merged 161 commits into from
Jan 21, 2025
Merged
Show file tree
Hide file tree
Changes from 146 commits
Commits
Show all changes
161 commits
Select commit Hold shift + click to select a range
6bbe288
[fix] Rework code according to Valgrind's recommendations
Oct 8, 2024
d27da93
Clear code
Oct 9, 2024
81ca870
Avoid PEP's warnings
Oct 9, 2024
2f9edd5
Update code
Oct 9, 2024
0ded690
Support sync transactions
Oct 14, 2024
de64dac
Rewrite query results wrapper
Oct 14, 2024
0392983
Properly use NewItem method. Reorganize code
Oct 15, 2024
1cbd051
Implement transaction py API
Oct 15, 2024
1a77b8e
Add transaction py-tests
Oct 15, 2024
3ac3485
Update Readme.md
Oct 16, 2024
0d44728
[fix] Fix build pyreindexer with reindexer v.4.15
Oct 16, 2024
2f01596
Try fix MacOS tests
Oct 17, 2024
ce25efa
Merge branch 'fix/rework_code_according_to_valgrinds_recommendations'…
Oct 17, 2024
da03def
Correct style
Oct 17, 2024
e016c33
Update tests
Oct 17, 2024
a7b03c2
Reorganize README
Oct 17, 2024
d0d732d
Correct test_item
Oct 18, 2024
afa4770
Merge branch 'fix/build_pyreindexer_with_v4.15.0_GetAggregationResult…
Oct 18, 2024
99099fa
Update transaction test
Oct 23, 2024
ae9dcd4
Simplify code
Oct 23, 2024
5454cf9
Style changes
Oct 23, 2024
247b231
Start of implementation of query builder
Oct 23, 2024
b3fad40
Part I: start of implementation of query builder
Oct 24, 2024
1f53241
Part II: start of implementation of query builder
Oct 25, 2024
122c00c
Part III: start of implementation of query builder
Oct 25, 2024
25c868a
Part IV: start of implementation of query builder
Oct 25, 2024
f403305
Merge branch 'master' into feature/add_transaction_support
Oct 25, 2024
a68c5b7
Merge branch 'feature/add_transaction_support' into feature/add_query…
Oct 25, 2024
5b10ceb
Part V: start of implementation of query builder
Oct 25, 2024
cbc34aa
Part VI: start of implementation of query builder
Oct 28, 2024
cd35c30
Add commit_with_count()
Oct 28, 2024
8f7542d
Add commit_with_count(). Fix code and check in test
Oct 29, 2024
0972bc7
Merge branch 'feature/add_transaction_support' into feature/add_query…
Oct 29, 2024
c9070b4
Rewrite query_wrapper
Oct 29, 2024
30b8806
Rewrite query_wrapper. Part II
Oct 30, 2024
bd55939
Rewrite query_wrapper. Part III: add where for Variant
Oct 30, 2024
2d64757
Rewrite query_wrapper. Part IV: add where for Variant. Clean code
Oct 30, 2024
2d6423f
Part VII: start of implementation of query builder. Add where
Oct 30, 2024
5280a2b
Part VIII: start of implementation of query builder. Add Point and dw…
Oct 30, 2024
dd7aceb
Part IX: start of implementation of query builder. Add simple aggregate
Oct 30, 2024
35dd3ed
Part X: start of implementation of query builder. Add sort
Oct 31, 2024
eee1335
Part XI: start of implementation of query builder. Add aggregation
Oct 31, 2024
87cf4a6
Part XI: start of implementation of query builder. Add aggregation. Fix
Nov 1, 2024
3e04ec2
Part XII: start of implementation of query builder. Add SetObject
Nov 1, 2024
ea30af2
Part XIII: start of implementation of query builder. Add Join
Nov 1, 2024
066efca
Part XIV: start of implementation of query builder. Add Delete. Part I
Nov 1, 2024
c14a9ae
Part XV: start of implementation of query builder. Add Insert\Delete\…
Nov 2, 2024
5cd9e40
Part XV: start of implementation of query builder. Add Insert\Delete\…
Nov 2, 2024
d051dba
Part XV: start of implementation of query builder. Add Insert\Delete\…
Nov 5, 2024
cc74f67
[ref] tests
Nov 5, 2024
e6fee62
Part XVI: start of implementation of query builder. Add Get(firstItem…
Nov 5, 2024
4286c1b
Part XVII: clear code. Hide all context unimplemented functions
Nov 5, 2024
f412d96
Prevent warning on build
Nov 5, 2024
e57ed58
Update comments
Nov 6, 2024
4ab751e
[ref] more test refactoring
Nov 6, 2024
5f242fe
[fix] bump setup version
Nov 6, 2024
7fca19e
[fix] fix imports for ci
Nov 6, 2024
35c2fd0
[fix] bump version
Nov 6, 2024
e57fa7a
[fix] update package_data
Nov 6, 2024
20b70d9
[fix] fix imports and package_data
Nov 6, 2024
5351f97
Part XVIII: add using fetch count
Nov 6, 2024
c328388
Merge branch 'ref_tests' into feature/add_query_support
Nov 6, 2024
c629d18
[ref] small code ref
Nov 6, 2024
4b40a64
[ref] ref tests with tx
Nov 6, 2024
319e615
Merge remote-tracking branch 'origin/feature/add_query_support' into …
Nov 6, 2024
c255552
Clear code after review
Nov 6, 2024
dc5a7d3
Part XVIII: rollback using fetch count. Fix execute method
Nov 6, 2024
d5bb868
Part XIX: remove Query aliases from connector (select\delete\update)
Nov 6, 2024
02908c5
Simplify example code
Nov 6, 2024
08941cf
Remove `-Wold-style-cast` cmake flag
Nov 6, 2024
d60c9f6
Merge remote-tracking branch 'origin/feature/add_query_support' into …
Nov 7, 2024
e2a8ea8
[fix] fix tests with tx
Nov 7, 2024
0c72d21
add initial query tests
Nov 7, 2024
19e881f
Part XX: remove support of FetchCount from Query object
Nov 7, 2024
17009b8
Simplify code
Nov 7, 2024
b3303be
add new query tests
Nov 7, 2024
c4b8f08
Part XXI: bug fix. Segmentation fault
Nov 8, 2024
3cf7a9a
add new query select tests
Nov 8, 2024
bb98bca
Merge branch 'master' into feature/add_query_support
Nov 8, 2024
edf2f41
Part XXII: update readme generation (pydoc_markdown)
Nov 8, 2024
599410c
Part XXIII: enum for LogLevel
Nov 8, 2024
e4fc259
Small updates
Nov 8, 2024
f38e721
Part XXIV: Add support GetExplainResults
Nov 11, 2024
192e168
add query tests: update, delete
Nov 11, 2024
603ac96
Update markdown
Nov 11, 2024
2553303
Part XXV: Bug fixes
Nov 11, 2024
fae7270
Part XXV: Bug fixes. Part II
Nov 12, 2024
c9e74da
[fix] err for select and update, querytests
Nov 12, 2024
835b7dc
[fix] more query test fixes
Nov 12, 2024
f4946b5
[fix] fix forced_sort test
Nov 13, 2024
49eb82e
Part XXV: Bug fixes. Part III: set & set_object
Nov 13, 2024
60d071a
add more query tests
Nov 13, 2024
b091390
Simplify code
Nov 13, 2024
46afcd5
Part XXV: Bug fixes. Part IV: expected: <[{'id': -1}]> but: was <[{'i…
Nov 13, 2024
7bc415e
add more tests with negative
Nov 13, 2024
2164fcf
add null to create items test
Nov 13, 2024
89a7465
Part XXVI: add return TotalCount
Nov 15, 2024
26d70c2
Part XXVII: disable WAL-queries
Nov 18, 2024
7c0bc72
Part XXVII: disable WAL-queries. Catch unhandled exception
Nov 18, 2024
4480dec
Part XXVIII: Return JOIN result. Part
Nov 19, 2024
9a4abfb
Part XXIX: where_query check
Nov 20, 2024
c57663f
Part XXX: where_composite fixes. Part
Nov 21, 2024
fdcfe30
Part XXX: where_composite fixes. Part
Nov 21, 2024
49ddd64
Part XXX: where_composite fixes. Part. Fix
Nov 21, 2024
cfce014
Part XXX: where_composite done
Nov 21, 2024
4378562
Part XXXI: join support/ Part
Nov 25, 2024
bbf4b41
Part XXXI: join support. Part
Nov 25, 2024
9ec593f
Part XXXI: join support. Simplify code
Nov 26, 2024
6910083
add join tests
Nov 27, 2024
0d09c11
Part XXXII: fix v.4 build
Nov 27, 2024
ca7916e
Corrections and fixes
Nov 29, 2024
c23fb33
Simplify code, correct style
Nov 29, 2024
ceea55b
Merge branch 'master' into feature/add_query_support
Dec 2, 2024
1f726be
Fix. Add default value
Dec 2, 2024
d2c0a66
Update minimal Reindexer version
Dec 2, 2024
d3fd315
Client configuration support for Reindexer
Dec 3, 2024
1bccaae
add exceptions
Dec 3, 2024
a2f80cd
Client configuration support for Reindexer. Fix MacOS build
Dec 3, 2024
db70321
Client configuration support for Reindexer. ReFix MacOS build
Dec 3, 2024
fcba1b5
fix raises in docstrings
Dec 3, 2024
aaeb1fa
small ref, fix expected exception
Dec 4, 2024
fb805a3
Add with_timeout func. Part I
Dec 4, 2024
b4360b3
Update readme file
Dec 4, 2024
28ff2dc
Fix build
Dec 4, 2024
4748384
Fix build
Dec 4, 2024
c7c3618
Add with_timeout func. Part II
Dec 5, 2024
794eb5f
Fix build
Dec 5, 2024
ab66015
Review changes
Dec 5, 2024
a39b966
Review changes
Dec 5, 2024
f280c66
Review changes
Dec 9, 2024
1325842
Merge branch 'master' into feature/add_query_support
Dec 9, 2024
1e4a49f
Review changes. Fix build
Dec 9, 2024
efd1436
Add query in transaction
Dec 10, 2024
b15395b
Update version
Dec 10, 2024
a1b0666
Merge branch 'feature/add_query_support' into feature/reindexer_confi…
Dec 10, 2024
22423d0
Merge branch 'feature/add_query_support' into feature/reindexer_conte…
Dec 10, 2024
c568aac
Merge branch 'master' into feature/reindexer_config_support
Dec 12, 2024
d6479e5
Update version
Dec 12, 2024
985b530
Correct README
Dec 12, 2024
fac565b
Merge branch 'feature/reindexer_config_support' into feature/reindexe…
Dec 13, 2024
28250be
Update version
Dec 13, 2024
cfc6faa
Merge branch 'master' into feature/reindexer_context_support
Dec 17, 2024
bc54a1a
Update version
Dec 17, 2024
03dd970
Use timedelta type for timeout
Dec 23, 2024
b5e9455
Rework timeout apply
Dec 27, 2024
4ff832a
Add warning
Jan 10, 2025
e1897cd
Update timeout comment
Jan 17, 2025
ad21eb3
Update timeout comment. Minimum
Jan 17, 2025
600ac97
add context tests
Jan 17, 2025
fbb22b4
add options tests
Jan 17, 2025
171d3ce
fix tests
Jan 17, 2025
3e0c594
fix err msg for macos
Jan 17, 2025
9a8c949
fix query test, add test with join and timeout
Jan 17, 2025
13b23de
increase test query items num
Jan 17, 2025
57a4510
Prevent error when fetch_amount=0, `PyObjectFromJson: Error parsing j…
Jan 17, 2025
16d777a
Add input value check for 'fetch_amount'
Jan 17, 2025
994be5d
increase test query items num
Jan 17, 2025
c87f7a7
ref tests
Jan 17, 2025
20a3334
Fix runtime exception: AttributeError: 'ConnectorAPI' object has no a…
Jan 20, 2025
d1153ca
Fix bug. Invalid transit for parameter
Jan 20, 2025
bfaa715
Update version
Jan 21, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
174 changes: 144 additions & 30 deletions README.md

Large diffs are not rendered by default.

14 changes: 8 additions & 6 deletions pyreindexer/example/main.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
from datetime import timedelta

from pyreindexer import RxConnector
from pyreindexer.exceptions import ApiError
from pyreindexer.query import CondType
Expand All @@ -22,7 +24,7 @@ def create_index_example(db, namespace):
try:
db.index_add(namespace, index_definition)
except ApiError:
db.index_drop(namespace, 'id')
db.index_drop(namespace, 'id', timedelta(milliseconds = 1000))
db.index_add(namespace, index_definition)


Expand Down Expand Up @@ -55,10 +57,10 @@ def create_items_example(db, namespace):
def select_item_query_example(db, namespace):
item_name_for_lookup = 'item_0'

return db.select("SELECT * FROM " + namespace + " WHERE name='" + item_name_for_lookup + "'")
return (db.select("SELECT * FROM " + namespace + " WHERE name='" + item_name_for_lookup + "'", timedelta(milliseconds = 1000)))

def select_all_item_query_example(db, namespace):
return db.select("SELECT * FROM " + namespace)
return db.select("SELECT * FROM " + namespace, timedelta(milliseconds = 1000))

def print_all_records_from_namespace(db, namespace, message):
selected_items_tr = select_all_item_query_example(db, namespace)
Expand All @@ -85,7 +87,7 @@ def transaction_example(db, namespace, items_in_base):
transaction.update(item)

# stop transaction and commit changes to namespace
transaction.commit()
transaction.commit(timedelta(milliseconds = 1000))

print_all_records_from_namespace(db, namespace, 'Transaction results count: ')

Expand All @@ -104,15 +106,15 @@ def query_example(db, namespace):
.where('value', CondType.CondEq, 'check')
.sort('id')
.limit(4)
.execute())
.execute(timedelta(milliseconds = 1000)))
print(f'Query results count (limited): {selected_items.count()}')
for item in selected_items:
print('Item: ', item)

# delete some items
del_count = (db.new_query(namespace)
.where('name', CondType.CondEq, 'item_1')
.delete())
.delete(timedelta(milliseconds = 1000)))
print(f'Deleted count: {del_count}')

# query all actual items
Expand Down
18 changes: 9 additions & 9 deletions pyreindexer/lib/include/pyobjtools.cc
Original file line number Diff line number Diff line change
Expand Up @@ -189,38 +189,38 @@ PyObject* pyValueFromJsonValue(const gason::JsonValue& value) {
PyObject* pyValue = nullptr;

switch (value.getTag()) {
case gason::JSON_NUMBER:
case gason::JsonTag::JSON_NUMBER:
pyValue = PyLong_FromLongLong(value.toNumber()); // new ref
break;
case gason::JSON_DOUBLE:
case gason::JsonTag::JSON_DOUBLE:
pyValue = PyFloat_FromDouble(value.toDouble()); // new ref
break;
case gason::JSON_STRING: {
case gason::JsonTag::JSON_STRING: {
auto sv = value.toString();
pyValue = PyUnicode_FromStringAndSize(sv.data(), sv.size()); // new ref
break;
}
case gason::JSON_NULL:
case gason::JsonTag::JSON_NULL:
pyValue = Py_None;
Py_INCREF(pyValue); // new ref
break;
case gason::JSON_TRUE:
case gason::JsonTag::JSON_TRUE:
pyValue = Py_True;
Py_INCREF(pyValue); // new ref
break;
case gason::JSON_FALSE:
case gason::JsonTag::JSON_FALSE:
pyValue = Py_False;
Py_INCREF(pyValue); // new ref
break;
case gason::JSON_ARRAY:
case gason::JsonTag::JSON_ARRAY:
pyValue = PyList_New(0); // new ref
for (const auto& v : value) {
PyObject* dictFromJson = pyValueFromJsonValue(v.value); // stolen ref
PyList_Append(pyValue, dictFromJson); // new ref
Py_XDECREF(dictFromJson);
}
break;
case gason::JSON_OBJECT:
case gason::JsonTag::JSON_OBJECT:
pyValue = PyDict_New(); // new ref
for (const auto& v : value) {
PyObject* dictFromJson = pyValueFromJsonValue(v.value); // stolen ref
Expand All @@ -230,7 +230,7 @@ PyObject* pyValueFromJsonValue(const gason::JsonValue& value) {
Py_XDECREF(dictFromJson);
}
break;
case gason::JSON_EMPTY:
case gason::JsonTag::JSON_EMPTY:
throw gason::Exception("Unexpected `JSON_EMPTY` tag");
break;
}
Expand Down
14 changes: 8 additions & 6 deletions pyreindexer/lib/include/query_wrapper.cc
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,8 @@ reindexer::Error QueryWrapper::buildQuery(reindexer::Query& query) {
return error;
}

reindexer::Error QueryWrapper::SelectQuery(std::unique_ptr<QueryResultsWrapper>& qr) {
reindexer::Error QueryWrapper::SelectQuery(std::unique_ptr<QueryResultsWrapper>& qr,
std::chrono::milliseconds timeout) {
reindexer::Query query;
auto err = buildQuery(query);
if (!err.ok()) {
Expand All @@ -232,26 +233,27 @@ reindexer::Error QueryWrapper::SelectQuery(std::unique_ptr<QueryResultsWrapper>&
}

qr = std::make_unique<QueryResultsWrapper>(db_);
return db_->SelectQuery(query, *qr);
return db_->SelectQuery(query, *qr, timeout);
}

reindexer::Error QueryWrapper::UpdateQuery(std::unique_ptr<QueryResultsWrapper>& qr) {
reindexer::Error QueryWrapper::UpdateQuery(std::unique_ptr<QueryResultsWrapper>& qr,
std::chrono::milliseconds timeout) {
reindexer::Query query;
auto err = buildQuery(query);
if (!err.ok()) {
return err;
}
qr = std::make_unique<QueryResultsWrapper>(db_);
return db_->UpdateQuery(query, *qr);
return db_->UpdateQuery(query, *qr, timeout);
}

reindexer::Error QueryWrapper::DeleteQuery(size_t& count) {
reindexer::Error QueryWrapper::DeleteQuery(size_t& count, std::chrono::milliseconds timeout) {
reindexer::Query query;
auto err = buildQuery(query);
if (!err.ok()) {
return err;
}
return db_->DeleteQuery(query, count);
return db_->DeleteQuery(query, count, timeout);
}

void QueryWrapper::Set(std::string_view field, const reindexer::VariantArray& values,
Expand Down
6 changes: 3 additions & 3 deletions pyreindexer/lib/include/query_wrapper.h
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,9 @@ class QueryWrapper {

void Modifier(QueryItemType type);

reindexer::Error SelectQuery(std::unique_ptr<QueryResultsWrapper>& qr);
reindexer::Error UpdateQuery(std::unique_ptr<QueryResultsWrapper>& qr);
reindexer::Error DeleteQuery(size_t& count);
reindexer::Error SelectQuery(std::unique_ptr<QueryResultsWrapper>& qr, std::chrono::milliseconds timeout);
reindexer::Error UpdateQuery(std::unique_ptr<QueryResultsWrapper>& qr, std::chrono::milliseconds timeout);
reindexer::Error DeleteQuery(size_t& count, std::chrono::milliseconds timeout);

enum class IsExpression { Yes, No };
void Set(std::string_view field, const reindexer::VariantArray& values, IsExpression isExpression);
Expand Down
4 changes: 2 additions & 2 deletions pyreindexer/lib/include/queryresults_wrapper.h
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ class QueryResultsWrapper {
it_ = qres_->begin();
}

Error Select(const std::string& query) {
return db_->Select(query, *this);
Error Select(std::string_view query, std::chrono::milliseconds timeout) {
return db_->Select(query, *this, timeout);
}

Error Status() {
Expand Down
12 changes: 6 additions & 6 deletions pyreindexer/lib/include/transaction_wrapper.h
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ class TransactionWrapper {
transaction_ = std::move(transaction);
}

Error Start(std::string_view ns) {
return db_->StartTransaction(ns, *this);
Error Start(std::string_view ns, std::chrono::milliseconds timeout) {
return db_->StartTransaction(ns, *this, timeout);
}

ItemT NewItem() {
Expand All @@ -47,14 +47,14 @@ class TransactionWrapper {
return db_->Modify(*transaction_, std::move(item), mode);
}

Error Commit(size_t& count) {
Error Commit(size_t& count, std::chrono::milliseconds timeout) {
assert(transaction_.has_value());
return db_->CommitTransaction(*transaction_, count);
return db_->CommitTransaction(*transaction_, count, timeout);
}

Error Rollback() {
Error Rollback(std::chrono::milliseconds timeout) {
assert(transaction_.has_value());
return db_->RollbackTransaction(*transaction_);
return db_->RollbackTransaction(*transaction_, timeout);
}

private:
Expand Down
Loading