Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rule schema key #48

Merged
merged 137 commits into from
Jan 29, 2025
Merged
Changes from 1 commit
Commits
Show all changes
137 commits
Select commit Hold shift + click to select a range
862778c
test(toKey): add check step value
mcakircali Jul 24, 2024
d60aeb8
feat(list): cherry-pick level
mcakircali Jul 29, 2024
d23778c
test(fdb-list): add new
mcakircali Jul 30, 2024
3be1f22
test(fdb-list): prepare grib file
mcakircali Jul 30, 2024
e0a0421
fix(fdb-list): init class member
mcakircali Jul 30, 2024
c2703ab
fix(fdb-list): unnecessary copy in loop
mcakircali Jul 30, 2024
36e6a54
fix(fdb-list): use make_unique not "new"
mcakircali Jul 30, 2024
77e3d77
fix(fdb-list): use "override" (not virtual)
mcakircali Jul 30, 2024
1e3381b
fix(fdb-list): default member init and tab size
mcakircali Jul 30, 2024
7f6be88
test(fdb-list): added --depth option
mcakircali Jul 30, 2024
5dfb6a0
fix(ListVisitor): empty keys
mcakircali Jul 30, 2024
71bbf51
style(fdb-list): clean code
mcakircali Jul 30, 2024
c4ebc5b
fix(fdb-list): porcelain defaults other options
mcakircali Jul 30, 2024
8ce657e
chore(remote/Handler): todo list w/ level
mcakircali Jul 30, 2024
8d9db4e
feat(list)!: separate ListElement
mcakircali Aug 5, 2024
fb4b8a0
fix(ListVisitor): use discrete ListElement ctor for DB, Cat, and Datum
mcakircali Aug 5, 2024
2c2b79d
fix(FDB): missing include
mcakircali Aug 5, 2024
31edc16
fix(FDB): name collisions
mcakircali Aug 5, 2024
376d842
fix(FDB): unnecessary copy in loop
mcakircali Aug 5, 2024
4a81dad
fix(FDB): after new ListElement
mcakircali Aug 5, 2024
e5edc6c
fix(FDB): 'some' include issues
mcakircali Aug 5, 2024
81a3924
chore(c api): param should be sha1
mcakircali Aug 5, 2024
6d67ae6
fix(c api): splitkey iterator
mcakircali Aug 5, 2024
b68231b
fix(fdb-patch): after ListElement
mcakircali Aug 5, 2024
a0cb5fa
fix(fdb-list): includes
mcakircali Aug 5, 2024
dbf974e
feat(fdb-list): options; assert depth and porcelain text
mcakircali Aug 5, 2024
4595fa7
fix(fdb-list): keys api
mcakircali Aug 5, 2024
7c42889
fix(FieldLocation): remove nullLocation
mcakircali Aug 5, 2024
f41ca4d
fix(MultiRetrieveVisitor): include issues
mcakircali Aug 5, 2024
70768b9
feat(remote): added level to Handler
mcakircali Aug 5, 2024
d4ebfa9
fix(RemoteFDB): fix include issues
mcakircali Aug 5, 2024
7ff9f39
fix(RemoteFDB): fix ListElement
mcakircali Aug 5, 2024
dfc1204
fix(LocalFDB): cleanup include
mcakircali Aug 5, 2024
7cc6f33
test(service): fix after ListElement with levels
mcakircali Aug 5, 2024
e085f57
test(fdb-list): fix after new output format
mcakircali Aug 5, 2024
bc0d333
fix(fdb-list): update porcelain text
mcakircali Aug 6, 2024
6d3fc15
fix(c api): added depth to fdb_list
mcakircali Aug 6, 2024
713c8f3
fix(ListElement): initial size = 0
mcakircali Aug 6, 2024
f1cc638
fix(c api): guard offset/len < 0
mcakircali Aug 6, 2024
24b70de
fix(fdb-list): updated doc for depth option
mcakircali Aug 7, 2024
39b27cd
test(fdb-list): rename cf -> an
mcakircali Aug 9, 2024
d8c3ec2
test(fdb-list): moved it to its own directory
mcakircali Aug 9, 2024
38c24d3
fix(c api): revert fdb_splitkey_t API
mcakircali Aug 13, 2024
12b398f
fix(c api): make fdb_splitkey_t work
mcakircali Aug 13, 2024
8cd9711
fix(c api): removed junk find()
mcakircali Aug 13, 2024
376d511
style(c api): typo
mcakircali Aug 13, 2024
de79888
feat(ListElement): throw with a msg
mcakircali Aug 13, 2024
59fdb36
feat(ListElement): added KeyChain (aka SplitKey)
mcakircali Aug 13, 2024
8e209ea
style: preserve api
mcakircali Aug 14, 2024
bea681d
style: replace auto
mcakircali Aug 14, 2024
7e73116
fix(fdb-list): param defaults and throw for porcelain option
mcakircali Aug 14, 2024
05890a3
Merge branch 'develop' into feature/list-depth-option
mcakircali Aug 14, 2024
346a47f
fix(c api): fdb_list assert bug
mcakircali Aug 14, 2024
7f72cc3
fix(ListElement): offset/length returns 0 for non-datum elements
mcakircali Aug 14, 2024
14d0f74
test(c api): added fdb_list with depth=1,2,3
mcakircali Aug 14, 2024
cbef3e9
fix(ListElement): bug print length
mcakircali Aug 14, 2024
75bcf1c
fix(ListElement): api issues around FieldLocation shared_ptr
mcakircali Aug 15, 2024
d638c98
docs(fdb-list): corrected depth range
mcakircali Aug 15, 2024
25e03d1
test(fdb-list): rename cf -> an
mcakircali Aug 9, 2024
2b7b185
feat(fdb-inspect): parallel option
mcakircali Aug 9, 2024
adfd8fd
test(fdb-inspect): start simple test
mcakircali Aug 9, 2024
34915b3
feat(fdb-inspect): output option
mcakircali Aug 14, 2024
c15f4f0
feat(EntryVisitMechanism): added previsit
mcakircali Aug 10, 2024
d098f1a
feat(Visitor): added preVisitDatabase to List and Axes
mcakircali Aug 11, 2024
6b7430a
fix(RetrieveVisitor): private -> protected
mcakircali Aug 11, 2024
177b96a
feat(rules): cherry-picked Rule and Schema from 'tree-server'
mcakircali Aug 11, 2024
03f2990
fix(EntryVisitMechanism): pass schema to preVisitDatabase
mcakircali Aug 11, 2024
8f56bc4
fix(AxesVisitor): copy mistake
mcakircali Aug 11, 2024
59cd746
fix(Schema): missing override
mcakircali Aug 11, 2024
c33208a
fix(EntryVisitMechanism): fix loop copy
mcakircali Aug 11, 2024
8bd5231
fix(EntryVisitMechanism): includes
mcakircali Aug 11, 2024
110120c
fix(Catalogue): deferred reading indexes
mcakircali Aug 12, 2024
24c7756
fix(Catalogue): includes
mcakircali Aug 12, 2024
340b481
fix(Catalogue): memory leak
mcakircali Aug 12, 2024
ac1521f
fix(database): recursive include
mcakircali Aug 12, 2024
8f6afcb
refactor(database): convert StringSet -> DenseSet
mcakircali Aug 12, 2024
e6dd5e8
fix(Index): added mayContainPartial
mcakircali Aug 12, 2024
6bc6c38
fix(IndexAxis): added containsPartial
mcakircali Aug 12, 2024
d41e921
feat(TocCatalogueReader): added cached key-matching and fixes
mcakircali Aug 13, 2024
f014b66
feat(rules): match by value
mcakircali Aug 27, 2024
9251b93
feat(Schema): use KeyChain
mcakircali Aug 30, 2024
4922334
refactor(rules): expand cleanup
mcakircali Sep 6, 2024
786e8ea
fix(rules): Schema headers
mcakircali Sep 7, 2024
806f000
refactor(rules): Schema expandSecond -> expandIndex
mcakircali Sep 9, 2024
18733b7
feat(rules): add Key::pushFrom/popFrom other keys
mcakircali Sep 16, 2024
385badd
fix(rules): optimize range loop
mcakircali Sep 16, 2024
b912ca5
fix(rules): turn off level stuff
mcakircali Sep 16, 2024
d2060b4
fix(rules): reworked Rule::expand and Schema::expand on ReadVisitor path
mcakircali Sep 16, 2024
5787faf
fix(rules): removed level stuff
mcakircali Sep 16, 2024
8ebb67a
fix(rules): add find matching keys and cleanup
mcakircali Sep 16, 2024
dbebbac
feat(rules): add expandDatabase and matchDatabase to Schema
mcakircali Sep 16, 2024
9f5aba3
fix(rules): axes and list visitors
mcakircali Sep 16, 2024
b1114f5
fix(rules): tools -> fdb-hide fdb-overlay and fdb-root
mcakircali Sep 16, 2024
fac8a15
refactor(rules): Schema
mcakircali Sep 16, 2024
c246cdc
fix(rules): Rule and Schema
mcakircali Sep 17, 2024
e2e4599
refactor(Schema): rename matchDatabase
mcakircali Sep 19, 2024
77a12d4
fix(rules): cleanup Rule
mcakircali Sep 19, 2024
86a1422
fix(rules): use automatic storage (not dynamic)
mcakircali Sep 20, 2024
f5b4442
fix(daos): adapt tests
mcakircali Sep 23, 2024
25007ef
feat(rules): parse and expand each level individually
mcakircali Sep 23, 2024
f82255d
refactor(rules): firstlevel -> database
mcakircali Sep 23, 2024
f9e769c
Merge branch 'develop' into feature/toc-mapped-indexes
mcakircali Sep 23, 2024
24ce773
Merge branch 'develop' into feature/toc-mapped-indexes
mcakircali Sep 24, 2024
9f71b93
fix(merge): Key Schema and TypesRegistry
mcakircali Sep 24, 2024
ded516e
fix(merge): remove unnecessary temp object
mcakircali Oct 6, 2024
9a366c7
fix(merge): remove unnecessary temp object
mcakircali Oct 6, 2024
56fcee6
fix(ListIterator): ListElement::combinedKey
mcakircali Oct 8, 2024
719fdb8
fix(merge): typedkey cleanup
mcakircali Oct 8, 2024
1ce0504
refactor(AxesVisitor): cosmetic
mcakircali Oct 11, 2024
cf78f9e
fix(merge): canonicalize request on list
mcakircali Oct 11, 2024
8f45355
fix(c api): guard next metadata
mcakircali Oct 11, 2024
054f5d0
fix(daos): constness thinko
mcakircali Oct 11, 2024
99c02eb
fix(merge): Key and registry issues
mcakircali Oct 11, 2024
65d89c5
fix(merge): test
mcakircali Oct 11, 2024
d81aa3c
fix(merge): cleanup
mcakircali Oct 11, 2024
39fa4ef
fix(merge): BaseKey popFrom
mcakircali Oct 14, 2024
fb48204
fix(merge): test_toKey
mcakircali Oct 14, 2024
4523f84
Merge branch 'develop' into feature/rule-schema-key
mcakircali Nov 11, 2024
9c6ba85
fix(tools): clenaup headers
mcakircali Nov 11, 2024
bb55e9f
fix(Key): templating issue
mcakircali Nov 11, 2024
0582c82
Merge branch 'develop' into feature/rule-schema-key
mcakircali Nov 15, 2024
e29a7ec
fix: merge feature/gribjump-performance
mcakircali Nov 15, 2024
73950fd
Merge branch 'develop' into feature/rule-schema-key
mcakircali Dec 6, 2024
3fb501c
Merge branch 'remoteFDB' into feature/rule-schema-key-remote
mcakircali Dec 10, 2024
b93e4e0
fix(merge): various issues
mcakircali Dec 10, 2024
906d700
feat(depth): add level to RemoteFDB::list
mcakircali Dec 10, 2024
a363005
fix(TypesRegistry): cleanup
mcakircali Dec 10, 2024
945dda0
fixed remoteFDb merge (schema serialisation)
danovaro Dec 11, 2024
4e185be
Merge remote-tracking branch 'refs/remotes/origin/feature/rule-schema…
mcakircali Dec 11, 2024
d076ce3
Merge branch 'remoteFDB' into feature/rule-schema-key-remote
danovaro Dec 11, 2024
50df580
Merge branch 'remoteFDB' into feature/rule-schema-key-remote
danovaro Dec 11, 2024
9ee12ea
removed unused files + reverted to remoteFDB Schema/Rule serialisation
danovaro Dec 17, 2024
0ddede6
Merge branch 'remoteFDB' into feature/rule-schema-key-remote
danovaro Jan 6, 2025
cf0aadd
fdb key encoding estimate
danovaro Jan 7, 2025
355f994
Merge branch 'remoteFDB' into feature/rule-schema-key-remote
danovaro Jan 7, 2025
2df89dd
Merge branch 'develop' into feature/rule-schema-key
danovaro Jan 29, 2025
4c179a0
fix fdb_list in c api
danovaro Jan 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix(FDB): name collisions
mcakircali committed Aug 8, 2024
commit 31edc16dc8522f33fe1298abc1e082c86c5e5cbe
8 changes: 4 additions & 4 deletions src/fdb5/api/FDB.cc
Original file line number Diff line number Diff line change
@@ -190,10 +190,10 @@ eckit::DataHandle* FDB::read(ListIterator& it, bool sorted) {
}

// checking all retrieved fields against the hypercube, to remove duplicates
ListElementDeduplicator dedup;
metkit::hypercube::HyperCubePayloaded<ListElement> cube(cubeRequest, dedup);
for(auto el: elements) {
cube.add(el.combinedKey().request(), el);
ListElementDeduplicator deduplicator;
metkit::hypercube::HyperCubePayloaded<ListElement> cube(cubeRequest, deduplicator);
for(auto elem: elements) {
cube.add(elem.combinedKey().request(), el);
}

if (cube.countVacant() > 0) {