Mondrian Version #692
Comments
|
We have a branch we add stuff to. Mondrian 3/8 is hardly maintained these days either, small bug fixes or db support, I'd consider them to be feature complete for our needs. Switching back to Mondrian 3 would be a PITA for our existing users, but there may be scope to write a 2nd backend later in the year for Saiku 4 |
|
I tried to compile mondrian 4.x to fix a (very old) issue with vertica that prevents saiku to work with it (https://jira.pentaho.com/plugins/servlet/mobile#issue/MONDRIAN-2249) but did not succeed. Looks like nobody has compiled that version for a long time, it is full of dust:) In mondrian 3/8 it works so I thought it could be possible to cherry pick the fix and apply it to 4.x... |
|
Bruno has a branch that compiles and might be able to help as he's working
on kylin support at the moment
…On Tue, 26 Feb 2019, 22:12 antomgarre, ***@***.***> wrote:
I tried to compile mondrian 4.x to fix a (very old) issue with vertica
that prevents saiku to work with it (
https://jira.pentaho.com/plugins/servlet/mobile#issue/MONDRIAN-2249) but
did not succeed. Looks like nobody has compiled that version for a long
time, it is full of dust:)
In mondrian 3/8 it works so I thought it could be possible to cherry pick
the fix and apply it to 4.x...
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#692 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAGUeL4Ho5iCBpnQcPwHhc0gHgOUzOwjks5vRbE3gaJpZM4bSCdO>
.
--
Spicule Limited is registered in England & Wales. Company Number:
09954122. Registered office: First Floor, Telecom House, 125-135 Preston
Road, Brighton, England, BN1 6AF. VAT No. 251478891.
All engagements
are subject to Spicule Terms and Conditions of Business. This email and its
contents are intended solely for the individual to whom it is addressed and
may contain information that is confidential, privileged or otherwise
protected from disclosure, distributing or copying. Any views or opinions
presented in this email are solely those of the author and do not
necessarily represent those of Spicule Limited. The company accepts no
liability for any damage caused by any virus transmitted by this email. If
you have received this message in error, please notify us immediately by
reply email before deleting it from your system. Service of legal notice
cannot be effected on Spicule Limited by email.
|
|
Thank you, I could clone Bruno's branch and compile it, and fix the vertica integration problem with INT/LONG types. I am testing it but looks ok. |
|
Sweet! PR’s are welcome! ;)
On 27 February 2019 at 13:00:11, antomgarre ([email protected]) wrote:
Thank you, I could clone Bruno's branch and compile it, and fix the vertica
integration problem with INT/LONG types. I am testing it but looks ok.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#692 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAGUeHqniaHwNMTFdvTJqjrnBbNWlgZIks5vRoFbgaJpZM4bSCdO>
.
…--
Spicule Limited is registered in England & Wales. Company Number:
09954122. Registered office: First Floor, Telecom House, 125-135 Preston
Road, Brighton, England, BN1 6AF. VAT No. 251478891.
All engagements
are subject to Spicule Terms and Conditions of Business. This email and its
contents are intended solely for the individual to whom it is addressed and
may contain information that is confidential, privileged or otherwise
protected from disclosure, distributing or copying. Any views or opinions
presented in this email are solely those of the author and do not
necessarily represent those of Spicule Limited. The company accepts no
liability for any damage caused by any virus transmitted by this email. If
you have received this message in error, please notify us immediately by
reply email before deleting it from your system. Service of legal notice
cannot be effected on Spicule Limited by email.
|
|
Of course! :) Thank you! |
|
Hello Tom!
After having fixed the mondiran bug with Saiku+Vertica, the performance is
sweet, works great with 1 large dimension, but looks like it falls
drastically when using a few large dimensions (100k-150k)
BD queries are resolved fast, but saiku takes a long time to process each
query, and it makes a lot of separate queries.
Do you think on any optimization can be done? something like 'delegate more
work to DB'?
Thank you!
El mié., 27 feb. 2019 a las 14:07, Tom Barber (<[email protected]>)
escribió:
… Sweet! PR’s are welcome! ;)
On 27 February 2019 at 13:00:11, antomgarre ***@***.***)
wrote:
Thank you, I could clone Bruno's branch and compile it, and fix the vertica
integration problem with INT/LONG types. I am testing it but looks ok.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#692 (comment)>, or mute
the thread
<
https://github.com/notifications/unsubscribe-auth/AAGUeHqniaHwNMTFdvTJqjrnBbNWlgZIks5vRoFbgaJpZM4bSCdO
>
.
--
Spicule Limited is registered in England & Wales. Company Number:
09954122. Registered office: First Floor, Telecom House, 125-135 Preston
Road, Brighton, England, BN1 6AF. VAT No. 251478891.
All engagements
are subject to Spicule Terms and Conditions of Business. This email and
its
contents are intended solely for the individual to whom it is addressed
and
may contain information that is confidential, privileged or otherwise
protected from disclosure, distributing or copying. Any views or opinions
presented in this email are solely those of the author and do not
necessarily represent those of Spicule Limited. The company accepts no
liability for any damage caused by any virus transmitted by this email. If
you have received this message in error, please notify us immediately by
reply email before deleting it from your system. Service of legal notice
cannot be effected on Spicule Limited by email.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#692 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AaCg9xrTFrE0x0_0BdT5OimYNpTuz8Zzks5vRoL3gaJpZM4bSCdO>
.
--
-----------------------------------------------------------------------
Antonio Martín Garre
[email protected]
"En los momentos de crisis, sólo la imaginación
es más importante que el conocimiento"
- A. Einstein
-----------------------------------------------------------------------
|
|
Hi Antonio,
Can you do me a favour, take the MDX from one of the slow queries, then rewrite the cross joins so you remove the * and instead write them long form and use Crossjoin() in a test MDX query and see what the execution time is like.
Thanks
Tom
On 17 June 2019 at 08:34:50, Antonio Martín Garre ([email protected]) wrote:
Hello Tom!
After having fixed the mondiran bug with Saiku+Vertica, the performance is
sweet, works great with 1 large dimension, but looks like it falls
drastically when using a few large dimensions (100k-150k)
BD queries are resolved fast, but saiku takes a long time to process each
query, and it makes a lot of separate queries.
Do you think on any optimization can be done? something like 'delegate more
work to DB'?
Thank you!
El mié., 27 feb. 2019 a las 14:07, Tom Barber (<[email protected]>)
escribió:
… Sweet! PR’s are welcome! ;)
On 27 February 2019 at 13:00:11, antomgarre ***@***.***)
wrote:
Thank you, I could clone Bruno's branch and compile it, and fix the vertica
integration problem with INT/LONG types. I am testing it but looks ok.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#692 (comment)>, or mute
the thread
<
https://github.com/notifications/unsubscribe-auth/AAGUeHqniaHwNMTFdvTJqjrnBbNWlgZIks5vRoFbgaJpZM4bSCdO
>
.
--
Spicule Limited is registered in England & Wales. Company Number:
09954122. Registered office: First Floor, Telecom House, 125-135 Preston
Road, Brighton, England, BN1 6AF. VAT No. 251478891.
All engagements
are subject to Spicule Terms and Conditions of Business. This email and
its
contents are intended solely for the individual to whom it is addressed
and
may contain information that is confidential, privileged or otherwise
protected from disclosure, distributing or copying. Any views or opinions
presented in this email are solely those of the author and do not
necessarily represent those of Spicule Limited. The company accepts no
liability for any damage caused by any virus transmitted by this email. If
you have received this message in error, please notify us immediately by
reply email before deleting it from your system. Service of legal notice
cannot be effected on Spicule Limited by email.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#692 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AaCg9xrTFrE0x0_0BdT5OimYNpTuz8Zzks5vRoL3gaJpZM4bSCdO>
.
--
-----------------------------------------------------------------------
Antonio Martín Garre
[email protected]
"En los momentos de crisis, sólo la imaginación
es más importante que el conocimiento"
- A. Einstein
-----------------------------------------------------------------------
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.
--
Spicule Limited is registered in England & Wales. Company Number:
09954122. Registered office: First Floor, Telecom House, 125-135 Preston
Road, Brighton, England, BN1 6AF. VAT No. 251478891.
All engagements
are subject to Spicule Terms and Conditions of Business. This email and its
contents are intended solely for the individual to whom it is addressed and
may contain information that is confidential, privileged or otherwise
protected from disclosure, distributing or copying. Any views or opinions
presented in this email are solely those of the author and do not
necessarily represent those of Spicule Limited. The company accepts no
liability for any damage caused by any virus transmitted by this email. If
you have received this message in error, please notify us immediately by
reply email before deleting it from your system. Service of legal notice
cannot be effected on Spicule Limited by email.
|
|
Hi Tom!
I will ,but the problem is not in the database. All queries take much less
than 1s. The problem is that the query is partitioned in segments, small
queries like:
DEBUG [mondrian.sql] 47: Segment.load: executing sql [select "column1 as
c1, column2 as c2... columnN as CN where "BIG_Dimension1"."Dim1_SK"* in
(163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177,
178, 179, 180)*
There are a lot of SQLs like this, If dimension "BIG_Dimension1" has
100.000 rows....
May be there is a way to make less segments (with more elements in the
where clause) ?
Regards,
El lun., 17 jun. 2019 a las 9:47, Tom Barber (<[email protected]>)
escribió:
… Hi Antonio,
Can you do me a favour, take the MDX from one of the slow queries, then
rewrite the cross joins so you remove the * and instead write them long
form and use Crossjoin() in a test MDX query and see what the execution
time is like.
Thanks
Tom
On 17 June 2019 at 08:34:50, Antonio Martín Garre (
***@***.***) wrote:
Hello Tom!
After having fixed the mondiran bug with Saiku+Vertica, the performance is
sweet, works great with 1 large dimension, but looks like it falls
drastically when using a few large dimensions (100k-150k)
BD queries are resolved fast, but saiku takes a long time to process each
query, and it makes a lot of separate queries.
Do you think on any optimization can be done? something like 'delegate more
work to DB'?
Thank you!
El mié., 27 feb. 2019 a las 14:07, Tom Barber ***@***.***>)
escribió:
> Sweet! PR’s are welcome! ;)
>
>
> On 27 February 2019 at 13:00:11, antomgarre ***@***.***)
> wrote:
>
> Thank you, I could clone Bruno's branch and compile it, and fix the
vertica
> integration problem with INT/LONG types. I am testing it but looks ok.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <#692 (comment)>, or
mute
> the thread
> <
>
https://github.com/notifications/unsubscribe-auth/AAGUeHqniaHwNMTFdvTJqjrnBbNWlgZIks5vRoFbgaJpZM4bSCdO
> >
> .
>
> --
>
>
> Spicule Limited is registered in England & Wales. Company Number:
> 09954122. Registered office: First Floor, Telecom House, 125-135 Preston
> Road, Brighton, England, BN1 6AF. VAT No. 251478891.
>
>
>
>
> All engagements
> are subject to Spicule Terms and Conditions of Business. This email and
> its
> contents are intended solely for the individual to whom it is addressed
> and
> may contain information that is confidential, privileged or otherwise
> protected from disclosure, distributing or copying. Any views or opinions
> presented in this email are solely those of the author and do not
> necessarily represent those of Spicule Limited. The company accepts no
> liability for any damage caused by any virus transmitted by this email.
If
> you have received this message in error, please notify us immediately by
> reply email before deleting it from your system. Service of legal notice
> cannot be effected on Spicule Limited by email.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#692 (comment)>, or
mute
> the thread
> <
https://github.com/notifications/unsubscribe-auth/AaCg9xrTFrE0x0_0BdT5OimYNpTuz8Zzks5vRoL3gaJpZM4bSCdO
>
> .
>
--
-----------------------------------------------------------------------
Antonio Martín Garre
***@***.***
"En los momentos de crisis, sólo la imaginación
es más importante que el conocimiento"
- A. Einstein
-----------------------------------------------------------------------
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.
--
Spicule Limited is registered in England & Wales. Company Number:
09954122. Registered office: First Floor, Telecom House, 125-135 Preston
Road, Brighton, England, BN1 6AF. VAT No. 251478891.
All engagements
are subject to Spicule Terms and Conditions of Business. This email and
its
contents are intended solely for the individual to whom it is addressed
and
may contain information that is confidential, privileged or otherwise
protected from disclosure, distributing or copying. Any views or opinions
presented in this email are solely those of the author and do not
necessarily represent those of Spicule Limited. The company accepts no
liability for any damage caused by any virus transmitted by this email. If
you have received this message in error, please notify us immediately by
reply email before deleting it from your system. Service of legal notice
cannot be effected on Spicule Limited by email.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#692?email_source=notifications&email_token=AGQKB535EBLCPN7SVE3BB3TP246QVA5CNFSM4G2IE5HKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX2KKXQ#issuecomment-502572382>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AGQKB56NSGKEIEE23NWTNUTP246QVANCNFSM4G2IE5HA>
.
--
-----------------------------------------------------------------------
Antonio Martín Garre
[email protected]
"En los momentos de crisis, sólo la imaginación
es más importante que el conocimiento"
- A. Einstein
-----------------------------------------------------------------------
|
|
Yeah I appreciate the database is fast. There's a few things to try but the
crossjoin test does affect SQL performance so it's a good one to test first
up.
On Mon, Jun 17, 2019, 09:38 Antonio Martín Garre <[email protected]>
wrote:
… Hi Tom!
I will ,but the problem is not in the database. All queries take much less
than 1s. The problem is that the query is partitioned in segments, small
queries like:
DEBUG [mondrian.sql] 47: Segment.load: executing sql [select "column1 as
c1, column2 as c2... columnN as CN where "BIG_Dimension1"."Dim1_SK"* in
(163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177,
178, 179, 180)*
There are a lot of SQLs like this, If dimension "BIG_Dimension1" has
100.000 rows....
May be there is a way to make less segments (with more elements in the
where clause) ?
Regards,
El lun., 17 jun. 2019 a las 9:47, Tom Barber ***@***.***>)
escribió:
> Hi Antonio,
>
> Can you do me a favour, take the MDX from one of the slow queries, then
> rewrite the cross joins so you remove the * and instead write them long
> form and use Crossjoin() in a test MDX query and see what the execution
> time is like.
>
> Thanks
>
> Tom
>
>
> On 17 June 2019 at 08:34:50, Antonio Martín Garre (
> ***@***.***) wrote:
>
> Hello Tom!
>
> After having fixed the mondiran bug with Saiku+Vertica, the performance
is
> sweet, works great with 1 large dimension, but looks like it falls
> drastically when using a few large dimensions (100k-150k)
> BD queries are resolved fast, but saiku takes a long time to process each
> query, and it makes a lot of separate queries.
>
> Do you think on any optimization can be done? something like 'delegate
more
> work to DB'?
>
> Thank you!
>
>
> El mié., 27 feb. 2019 a las 14:07, Tom Barber ***@***.***
>)
> escribió:
>
> > Sweet! PR’s are welcome! ;)
> >
> >
> > On 27 February 2019 at 13:00:11, antomgarre ***@***.***)
> > wrote:
> >
> > Thank you, I could clone Bruno's branch and compile it, and fix the
> vertica
> > integration problem with INT/LONG types. I am testing it but looks ok.
> >
> > —
> > You are receiving this because you commented.
> > Reply to this email directly, view it on GitHub
> > <#692 (comment)>, or
> mute
> > the thread
> > <
> >
>
https://github.com/notifications/unsubscribe-auth/AAGUeHqniaHwNMTFdvTJqjrnBbNWlgZIks5vRoFbgaJpZM4bSCdO
> > >
> > .
> >
> > --
> >
> >
> > Spicule Limited is registered in England & Wales. Company Number:
> > 09954122. Registered office: First Floor, Telecom House, 125-135
Preston
> > Road, Brighton, England, BN1 6AF. VAT No. 251478891.
> >
> >
> >
> >
> > All engagements
> > are subject to Spicule Terms and Conditions of Business. This email and
> > its
> > contents are intended solely for the individual to whom it is addressed
> > and
> > may contain information that is confidential, privileged or otherwise
> > protected from disclosure, distributing or copying. Any views or
opinions
> > presented in this email are solely those of the author and do not
> > necessarily represent those of Spicule Limited. The company accepts no
> > liability for any damage caused by any virus transmitted by this email.
> If
> > you have received this message in error, please notify us immediately
by
> > reply email before deleting it from your system. Service of legal
notice
> > cannot be effected on Spicule Limited by email.
> >
> > —
> > You are receiving this because you authored the thread.
> > Reply to this email directly, view it on GitHub
> > <#692 (comment)>, or
> mute
> > the thread
> > <
>
https://github.com/notifications/unsubscribe-auth/AaCg9xrTFrE0x0_0BdT5OimYNpTuz8Zzks5vRoL3gaJpZM4bSCdO
> >
> > .
> >
>
>
> --
> -----------------------------------------------------------------------
> Antonio Martín Garre
> ***@***.***
>
> "En los momentos de crisis, sólo la imaginación
> es más importante que el conocimiento"
> - A. Einstein
> -----------------------------------------------------------------------
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or mute the thread.
> --
>
>
> Spicule Limited is registered in England & Wales. Company Number:
> 09954122. Registered office: First Floor, Telecom House, 125-135 Preston
> Road, Brighton, England, BN1 6AF. VAT No. 251478891.
>
>
>
>
> All engagements
> are subject to Spicule Terms and Conditions of Business. This email and
> its
> contents are intended solely for the individual to whom it is addressed
> and
> may contain information that is confidential, privileged or otherwise
> protected from disclosure, distributing or copying. Any views or opinions
> presented in this email are solely those of the author and do not
> necessarily represent those of Spicule Limited. The company accepts no
> liability for any damage caused by any virus transmitted by this email.
If
> you have received this message in error, please notify us immediately by
> reply email before deleting it from your system. Service of legal notice
> cannot be effected on Spicule Limited by email.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <
#692?email_source=notifications&email_token=AGQKB535EBLCPN7SVE3BB3TP246QVA5CNFSM4G2IE5HKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX2KKXQ#issuecomment-502572382
>,
> or mute the thread
> <
https://github.com/notifications/unsubscribe-auth/AGQKB56NSGKEIEE23NWTNUTP246QVANCNFSM4G2IE5HA
>
> .
>
--
-----------------------------------------------------------------------
Antonio Martín Garre
***@***.***
"En los momentos de crisis, sólo la imaginación
es más importante que el conocimiento"
- A. Einstein
-----------------------------------------------------------------------
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#692?email_source=notifications&email_token=AAAZI6AJF5262AOZZBBUCPLP25EQHA5CNFSM4G2IE5HKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX2ONJY#issuecomment-502589095>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAAZI6BAFSKJCJBJW3JDX3LP25EQHANCNFSM4G2IE5HA>
.
--
Spicule Limited is registered in England & Wales. Company Number:
09954122. Registered office: First Floor, Telecom House, 125-135 Preston
Road, Brighton, England, BN1 6AF. VAT No. 251478891.
All engagements
are subject to Spicule Terms and Conditions of Business. This email and its
contents are intended solely for the individual to whom it is addressed and
may contain information that is confidential, privileged or otherwise
protected from disclosure, distributing or copying. Any views or opinions
presented in this email are solely those of the author and do not
necessarily represent those of Spicule Limited. The company accepts no
liability for any damage caused by any virus transmitted by this email. If
you have received this message in error, please notify us immediately by
reply email before deleting it from your system. Service of legal notice
cannot be effected on Spicule Limited by email.
|
|
I have done the test MDX, with a few columns using the crossjoin.
The results are similar.
Queries are a little bit faster, but there are again many, following this
pattern:
...
2019-06-17 10:41:22,122 DEBUG [mondrian.sql] 86: , exec 580 ms
2019-06-17 10:41:22,153 DEBUG [mondrian.sql] 86: , exec+fetch 609 ms, 1047
rows
2019-06-17 10:42:02,700 DEBUG [mondrian.sql] 87: Segment.load: executing
sql [select ... where dim_sk in (883, 884, 885, 886, 887, 888, 889, 890,
891, 892, 893, 894, 895, 896, 897, 898, 899, 900) and ...
2019-06-17 10:42:03,169 DEBUG [mondrian.sql] 87: , exec 473 ms
2019-06-17 10:42:03,184 DEBUG [mondrian.sql] 87: , exec+fetch 484 ms, 1030
rows
2019-06-17 10:42:46,919 DEBUG [mondrian.sql] 88: Segment.load: executing
sql [select ... where dim_sk in (901, 902, 903, 904, 905, 906, 907, 908,
909, 910, 911, 912, 913, 914, 915, 916, 917, 918) and ...
2019-06-17 10:42:47,450 DEBUG [mondrian.sql] 88: , exec 540 ms
2019-06-17 10:42:47,465 DEBUG [mondrian.sql] 88: , exec+fetch 546 ms, 1018
rows
2019-06-17 10:43:31,590 DEBUG [mondrian.sql] 89: Segment.load: executing
sql [select ... where dim_sk in (919, 920, 921, 922, 923, 924, 925, 926,
927, 928, 929, 930, 931, 932, 933, 934, 935, 936) and ...
2019-06-17 10:43:32,090 DEBUG [mondrian.sql] 89: , exec 483 ms
2019-06-17 10:43:32,090 DEBUG [mondrian.sql] 89: , exec+fetch 500 ms, 546
rows
2019-06-17 10:44:18,106 DEBUG [mondrian.sql] 90: Segment.load: executing
sql [select ... where dim_sk in (937, 938, 939, 940, 941, 942, 943, 944,
945, 946, 947, 948, 949, 950, 951, 952, 953, 954) and ...
2019-06-17 10:44:18,622 DEBUG [mondrian.sql] 90: , exec 523 ms
2019-06-17 10:44:18,637 DEBUG [mondrian.sql] 90: , exec+fetch 531 ms, 803
rows
2019-06-17 10:45:09,184 DEBUG [mondrian.sql] 91: Segment.load: executing
sql [select ... where dim_sk in (955, 956, 957, 958, 959, 960, 961, 962,
963, 964, 965, 966, 967, 968, 969, 970, 971, 972) and ...
2019-06-17 10:45:09,809 DEBUG [mondrian.sql] 91: , exec 614 ms
2019-06-17 10:45:09,825 DEBUG [mondrian.sql] 91: , exec+fetch 641 ms, 987
rows
2019-06-17 10:45:58,997 DEBUG [mondrian.sql] 92: Segment.load: executing
sql [select ... where dim_sk in (973, 974, 975, 976, 977, 978, 979, 980,
981, 982, 983, 984, 985, 986, 987, 988, 989, 990) and ...
2019-06-17 10:45:59,544 DEBUG [mondrian.sql] 92: , exec 526 ms
2019-06-17 10:45:59,559 DEBUG [mondrian.sql] 92: , exec+fetch 562 ms, 859
rows
2019-06-17 10:46:51,137 DEBUG [mondrian.sql] 93: Segment.load: executing
sql [select ... where dim_sk in (991, 992, 993, 994, 995, 996, 997, 998,
999, 1000, 1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008) and ...
2019-06-17 10:46:51,653 DEBUG [mondrian.sql] 93: , exec 507 ms
2019-06-17 10:46:51,669 DEBUG [mondrian.sql] 93: , exec+fetch 532 ms, 738
rows
2019-06-17 10:47:46,606 DEBUG [mondrian.sql] 94: Segment.load: executing
sql [select ... where dim_sk in (1009, 1010, 1011, 1012, 1013, 1014, 1015,
1016, 1017, 1018, 1019, 1020, 1021, 1022, 1023, 1024, 1025, 1026) and ...
2019-06-17 10:47:47,122 DEBUG [mondrian.sql] 94: , exec 521 ms
2019-06-17 10:47:47,137 DEBUG [mondrian.sql] 94: , exec+fetch 531 ms, 403
rows
2019-06-17 10:48:42,434 DEBUG [mondrian.sql] 95: Segment.load: executing
sql [select ... where dim_sk in (1027, 1028, 1029, 1030, 1031, 1032, 1033,
1034, 1035, 1036, 1037, 1038, 1039, 1040, 1041, 1042, 1043, 1044) and ...
....
El lun., 17 jun. 2019 a las 10:42, Tom Barber (<[email protected]>)
escribió:
… Yeah I appreciate the database is fast. There's a few things to try but the
crossjoin test does affect SQL performance so it's a good one to test first
up.
On Mon, Jun 17, 2019, 09:38 Antonio Martín Garre ***@***.***
>
wrote:
> Hi Tom!
>
> I will ,but the problem is not in the database. All queries take much
less
> than 1s. The problem is that the query is partitioned in segments, small
> queries like:
>
> DEBUG [mondrian.sql] 47: Segment.load: executing sql [select "column1 as
> c1, column2 as c2... columnN as CN where "BIG_Dimension1"."Dim1_SK"* in
> (163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176,
177,
> 178, 179, 180)*
>
>
>
> There are a lot of SQLs like this, If dimension "BIG_Dimension1" has
> 100.000 rows....
>
> May be there is a way to make less segments (with more elements in the
> where clause) ?
>
> Regards,
>
>
>
> El lun., 17 jun. 2019 a las 9:47, Tom Barber ***@***.***
>)
> escribió:
>
> > Hi Antonio,
> >
> > Can you do me a favour, take the MDX from one of the slow queries, then
> > rewrite the cross joins so you remove the * and instead write them long
> > form and use Crossjoin() in a test MDX query and see what the execution
> > time is like.
> >
> > Thanks
> >
> > Tom
> >
> >
> > On 17 June 2019 at 08:34:50, Antonio Martín Garre (
> > ***@***.***) wrote:
> >
> > Hello Tom!
> >
> > After having fixed the mondiran bug with Saiku+Vertica, the performance
> is
> > sweet, works great with 1 large dimension, but looks like it falls
> > drastically when using a few large dimensions (100k-150k)
> > BD queries are resolved fast, but saiku takes a long time to process
each
> > query, and it makes a lot of separate queries.
> >
> > Do you think on any optimization can be done? something like 'delegate
> more
> > work to DB'?
> >
> > Thank you!
> >
> >
> > El mié., 27 feb. 2019 a las 14:07, Tom Barber (<
***@***.***
> >)
> > escribió:
> >
> > > Sweet! PR’s are welcome! ;)
> > >
> > >
> > > On 27 February 2019 at 13:00:11, antomgarre (
***@***.***)
> > > wrote:
> > >
> > > Thank you, I could clone Bruno's branch and compile it, and fix the
> > vertica
> > > integration problem with INT/LONG types. I am testing it but looks
ok.
> > >
> > > —
> > > You are receiving this because you commented.
> > > Reply to this email directly, view it on GitHub
> > > <#692 (comment)>,
or
> > mute
> > > the thread
> > > <
> > >
> >
>
https://github.com/notifications/unsubscribe-auth/AAGUeHqniaHwNMTFdvTJqjrnBbNWlgZIks5vRoFbgaJpZM4bSCdO
> > > >
> > > .
> > >
> > > --
> > >
> > >
> > > Spicule Limited is registered in England & Wales. Company Number:
> > > 09954122. Registered office: First Floor, Telecom House, 125-135
> Preston
> > > Road, Brighton, England, BN1 6AF. VAT No. 251478891.
> > >
> > >
> > >
> > >
> > > All engagements
> > > are subject to Spicule Terms and Conditions of Business. This email
and
> > > its
> > > contents are intended solely for the individual to whom it is
addressed
> > > and
> > > may contain information that is confidential, privileged or otherwise
> > > protected from disclosure, distributing or copying. Any views or
> opinions
> > > presented in this email are solely those of the author and do not
> > > necessarily represent those of Spicule Limited. The company accepts
no
> > > liability for any damage caused by any virus transmitted by this
email.
> > If
> > > you have received this message in error, please notify us immediately
> by
> > > reply email before deleting it from your system. Service of legal
> notice
> > > cannot be effected on Spicule Limited by email.
> > >
> > > —
> > > You are receiving this because you authored the thread.
> > > Reply to this email directly, view it on GitHub
> > > <#692 (comment)>,
or
> > mute
> > > the thread
> > > <
> >
>
https://github.com/notifications/unsubscribe-auth/AaCg9xrTFrE0x0_0BdT5OimYNpTuz8Zzks5vRoL3gaJpZM4bSCdO
> > >
> > > .
> > >
> >
> >
> > --
> > -----------------------------------------------------------------------
> > Antonio Martín Garre
> > ***@***.***
> >
> > "En los momentos de crisis, sólo la imaginación
> > es más importante que el conocimiento"
> > - A. Einstein
> > -----------------------------------------------------------------------
> > —
> > You are receiving this because you commented.
> > Reply to this email directly, view it on GitHub, or mute the thread.
> > --
> >
> >
> > Spicule Limited is registered in England & Wales. Company Number:
> > 09954122. Registered office: First Floor, Telecom House, 125-135
Preston
> > Road, Brighton, England, BN1 6AF. VAT No. 251478891.
> >
> >
> >
> >
> > All engagements
> > are subject to Spicule Terms and Conditions of Business. This email and
> > its
> > contents are intended solely for the individual to whom it is addressed
> > and
> > may contain information that is confidential, privileged or otherwise
> > protected from disclosure, distributing or copying. Any views or
opinions
> > presented in this email are solely those of the author and do not
> > necessarily represent those of Spicule Limited. The company accepts no
> > liability for any damage caused by any virus transmitted by this email.
> If
> > you have received this message in error, please notify us immediately
by
> > reply email before deleting it from your system. Service of legal
notice
> > cannot be effected on Spicule Limited by email.
> >
> > —
> > You are receiving this because you authored the thread.
> > Reply to this email directly, view it on GitHub
> > <
>
#692?email_source=notifications&email_token=AGQKB535EBLCPN7SVE3BB3TP246QVA5CNFSM4G2IE5HKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX2KKXQ#issuecomment-502572382
> >,
> > or mute the thread
> > <
>
https://github.com/notifications/unsubscribe-auth/AGQKB56NSGKEIEE23NWTNUTP246QVANCNFSM4G2IE5HA
> >
> > .
> >
>
>
> --
> -----------------------------------------------------------------------
> Antonio Martín Garre
> ***@***.***
>
> "En los momentos de crisis, sólo la imaginación
> es más importante que el conocimiento"
> - A. Einstein
> -----------------------------------------------------------------------
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <
#692?email_source=notifications&email_token=AAAZI6AJF5262AOZZBBUCPLP25EQHA5CNFSM4G2IE5HKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX2ONJY#issuecomment-502589095
>,
> or mute the thread
> <
https://github.com/notifications/unsubscribe-auth/AAAZI6BAFSKJCJBJW3JDX3LP25EQHANCNFSM4G2IE5HA
>
> .
>
--
Spicule Limited is registered in England & Wales. Company Number:
09954122. Registered office: First Floor, Telecom House, 125-135 Preston
Road, Brighton, England, BN1 6AF. VAT No. 251478891.
All engagements
are subject to Spicule Terms and Conditions of Business. This email and
its
contents are intended solely for the individual to whom it is addressed
and
may contain information that is confidential, privileged or otherwise
protected from disclosure, distributing or copying. Any views or opinions
presented in this email are solely those of the author and do not
necessarily represent those of Spicule Limited. The company accepts no
liability for any damage caused by any virus transmitted by this email. If
you have received this message in error, please notify us immediately by
reply email before deleting it from your system. Service of legal notice
cannot be effected on Spicule Limited by email.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#692?email_source=notifications&email_token=AGQKB54ZDHTHEHJ7RBTAWVLP25FARA5CNFSM4G2IE5HKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX2OYNQ#issuecomment-502590518>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AGQKB5YU4JNOASXQQO6FFPDP25FARANCNFSM4G2IE5HA>
.
--
-----------------------------------------------------------------------
Antonio Martín Garre
[email protected]
"En los momentos de crisis, sólo la imaginación
es más importante que el conocimiento"
- A. Einstein
-----------------------------------------------------------------------
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello!
Saiku uses mondrian 4 , but this version looks like it is no longer mantained. Is this correct?
If so, is there any way to use mondrian 3 (or 8)?
Regards,
The text was updated successfully, but these errors were encountered: