Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](nereids) Create Runtime filter even if equal.right is not in aliasTransferMap-branch3.0 #46224

Open
wants to merge 1 commit into
base: branch-3.0
Choose a base branch
from

Conversation

englefly
Copy link
Contributor

@englefly englefly commented Dec 31, 2024

What problem does this PR solve?

Issue Number: close #xxx
pick #46406
Related PR: #40815

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 31, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41212 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fcfe2539b3e6d5f3ce89e6b9d2ed62328b7913ff, data reload: false

------ Round 1 ----------------------------------
q1	17573	7382	7338	7338
q2	2079	172	178	172
q3	10556	1095	1226	1095
q4	10524	752	775	752
q5	7738	2916	2834	2834
q6	243	149	147	147
q7	996	623	612	612
q8	9359	1974	2039	1974
q9	6680	6458	6418	6418
q10	7004	2291	2460	2291
q11	469	263	263	263
q12	425	217	205	205
q13	17780	2988	3042	2988
q14	247	211	217	211
q15	564	537	512	512
q16	673	630	617	617
q17	974	579	584	579
q18	7241	6724	6736	6724
q19	1389	1057	1134	1057
q20	466	210	196	196
q21	4072	3238	3253	3238
q22	1127	1006	989	989
Total cold run time: 108179 ms
Total hot run time: 41212 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7290	7242	7223	7223
q2	323	236	227	227
q3	2937	2932	3003	2932
q4	2057	1840	1840	1840
q5	5798	5766	5811	5766
q6	233	147	146	146
q7	2254	1868	1797	1797
q8	3386	3638	3490	3490
q9	8991	8955	8936	8936
q10	3587	3579	3536	3536
q11	598	516	511	511
q12	790	611	614	611
q13	9520	3220	3176	3176
q14	305	293	273	273
q15	568	529	525	525
q16	737	681	703	681
q17	1882	1621	1605	1605
q18	8323	7721	7622	7622
q19	1688	1609	1642	1609
q20	2119	1878	1855	1855
q21	5587	5387	5604	5387
q22	1119	1054	1030	1030
Total cold run time: 70092 ms
Total hot run time: 60778 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197507 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fcfe2539b3e6d5f3ce89e6b9d2ed62328b7913ff, data reload: false

query1	1316	917	912	912
query2	6230	2119	2080	2080
query3	10813	4253	4357	4253
query4	65796	29289	23366	23366
query5	5059	474	466	466
query6	410	170	165	165
query7	5532	308	305	305
query8	312	235	233	233
query9	8767	2736	2725	2725
query10	458	268	254	254
query11	17309	15191	15844	15191
query12	163	105	95	95
query13	1476	442	422	422
query14	10354	7275	6766	6766
query15	225	181	193	181
query16	7186	418	472	418
query17	1073	606	588	588
query18	1941	320	323	320
query19	213	163	158	158
query20	122	113	113	113
query21	64	48	44	44
query22	4671	4627	4610	4610
query23	35832	34388	34566	34388
query24	6093	2963	2953	2953
query25	540	435	417	417
query26	669	171	165	165
query27	2029	305	305	305
query28	4377	2585	2504	2504
query29	718	476	449	449
query30	250	172	161	161
query31	1009	847	859	847
query32	68	56	60	56
query33	460	288	280	280
query34	922	522	502	502
query35	850	744	729	729
query36	1072	982	972	972
query37	120	69	78	69
query38	4171	4029	4108	4029
query39	1501	1478	1452	1452
query40	139	82	81	81
query41	50	46	48	46
query42	119	100	102	100
query43	563	499	501	499
query44	1223	823	828	823
query45	185	175	169	169
query46	1154	778	741	741
query47	2023	1901	1921	1901
query48	464	372	394	372
query49	719	386	388	386
query50	851	427	419	419
query51	7452	7226	7263	7226
query52	98	83	88	83
query53	251	184	176	176
query54	550	455	443	443
query55	75	78	75	75
query56	255	260	249	249
query57	1228	1116	1102	1102
query58	209	198	199	198
query59	3316	2969	3058	2969
query60	287	275	263	263
query61	112	108	108	108
query62	777	691	696	691
query63	215	182	185	182
query64	1412	674	651	651
query65	3271	3222	3222	3222
query66	698	301	309	301
query67	15992	15509	15474	15474
query68	4086	574	587	574
query69	437	264	274	264
query70	1210	1101	1146	1101
query71	341	255	258	255
query72	6187	4048	4094	4048
query73	759	348	350	348
query74	10315	9023	8954	8954
query75	3344	2622	2658	2622
query76	1907	1006	1033	1006
query77	505	274	276	274
query78	10701	9664	9606	9606
query79	1473	594	592	592
query80	856	444	420	420
query81	513	244	242	242
query82	1255	116	113	113
query83	220	140	141	140
query84	294	89	80	80
query85	878	331	296	296
query86	354	296	290	290
query87	4412	4380	4301	4301
query88	3795	2467	2392	2392
query89	421	292	288	288
query90	1961	189	185	185
query91	183	145	148	145
query92	64	51	50	50
query93	1900	549	553	549
query94	769	282	288	282
query95	353	258	260	258
query96	624	285	284	284
query97	3312	3184	3169	3169
query98	211	202	203	202
query99	1591	1284	1297	1284
Total cold run time: 319090 ms
Total hot run time: 197507 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.12 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fcfe2539b3e6d5f3ce89e6b9d2ed62328b7913ff, data reload: false

query1	0.03	0.02	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.62	0.10	0.10
query5	0.53	0.49	0.50
query6	1.13	0.72	0.73
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.51	0.48
query10	0.55	0.54	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.59	0.59
query14	3.09	3.12	2.96
query15	0.89	0.86	0.82
query16	0.38	0.38	0.38
query17	1.00	0.99	0.98
query18	0.24	0.22	0.21
query19	1.86	1.83	1.98
query20	0.01	0.01	0.02
query21	15.36	0.59	0.58
query22	2.73	2.16	2.05
query23	17.00	0.99	0.86
query24	3.42	1.28	0.94
query25	0.21	0.17	0.08
query26	0.47	0.14	0.14
query27	0.05	0.04	0.05
query28	10.39	1.11	1.07
query29	12.58	3.22	3.22
query30	0.25	0.06	0.05
query31	2.85	0.38	0.38
query32	3.26	0.46	0.47
query33	2.99	3.01	3.06
query34	17.19	4.49	4.54
query35	4.53	4.48	4.59
query36	0.67	0.48	0.47
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.61 s
Total hot run time: 33.12 s

('ad', 'bc', 'cc', '2024-07-03 01:15:30', 4);
"""

qt_1 """
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reference out is missing

@morrySnow morrySnow added dev/2.1.x dev/3.0.x usercase Important user case type label p0_b labels Jan 2, 2025
@morrySnow
Copy link
Contributor

please add desc

@englefly englefly closed this Jan 3, 2025
@englefly englefly reopened this Jan 3, 2025
@englefly englefly changed the title [fix](nereids) Create Runtime filter even if equal.right is not in aliasTransferMap [fix](nereids) Create Runtime filter even if equal.right is not in aliasTransferMap-branch3.0 Jan 3, 2025
@englefly
Copy link
Contributor Author

englefly commented Jan 3, 2025

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev/2.1.x dev/3.0.x p0_b usercase Important user case type label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants