Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](bf index) add ngram bf index validation in nereids index definition check (#45780) #46215

Merged

Conversation

airborne12
Copy link
Member

pick #45780

…tion check (apache#45780)

Related PR: apache#44973

Problem Summary:

Need to check ngram bf index properties in index definition for nereids.
@Thearas
Copy link
Contributor

Thearas commented Dec 31, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@airborne12
Copy link
Member Author

run buildall

@airborne12
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40869 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0b6a32013312b29453e2fe29781eb8f56f11b4d1, data reload: false

------ Round 1 ----------------------------------
q1	17574	7421	7300	7300
q2	2048	204	171	171
q3	10637	1087	1198	1087
q4	10560	800	785	785
q5	7760	2848	2764	2764
q6	235	151	145	145
q7	975	609	612	609
q8	9352	1923	1983	1923
q9	6626	6444	6394	6394
q10	7088	2297	2354	2297
q11	460	271	268	268
q12	408	218	222	218
q13	17817	2979	3026	2979
q14	245	212	209	209
q15	570	534	528	528
q16	704	611	623	611
q17	969	571	516	516
q18	7261	6652	6763	6652
q19	1381	1021	1054	1021
q20	477	205	206	205
q21	4003	3197	3239	3197
q22	1120	990	992	990
Total cold run time: 108270 ms
Total hot run time: 40869 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7281	7236	7218	7218
q2	324	236	239	236
q3	2927	2913	2967	2913
q4	2004	1880	1838	1838
q5	5673	5704	5749	5704
q6	229	149	146	146
q7	2259	1858	1854	1854
q8	3371	3562	3430	3430
q9	8911	8931	8939	8931
q10	3621	3578	3527	3527
q11	600	514	513	513
q12	831	624	606	606
q13	9369	3179	3168	3168
q14	302	289	265	265
q15	571	532	532	532
q16	698	695	651	651
q17	1856	1601	1645	1601
q18	8308	7779	7702	7702
q19	1690	1593	1454	1454
q20	2116	1899	1851	1851
q21	5567	5478	5296	5296
q22	1177	1054	1087	1054
Total cold run time: 69685 ms
Total hot run time: 60490 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197794 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0b6a32013312b29453e2fe29781eb8f56f11b4d1, data reload: false

query1	1284	919	915	915
query2	6227	2070	2016	2016
query3	10919	4405	4323	4323
query4	67082	29187	23438	23438
query5	4889	450	448	448
query6	423	189	182	182
query7	5542	308	314	308
query8	316	234	237	234
query9	8748	2734	2715	2715
query10	446	268	257	257
query11	17214	15116	15924	15116
query12	158	114	107	107
query13	1457	483	453	453
query14	9704	7946	7553	7553
query15	211	186	187	186
query16	7047	492	539	492
query17	1110	577	594	577
query18	1824	325	335	325
query19	202	170	175	170
query20	123	110	107	107
query21	64	47	44	44
query22	4781	4542	4462	4462
query23	34908	34188	34408	34188
query24	6376	2930	2948	2930
query25	517	423	450	423
query26	659	167	169	167
query27	1730	310	312	310
query28	4274	2566	2493	2493
query29	703	445	430	430
query30	244	163	175	163
query31	1029	863	805	805
query32	69	57	55	55
query33	404	288	288	288
query34	917	509	518	509
query35	879	741	744	741
query36	1071	965	961	961
query37	123	70	72	70
query38	4083	4016	4029	4016
query39	1500	1469	1459	1459
query40	142	81	80	80
query41	50	46	47	46
query42	109	99	98	98
query43	523	491	508	491
query44	1191	844	825	825
query45	185	167	163	163
query46	1138	744	737	737
query47	2035	1930	1925	1925
query48	467	390	385	385
query49	719	375	388	375
query50	847	432	425	425
query51	7291	7409	7305	7305
query52	97	83	87	83
query53	253	186	182	182
query54	575	444	449	444
query55	78	71	71	71
query56	274	244	243	243
query57	1236	1093	1097	1093
query58	205	200	205	200
query59	3352	2948	2933	2933
query60	278	253	251	251
query61	110	111	111	111
query62	784	685	672	672
query63	217	188	189	188
query64	1413	698	627	627
query65	3249	3198	3255	3198
query66	710	303	297	297
query67	15896	15569	15535	15535
query68	4014	571	570	570
query69	426	262	254	254
query70	1189	1141	1122	1122
query71	359	246	259	246
query72	6452	4126	3857	3857
query73	747	352	351	351
query74	10275	8952	8988	8952
query75	3322	2617	2686	2617
query76	1908	1090	955	955
query77	466	282	274	274
query78	10679	9669	9625	9625
query79	1406	595	602	595
query80	848	443	429	429
query81	505	242	235	235
query82	1287	121	112	112
query83	242	146	144	144
query84	308	78	76	76
query85	878	293	292	292
query86	333	306	296	296
query87	4388	4316	4301	4301
query88	3635	2472	2385	2385
query89	410	301	284	284
query90	2000	188	189	188
query91	183	146	150	146
query92	63	48	49	48
query93	1505	550	556	550
query94	840	302	287	287
query95	354	252	254	252
query96	615	276	294	276
query97	3330	3187	3170	3170
query98	206	196	198	196
query99	1597	1338	1290	1290
Total cold run time: 317451 ms
Total hot run time: 197794 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.92 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0b6a32013312b29453e2fe29781eb8f56f11b4d1, data reload: false

query1	0.03	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.62	0.10	0.10
query5	0.50	0.52	0.52
query6	1.14	0.73	0.71
query7	0.02	0.01	0.02
query8	0.04	0.04	0.03
query9	0.57	0.49	0.50
query10	0.54	0.55	0.55
query11	0.14	0.11	0.11
query12	0.14	0.12	0.11
query13	0.63	0.59	0.60
query14	2.89	3.01	3.01
query15	0.89	0.82	0.82
query16	0.39	0.40	0.38
query17	1.04	1.06	1.06
query18	0.24	0.21	0.22
query19	1.94	1.97	1.97
query20	0.02	0.01	0.01
query21	15.72	0.58	0.58
query22	2.43	2.92	1.82
query23	16.75	1.04	0.80
query24	3.03	1.73	0.71
query25	0.17	0.22	0.09
query26	0.43	0.13	0.13
query27	0.04	0.04	0.03
query28	10.28	1.10	1.08
query29	12.52	3.18	3.20
query30	0.24	0.06	0.06
query31	2.84	0.38	0.37
query32	3.27	0.45	0.47
query33	3.02	2.99	3.05
query34	17.09	4.53	4.52
query35	4.53	4.53	4.54
query36	0.66	0.50	0.48
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.59 s
Total hot run time: 32.92 s

@airborne12 airborne12 merged commit 5ced8ba into apache:branch-3.0 Jan 1, 2025
20 checks passed
@airborne12 airborne12 deleted the pick_45780_to_origin_branch-3.0 branch January 1, 2025 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants