Skip to content

Conversation

@yujun777
Copy link
Contributor

@yujun777 yujun777 commented Jan 23, 2026

Remove Rule EliminateNotNull because it may causes mistaken deletion.

Step as follow:

for expression E1

  1. rule InferNotNull rewrite E1, got E2 = E1 and a is not null
  2. some rule may simplify E2, then got E3 = a is not null
  3. rule EliminateNotNull remove the generated expression a is not null, and got E4=TRUE.

then it will have E1 => TRUE, this have bug. For rule InferNotNull, we know that E1 will evaluate to NULL/FALSE when a is null, so E1 cannot always evaluate to true.

This PR also remove field isGeneratedIsNotNull for NOT.

related PR: #16411

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 23, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@yujun777
Copy link
Contributor Author

run buildall

@yujun777
Copy link
Contributor Author

run buildall

@yujun777
Copy link
Contributor Author

run buildall

@yujun777
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31571 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3000b9ea4f5d273e1f9f41946e9e459f67d650db, data reload: false

------ Round 1 ----------------------------------
q1	17633	4795	4585	4585
q2	2071	360	209	209
q3	10166	1281	769	769
q4	10252	846	346	346
q5	7682	2226	2039	2039
q6	190	175	143	143
q7	930	732	607	607
q8	9263	1488	1216	1216
q9	6524	5185	5210	5185
q10	6748	1671	1312	1312
q11	520	317	296	296
q12	333	388	241	241
q13	17755	3840	3098	3098
q14	236	232	211	211
q15	597	540	537	537
q16	652	644	584	584
q17	658	749	551	551
q18	6743	6703	6450	6450
q19	1253	811	817	811
q20	397	380	235	235
q21	2608	2222	1852	1852
q22	374	325	294	294
Total cold run time: 103585 ms
Total hot run time: 31571 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4772	4726	4727	4726
q2	351	428	313	313
q3	2205	2673	2242	2242
q4	1363	1808	1363	1363
q5	4246	4094	4116	4094
q6	223	175	139	139
q7	2021	1846	1731	1731
q8	2626	2493	2407	2407
q9	7025	7022	7029	7022
q10	2339	2577	2169	2169
q11	574	480	459	459
q12	679	751	622	622
q13	3307	3805	3118	3118
q14	275	292	279	279
q15	556	511	513	511
q16	632	680	620	620
q17	1227	1346	1408	1346
q18	7597	7385	7515	7385
q19	902	862	846	846
q20	1943	1997	1826	1826
q21	4633	4368	4313	4313
q22	614	590	521	521
Total cold run time: 50110 ms
Total hot run time: 48052 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 176840 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3000b9ea4f5d273e1f9f41946e9e459f67d650db, data reload: false

query5	6076	710	524	524
query6	353	230	198	198
query7	4221	467	271	271
query8	356	284	254	254
query9	8730	2772	2770	2770
query10	516	337	290	290
query11	15333	15255	15058	15058
query12	206	124	121	121
query13	1333	445	333	333
query14	7779	3072	2833	2833
query14_1	2654	2690	2635	2635
query15	315	210	193	193
query16	1044	481	450	450
query17	1241	685	580	580
query18	2686	453	354	354
query19	205	197	167	167
query20	126	126	120	120
query21	219	142	124	124
query22	4737	4851	4766	4766
query23	16381	16084	15667	15667
query23_1	15826	15719	15771	15719
query24	7410	1750	1379	1379
query24_1	1347	1407	1383	1383
query25	558	523	410	410
query26	1185	290	178	178
query27	2599	493	316	316
query28	4527	2169	2142	2142
query29	815	576	476	476
query30	334	260	231	231
query31	839	714	602	602
query32	89	81	78	78
query33	539	365	324	324
query34	1009	891	528	528
query35	741	788	711	711
query36	947	924	853	853
query37	140	100	103	100
query38	2828	2848	2758	2758
query39	770	757	747	747
query39_1	723	710	708	708
query40	221	141	129	129
query41	72	67	67	67
query42	98	95	107	95
query43	439	455	425	425
query44	1325	753	758	753
query45	219	215	199	199
query46	965	1051	666	666
query47	1476	1440	1360	1360
query48	332	314	246	246
query49	633	462	378	378
query50	865	236	188	188
query51	3765	3773	3801	3773
query52	97	88	83	83
query53	238	211	192	192
query54	325	290	283	283
query55	82	79	78	78
query56	324	357	305	305
query57	1059	1011	928	928
query58	295	292	283	283
query59	2084	2045	1985	1985
query60	334	329	326	326
query61	154	156	150	150
query62	386	351	314	314
query63	220	180	194	180
query64	3377	1165	923	923
query65	3788	3761	3737	3737
query66	1662	444	338	338
query67	15644	15636	15438	15438
query68	2356	1082	692	692
query69	420	318	294	294
query70	949	935	936	935
query71	330	305	280	280
query72	5424	3277	3272	3272
query73	584	726	292	292
query74	8771	8845	8584	8584
query75	2298	2349	1912	1912
query76	2281	1251	904	904
query77	369	410	336	336
query78	9817	10001	9243	9243
query79	1139	918	613	613
query80	1583	526	464	464
query81	584	287	254	254
query82	1027	159	124	124
query83	312	256	234	234
query84	266	128	108	108
query85	1157	489	424	424
query86	497	322	307	307
query87	2966	2962	2878	2878
query88	3525	2615	2623	2615
query89	331	270	250	250
query90	1880	179	169	169
query91	178	164	133	133
query92	76	75	72	72
query93	1079	1036	641	641
query94	693	329	339	329
query95	641	429	338	338
query96	659	516	231	231
query97	2304	2373	2339	2339
query98	213	204	202	202
query99	588	606	492	492
Total cold run time: 252540 ms
Total hot run time: 176840 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 26.78 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3000b9ea4f5d273e1f9f41946e9e459f67d650db, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.08
query4	1.61	0.12	0.12
query5	0.27	0.25	0.24
query6	1.15	0.66	0.65
query7	0.03	0.03	0.02
query8	0.06	0.04	0.04
query9	0.56	0.52	0.50
query10	0.55	0.53	0.54
query11	0.15	0.10	0.10
query12	0.16	0.11	0.12
query13	0.60	0.59	0.60
query14	0.94	0.95	0.95
query15	0.81	0.78	0.78
query16	0.40	0.39	0.40
query17	1.03	1.05	1.10
query18	0.24	0.22	0.22
query19	1.98	1.83	1.86
query20	0.02	0.01	0.01
query21	16.43	0.25	0.14
query22	5.45	0.04	0.04
query23	16.91	0.51	0.17
query24	1.33	0.44	0.16
query25	0.12	0.15	0.06
query26	0.27	0.16	0.14
query27	0.07	0.06	0.05
query28	4.12	1.08	0.89
query29	12.56	4.02	3.17
query30	0.28	0.14	0.14
query31	2.92	0.64	0.38
query32	3.24	0.56	0.46
query33	3.05	2.98	3.14
query34	16.03	5.10	4.40
query35	4.49	4.46	4.45
query36	0.65	0.50	0.49
query37	0.11	0.07	0.06
query38	0.07	0.04	0.04
query39	0.05	0.03	0.03
query40	0.17	0.15	0.14
query41	0.10	0.03	0.04
query42	0.04	0.03	0.02
query43	0.05	0.03	0.03
Total cold run time: 99.49 s
Total hot run time: 26.78 s

@yujun777
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31470 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d083f22de91c27c01412fc921dd48252d8a563d3, data reload: false

------ Round 1 ----------------------------------
q1	17604	4780	4534	4534
q2	2065	319	209	209
q3	10203	1287	739	739
q4	10256	948	350	350
q5	7713	2207	1978	1978
q6	203	175	146	146
q7	900	724	615	615
q8	9293	1481	1152	1152
q9	6639	5243	5153	5153
q10	6794	1675	1301	1301
q11	520	309	301	301
q12	343	396	233	233
q13	17767	3804	3068	3068
q14	228	228	217	217
q15	606	550	521	521
q16	656	635	601	601
q17	663	802	456	456
q18	6822	6485	7430	6485
q19	1293	880	851	851
q20	432	404	238	238
q21	2788	2321	2030	2030
q22	386	330	292	292
Total cold run time: 104174 ms
Total hot run time: 31470 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5014	5012	4988	4988
q2	346	421	352	352
q3	2420	2858	2603	2603
q4	1476	1932	1434	1434
q5	4857	4537	4485	4485
q6	225	183	147	147
q7	2148	2037	1968	1968
q8	2683	2530	2514	2514
q9	7389	7642	7476	7476
q10	2593	2818	2328	2328
q11	570	479	462	462
q12	713	796	626	626
q13	3470	3736	3100	3100
q14	273	292	256	256
q15	536	508	499	499
q16	622	653	608	608
q17	1172	1267	1296	1267
q18	7451	7280	7313	7280
q19	856	829	825	825
q20	1918	1974	1841	1841
q21	4567	4316	4210	4210
q22	569	566	520	520
Total cold run time: 51868 ms
Total hot run time: 49789 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 175343 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d083f22de91c27c01412fc921dd48252d8a563d3, data reload: false

query5	4688	731	554	554
query6	348	234	214	214
query7	4231	462	282	282
query8	381	285	268	268
query9	8745	2793	2793	2793
query10	486	349	292	292
query11	15363	15126	15000	15000
query12	182	132	127	127
query13	1264	455	347	347
query14	6198	3088	2955	2955
query14_1	2710	2690	2706	2690
query15	265	206	195	195
query16	1026	484	466	466
query17	1104	718	608	608
query18	2486	471	347	347
query19	205	193	192	192
query20	119	115	121	115
query21	226	136	123	123
query22	4466	4662	4505	4505
query23	16409	15733	15532	15532
query23_1	15803	15608	15690	15608
query24	7768	1689	1345	1345
query24_1	1360	1392	1418	1392
query25	513	438	378	378
query26	1201	271	162	162
query27	2631	462	312	312
query28	4487	2164	2132	2132
query29	748	529	450	450
query30	324	255	216	216
query31	848	718	583	583
query32	82	77	74	74
query33	539	372	321	321
query34	1063	887	533	533
query35	730	780	700	700
query36	936	916	863	863
query37	136	93	86	86
query38	2879	2872	2808	2808
query39	795	748	714	714
query39_1	707	715	730	715
query40	236	138	122	122
query41	63	61	61	61
query42	99	97	98	97
query43	437	427	409	409
query44	1299	742	742	742
query45	213	233	198	198
query46	937	1043	680	680
query47	1458	1412	1345	1345
query48	316	311	236	236
query49	627	443	353	353
query50	857	258	180	180
query51	3783	3829	3744	3744
query52	89	85	86	85
query53	226	206	188	188
query54	310	292	266	266
query55	79	79	74	74
query56	317	305	315	305
query57	1067	1037	908	908
query58	298	282	279	279
query59	2096	1982	2041	1982
query60	334	329	316	316
query61	148	151	160	151
query62	363	357	305	305
query63	215	180	179	179
query64	3467	1142	848	848
query65	3852	3634	3632	3632
query66	1730	443	321	321
query67	15580	15535	15477	15477
query68	2217	1140	666	666
query69	416	313	311	311
query70	994	863	883	863
query71	319	312	290	290
query72	5291	3216	3301	3216
query73	586	724	305	305
query74	8827	8808	8522	8522
query75	2276	2364	1930	1930
query76	2274	1249	893	893
query77	380	399	320	320
query78	9573	9754	9093	9093
query79	1109	906	610	610
query80	663	553	496	496
query81	495	283	257	257
query82	1396	158	120	120
query83	277	246	237	237
query84	255	119	98	98
query85	811	480	418	418
query86	360	269	330	269
query87	2983	2930	2844	2844
query88	3490	2604	2581	2581
query89	344	275	246	246
query90	1762	173	176	173
query91	176	155	131	131
query92	71	76	73	73
query93	1067	998	644	644
query94	460	323	302	302
query95	628	350	323	323
query96	654	513	232	232
query97	2334	2349	2336	2336
query98	209	221	198	198
query99	589	542	446	446
Total cold run time: 247465 ms
Total hot run time: 175343 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.29 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d083f22de91c27c01412fc921dd48252d8a563d3, data reload: false

query1	0.06	0.05	0.05
query2	0.11	0.05	0.05
query3	0.26	0.08	0.08
query4	1.62	0.12	0.12
query5	0.27	0.25	0.25
query6	1.15	0.66	0.66
query7	0.04	0.02	0.02
query8	0.05	0.05	0.04
query9	0.56	0.51	0.51
query10	0.56	0.54	0.54
query11	0.15	0.10	0.10
query12	0.14	0.11	0.10
query13	0.61	0.59	0.59
query14	0.95	0.95	0.94
query15	0.79	0.77	0.78
query16	0.39	0.40	0.40
query17	1.03	1.07	1.08
query18	0.23	0.21	0.22
query19	1.96	1.83	1.86
query20	0.02	0.01	0.01
query21	15.44	0.27	0.14
query22	5.27	0.06	0.05
query23	16.58	0.35	0.14
query24	2.21	0.70	0.58
query25	0.20	0.16	0.19
query26	0.52	0.14	0.13
query27	0.08	0.08	0.05
query28	5.05	1.06	0.90
query29	12.53	3.92	3.17
query30	0.29	0.14	0.12
query31	2.81	0.62	0.39
query32	3.24	0.57	0.46
query33	2.98	2.96	3.02
query34	16.16	5.07	4.43
query35	4.44	4.41	4.50
query36	0.65	0.50	0.50
query37	0.12	0.07	0.06
query38	0.07	0.05	0.04
query39	0.05	0.03	0.03
query40	0.18	0.14	0.14
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 99.99 s
Total hot run time: 27.29 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 97.30% (36/37) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 91.89% (34/37) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants