Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PG17 compatibility: Fix Test Failure in local_dist_join_mixed #7731

Draft
wants to merge 13 commits into
base: naisila/pg17_support
Choose a base branch
from

Conversation

m3hm3t
Copy link
Contributor

@m3hm3t m3hm3t commented Nov 11, 2024

PostgreSQL 16 adds an extra condition (id IS NOT NULL) to the subquery. This condition is likely used to ensure that no null values are processed in the subquery. Instead of using the condition id IS NOT NULL, PostgreSQL 17 generates the subplan with a trivial condition (WHERE true), indicating that it does not need to explicitly check for non-null values.

PostgreSQL 17 likely includes optimizations to handle null checks more efficiently. The WHERE (id IS NOT NULL) condition that was present in PostgreSQL 16 may now be considered redundant by the planner, as it is implicitly handled by the query execution engine.

postgres/postgres@b262ad44

 SELECT
        foo1.id
    FROM
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo9,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo8,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo7,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo6,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo5,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo4,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo3,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo2,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo10,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo1
 WHERE
  foo1.id =  foo9.id AND
  foo1.id =  foo8.id AND
  foo1.id =  foo7.id AND
  foo1.id =  foo6.id AND
  foo1.id =  foo5.id AND
  foo1.id =  foo4.id AND
  foo1.id =  foo3.id AND
  foo1.id =  foo2.id AND
  foo1.id =  foo10.id AND
  foo1.id =  foo1.id
ORDER BY 1;
...
-DEBUG:  generating subplan XXX_10 for subquery SELECT id FROM local_dist_join_mixed.local WHERE (id IS NOT NULL)
+DEBUG:  generating subplan XXX_10 for subquery SELECT id FROM local_dist_join_mixed.local WHERE true
...

naisila and others added 11 commits November 11, 2024 11:57
(cherry picked from commit ae3ed7d)
(cherry picked from commit 76f60a7)
(cherry picked from commit df9c7b4)
In PG17, the outer loop in acquire_sample_rows() changed
from
while (BlockSampler_HasMore(&bs))
to
while (table_scan_analyze_next_block(scan, stream))

Relevant PG commit:
041b96802efa33d2bc9456f2ad946976b92b5ae1
postgres/postgres@041b968

It is expected that the scan_analyze_next_block function will
check if there are any blocks left. So we add that check in
columnar_scan_analyze_next_block

(cherry picked from commit 7eb0ad5)
PG 17 added support for DEFAULT in ALTER TABLE .. SET ACCESS METHOD

Relevant PG commit:
d61a6cad6418f643a5773352038d0dfe5d3535b8
postgres/postgres@d61a6ca

In that case, name in AlterTableCmd would be null.
Add a null check here to avoid crash.

(cherry picked from commit 71b9974)
…tests

Fix pg15 pg16 multi_mx_create_table multi_schema_support

Relevant PG commit:
postgres/postgres@f696c0c
f696c0cd5f299f1b51e214efc55a22a782cc175d

(cherry picked from commit 17a2ed0)
Relevant PG commit:
f69319f2f1fb16eda4b535bcccec90dff3a6795e
postgres/postgres@f69319f

(cherry picked from commit 6c12b10)
This PR addresses a regression test failure in the multi-mx feature of
Citus with the new PostgreSQL 17 version. The regression was identified
during the execution of multi-node tests, specifically targeting
compatibility issues introduced with PostgreSQL 17.

---------

Co-authored-by: Mehmet YILMAZ <[email protected]>
(cherry picked from commit 70cf729)
This reverts commit e4040dd.

Reverting for now as this commit is fixing more than one thing
at once at multi_extension.out file

Its a harmless revert for testing purposes
@m3hm3t m3hm3t self-assigned this Nov 11, 2024
@m3hm3t m3hm3t changed the base branch from main to naisila/pg17_support November 11, 2024 19:03
@m3hm3t m3hm3t changed the title M3hm3t/local dist join mixed Fix Test Failure in local_dist_join_mixed in PG17 Nov 11, 2024
Copy link

codecov bot commented Nov 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.60%. Comparing base (b29c332) to head (0a8e7ab).

Additional details and impacted files
@@                   Coverage Diff                    @@
##           naisila/pg17_support    #7731      +/-   ##
========================================================
- Coverage                 89.61%   89.60%   -0.01%     
========================================================
  Files                       274      274              
  Lines                     59689    59689              
  Branches                   7446     7446              
========================================================
- Hits                      53490    53486       -4     
- Misses                     4069     4072       +3     
- Partials                   2130     2131       +1     

@m3hm3t m3hm3t marked this pull request as ready for review November 11, 2024 19:23
@@ -1266,7 +1271,7 @@ DEBUG: generating subplan XXX_8 for subquery SELECT id FROM local_dist_join_mix
DEBUG: Wrapping relation "local" to a subquery
DEBUG: generating subplan XXX_9 for subquery SELECT id FROM local_dist_join_mixed.local WHERE true
DEBUG: Wrapping relation "local" to a subquery
DEBUG: generating subplan XXX_10 for subquery SELECT id FROM local_dist_join_mixed.local WHERE (id IS NOT NULL)
DEBUG: generating subplan XXX_10 for subquery SELECT id FROM local_dist_join_mixed.local WHERE true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am reluctant to merge a whole new test output for just a single line change. Let's think a bit more how we can fix this.

@naisila naisila changed the title Fix Test Failure in local_dist_join_mixed in PG17 PG17 compatibility: Fix Test Failure in local_dist_join_mixed Nov 12, 2024
@m3hm3t m3hm3t marked this pull request as draft November 14, 2024 10:01
@naisila naisila force-pushed the naisila/pg17_support branch 4 times, most recently from 46dc966 to 6d036b0 Compare November 20, 2024 11:54
@naisila
Copy link
Member

naisila commented Nov 20, 2024

Here we can simply remove the redundant filter because it wasn't the original intention of this test.

 SELECT
        foo1.id
    FROM
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo9,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo8,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo7,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo6,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo5,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo4,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo3,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo2,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo10,
 (SELECT local.id, local.title FROM local, distributed WHERE local.id = distributed.id ) as foo1
 WHERE
  foo1.id =  foo9.id AND
  foo1.id =  foo8.id AND
  foo1.id =  foo7.id AND
  foo1.id =  foo6.id AND
  foo1.id =  foo5.id AND
  foo1.id =  foo4.id AND
  foo1.id =  foo3.id AND
  foo1.id =  foo2.id AND
  foo1.id =  foo10.id AND
-  foo1.id =  foo1.id
ORDER BY 1;

@naisila naisila force-pushed the naisila/pg17_support branch 6 times, most recently from e108bb8 to c396ce6 Compare November 22, 2024 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants