Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

applied fix for PG16 sanitizers #342

Open
wants to merge 13 commits into
base: REL_16_STABLE_neon
Choose a base branch
from
Open

applied fix for PG16 sanitizers #342

wants to merge 13 commits into from

Conversation

abhigets
Copy link

@abhigets abhigets commented Jan 11, 2024 •
edited by bayandin
Loading

Enabling sanitizers as a part of enabling build sanitizers
We need to add the following patch to PG16.

fix-REL_16_STABLE_neon-for-sanitizers.patch`

Before the fix [before fix 2_regress-tests (debug, v16).txt]
(https://github.com/neondatabase/postgres/files/13923603/before.fix.2_regress-tests.debug.v16.txt)
18 failed, 337 passed, 16 skipped, 2 xfailed, 9 errors, 1 rerun in 645.01s (0:10:45) =

After the fix [after the fix 2_regress-tests (debug, v16).txt]
(https://github.com/neondatabase/postgres/files/13923559/after.the.fix.2_regress-tests.debug.v16.txt)
1 failed, 355 passed, 16 skipped, 1 xfailed, 1 xpassed, 2 errors in 880.45s (0:14:40) =

One of the stack trace

DROP_REPLICATION_SLOT pg_16396_sync_16384_7322594095505538972 WAIT
/__w/neon/neon//vendor/postgres-v16/src/backend/access/transam/xlog.c:1284:19: runtime error: null pointer passed as argument 2, which is declared to never be null
/usr/include/string.h:44:28: note: nonnull attribute specified here
    #0 0x55be3a33dab1 in CopyXLogRecordToWAL /__w/neon/neon/vendor/postgres-v16/src/backend/access/transam/xlog.c:1284:3
    #1 0x55be3a33a869 in XLogInsertRecord /__w/neon/neon/vendor/postgres-v16/src/backend/access/transam/xlog.c:916:3
    #2 0x55be3a391369 in XLogInsert /__w/neon/neon/vendor/postgres-v16/src/backend/access/transam/xloginsert.c:522:12
    #3 0x55be3be6a497 in LogLogicalMessage /__w/neon/neon/vendor/postgres-v16/src/backend/replication/logical/message.c:74:9
    #4 0x55be3bf4df8e in ReplicationSlotDropPtr /__w/neon/neon/vendor/postgres-v16/src/backend/replication/slot.c:695:13
    #5 0x55be3bf4c9f1 in ReplicationSlotDropAcquired /__w/neon/neon/vendor/postgres-v16/src/backend/replication/slot.c:665:2
    #6 0x55be3bf4ebcc in ReplicationSlotDrop /__w/neon/neon/vendor/postgres-v16/src/backend/replication/slot.c:649:2
    #7 0x55be3bf96a2d in DropReplicationSlot /__w/neon/neon/vendor/postgres-v16/src/backend/replication/walsender.c:1251:2
    #8 0x55be3bf9265f in exec_replication_command /__w/neon/neon/vendor/postgres-v16/src/backend/replication/walsender.c:1823:4
    #9 0x55be3c281283 in PostgresMain /__w/neon/neon/vendor/postgres-v16/src/backend/tcop/postgres.c:4643:12
    #10 0x55be3bd16940 in BackendRun /__w/neon/neon/vendor/postgres-v16/src/backend/postmaster/postmaster.c:4464:2
    #11 0x55be3bd0c8cf in BackendStartup /__w/neon/neon/vendor/postgres-v16/src/backend/postmaster/postmaster.c:4192:3
    #12 0x55be3bd062d9 in ServerLoop /__w/neon/neon/vendor/postgres-v16/src/backend/postmaster/postmaster.c:1782:6
    #13 0x55be3bd02f6d in PostmasterMain /__w/neon/neon/vendor/postgres-v16/src/backend/postmaster/postmaster.c:1466:11
    #14 0x55be3b364f1d in main /__w/neon/neon/vendor/postgres-v16/src/backend/main/main.c:238:3
    #15 0x7ff099d6ad09 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x23d09) (BuildId: a3780b0b8a5bf5876e31d16b0a9d8fc6ba69a1f2)
    #16 0x55be39c3a789 in _start (/tmp/neon/pg_install/v16/bin/postgres+0x21e0789) (BuildId: f02f82f188582f4d32765cb22cb010fc26a2aaab)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /__w/neon/neon//vendor/postgres-v16/src/backend/access/transam/xlog.c:1284:19

MMeent and others added 11 commits November 8, 2023 16:07
@MMeent @tristan957
Patch PostgreSQL v16 with squashed patchset
a297d13 
This prepares PostgreSQL for compatibility with Neon's storage.
Significant changes compared to the PostgreSQL 15 patchset include:

- Backported changes for users and roles are no longer required
- Use RM_NEON_ID for changes in WAL, instead of modifying core WAL records
@hlinnaka @tristan957
Fix WAL-logging XLOG_NEON_HEAP_LOCK records.
ed56101
@knizhnik @tristan957
Upadte last written LSN after walloging all createdb stuff (#308)
bc5f001 
Co-authored-by: Konstantin Knizhnik <[email protected]>
@knizhnik @arssher @tristan957
Neon logical replication support for PG16 (#310)
17bc650 
* Neon logical replication support for PG16

* Log heap rewrite file after creation.

---------

Co-authored-by: Konstantin Knizhnik <[email protected]>
Co-authored-by: Arseny Sher <[email protected]>
@knizhnik @tristan957
Fix elog format error in wallog_mapping_file (#317)
7f8a2cb 
Co-authored-by: Konstantin Knizhnik <[email protected]>
@knizhnik @tristan957
Update WAL buffers when restoring WAL at compute needed for LR (#323)
812c5a2 
* Update WAL buffers when restoring WAL at compute needed for LR

* Fix copying data in WAL buffers

---------

Co-authored-by: Konstantin Knizhnik <[email protected]>
@knizhnik @tristan957
Optimize stroing zero FPI in WAL (#327)
863b715 
PG16 adds new function to SMGR: zeroextend
It's implementation in Neon actually wal-log zero pages of extended relation.
This zero page is wal-logged using XLOG_FPI.
As far as page is zero, the hole optimization (excluding from the image everything between pg_upper and pd_lower) doesn't work.

This PR allows to set hole size to BLCKSZ in case of zero page (PageIsNull() returns true).
---------

Co-authored-by: Konstantin Knizhnik <[email protected]>
Prevent output callbacks from hearing about neon-file messages (#328)
1127d0a 
* Prevent output callbacks from hearing about neon-file messages
@save-buffer
Use strncmp instead of strcmp
de8242c
@arssher
Flush slot creation/drop.
225071f
@arssher
Remove excessive walsender reply logging.
73eda3d
@abhigets abhigets requested a review from bayandin January 11, 2024 11:03
@abhigets abhigets self-assigned this Jan 11, 2024
@bayandin bayandin removed their request for review January 11, 2024 14:24
@abhigets abhigets requested a review from bayandin January 11, 2024 14:56
@bayandin bayandin removed their request for review January 11, 2024 14:57
@tristan957
Copy link
Member

Commenting here is case others missed my slack message, but this is upstream Postgres code. I would like to understand the issue at hand better.

@abhigets abhigets force-pushed the add-build-sanitizers branch from 2088022 to 6f1ef95 Compare January 12, 2024 14:56
@abhigets abhigets requested a review from tristan957 January 18, 2024 14:53
@tristan957 tristan957 force-pushed the REL_16_STABLE_neon branch from 8ef3c33 to c271017 Compare May 19, 2024 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tristan957 tristan957 Awaiting requested review from tristan957

At least 1 approving review is required to merge this pull request.

Assignees

@abhigets abhigets

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@abhigets @tristan957 @MMeent @hlinnaka @knizhnik @save-buffer @arssher