Bug: Btree cycle ID isn't recorded in WAL causing incomplete nbtree vacuum of concurrent backward splits. #9929

MMeent · 2024-11-28T16:25:53Z

Steps to reproduce

S1: Start btree vacuum scan, scan up to blockno BTP1
S2: Split btree page with blockno BTP2 into BTP2 and BTP3; such that the blockno of BTP2 > BTP1, and the blockno of BTP3 < BTP1.
(any session) flush/write BTP2 out of buffers and caches.
S1: continue btree vacuum scan to completion.

Expected result

BTP3 is re-scanned by btree vacuum scan.

Actual result

BTP3 is not re-scanned by btree vacuum scan, because the cycleid on page BTP2 was lost, causing bt vacuum to have no indication it should check the right sibling of the split page.

Environment

Any

This fixes the PostgreSQL side of neondatabase/neon#9929.

This fixes #9929.

This fixes the PostgreSQL side of neondatabase/neon#9929.

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes #9929.

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes the PostgreSQL side of neondatabase/neon#9929.

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes the PostgreSQL side of #9929. Emit nbtree vacuum cycle id in nbtree xlog through FPIs NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes #9929.

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes the PostgreSQL side of neondatabase/neon#9929.

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes the PostgreSQL side of #9929. Emit nbtree vacuum cycle id in nbtree xlog through FPIs NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes the PostgreSQL side of #9929. Emit nbtree vacuum cycle id in nbtree xlog through FPIs NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes #9929.

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes the PostgreSQL side of neondatabase/neon#9929.

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes the PostgreSQL side of #9929. Emit nbtree vacuum cycle id in nbtree xlog through FPIs NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes the PostgreSQL side of #9929. Emit nbtree vacuum cycle id in nbtree xlog through FPIs NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes the PostgreSQL side of #9929. Emit nbtree vacuum cycle id in nbtree xlog through FPIs NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. This fixes #9929.

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. During recovery, we extract the cycle ID from the original page if this page had an FPI, either directly (when the page was restored) or indirectly (from the record data). This fixes neondatabase/neon#9929

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. During recovery, we extract the cycle ID from the original page if this page had an FPI, either directly (when the page was restored) or indirectly (from the record data). This fixes #9929

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. During recovery, we extract the cycle ID from the original page if this page had an FPI, either directly (when the page was restored) or indirectly (from the record data). This fixes neondatabase/neon#9929

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. During recovery, we extract the cycle ID from the original page if this page had an FPI, either directly (when the page was restored) or indirectly (from the record data). This fixes #9929

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. During recovery, we extract the cycle ID from the original page if this page had an FPI, either directly (when the page was restored) or indirectly (from the record data). This fixes neondatabase/neon#9929

MMeent · 2024-12-10T14:56:39Z

Status: Waiting for PR #9932 approval

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. During recovery, we extract the cycle ID from the original page if this page had an FPI, either directly (when the page was restored) or indirectly (from the record data). This fixes neondatabase/neon#9929

NBTree needs a vacuum cycle ID on pages of whom the split resulted in a new right page that is located before the original page, or who were split from such split pages in the current vacuum cycle. By WAL-logging the cycle_id and restoring it in recovery, we assure vacuum doesn't fail to clean up the earlier pages. During recovery, we extract the cycle ID from the original page if this page had an FPI, either directly (when the page was restored) or indirectly (from the record data). This fixes #9929

This fixes #9929. ## Postgres repo PRS: - PG17: neondatabase/postgres#538 - PG16: neondatabase/postgres#539 - PG15: neondatabase/postgres#540 - PG14: neondatabase/postgres#541 ## Problem see #9929 ## Summary of changes We update the split code to force the code to emit an FPI whenever the cycle ID might be interesting for concurrent btree vacuum.

MMeent added c/compute Component: compute, excluding postgres itself c/PostgreSQL Component: PostgreSQL features and bugs t/bug Issue Type: Bug labels Nov 28, 2024

MMeent added a commit to neondatabase/postgres that referenced this issue Nov 28, 2024

Emit nbtree vacuum cycle id in nbtree xlog through FPIs

3ead091

This fixes the PostgreSQL side of neondatabase/neon#9929.

MMeent mentioned this issue Nov 28, 2024

Emit nbtree vacuum cycle id in nbtree xlog through FPIs neondatabase/postgres#538

Merged

MMeent added a commit that referenced this issue Nov 28, 2024

Emit nbtree vacuum cycle id in nbtree xlog through forced FPIs

a61d7cf

This fixes #9929.

MMeent added a commit to neondatabase/postgres that referenced this issue Nov 28, 2024

Emit nbtree vacuum cycle id in nbtree xlog through FPIs

2ad772b

This fixes the PostgreSQL side of neondatabase/neon#9929.

MMeent added a commit to neondatabase/postgres that referenced this issue Nov 28, 2024

Emit nbtree vacuum cycle id in nbtree xlog through FPIs

28f802c

This fixes the PostgreSQL side of neondatabase/neon#9929.

MMeent added a commit to neondatabase/postgres that referenced this issue Nov 28, 2024

Emit nbtree vacuum cycle id in nbtree xlog through FPIs

29e63d4

This fixes the PostgreSQL side of neondatabase/neon#9929.

MMeent self-assigned this Dec 3, 2024

MMeent closed this as completed in #9932 Dec 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: Btree cycle ID isn't recorded in WAL causing incomplete nbtree vacuum of concurrent backward splits. #9929

Bug: Btree cycle ID isn't recorded in WAL causing incomplete nbtree vacuum of concurrent backward splits. #9929

MMeent commented Nov 28, 2024

MMeent commented Dec 10, 2024

Bug: Btree cycle ID isn't recorded in WAL causing incomplete nbtree vacuum of concurrent backward splits. #9929

Bug: Btree cycle ID isn't recorded in WAL causing incomplete nbtree vacuum of concurrent backward splits. #9929

Comments

MMeent commented Nov 28, 2024

Steps to reproduce

Expected result

Actual result

Environment

MMeent commented Dec 10, 2024