Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use TRUNCATE + COPY FREEZE when copying a shard #7403

Open
JelteF opened this issue Jan 5, 2024 · 1 comment · May be fixed by #7687
Open

Use TRUNCATE + COPY FREEZE when copying a shard #7403

JelteF opened this issue Jan 5, 2024 · 1 comment · May be fixed by #7687

Comments

@JelteF
Copy link
Contributor

JelteF commented Jan 5, 2024

In src/backend/distributed/operations/worker_shard_copy.c file contains our main COPY logic for for shard moves and shard splits. After talking with @DimCitus I realized that we could use the FREEZE option of COPY to reduce the the need for heavy vacuuming after the copy is done. To benefit from the freeze option it's required to truncate the target table in the same transaction as the copy. This is fine for all our (current) use cases, because the target shard has just been created and is thus empty.

What I think is needed to achieve this:

  1. Add the FREEZE option to the COPY command that we generate in ConstructShardCopyStatement
  2. Add the FREEZE option to the list of options created in LocalCopyToShard
  3. Start a transaction in ConnectToRemoteAndStartCopy
  4. Truncate the table in ConnectToRemoteAndStartCopy (before starting the COPY)
  5. End the transaction in ShardCopyDestReceiverShutdown

Apart from the actual implementation this needs tests to see that indeed vacuum is not necessary on the new table after a shard move.

@marcoslot
Copy link
Collaborator

I wonder whether statistics gathering would be affected. It might anyway be good to do an ANALYZE as part of a move to get statistics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants