You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In src/backend/distributed/operations/worker_shard_copy.c file contains our main COPY logic for for shard moves and shard splits. After talking with @DimCitus I realized that we could use the FREEZE option of COPY to reduce the the need for heavy vacuuming after the copy is done. To benefit from the freeze option it's required to truncate the target table in the same transaction as the copy. This is fine for all our (current) use cases, because the target shard has just been created and is thus empty.
What I think is needed to achieve this:
Add the FREEZE option to the COPY command that we generate in ConstructShardCopyStatement
Add the FREEZE option to the list of options created in LocalCopyToShard
Start a transaction in ConnectToRemoteAndStartCopy
Truncate the table in ConnectToRemoteAndStartCopy (before starting the COPY)
End the transaction in ShardCopyDestReceiverShutdown
Apart from the actual implementation this needs tests to see that indeed vacuum is not necessary on the new table after a shard move.
The text was updated successfully, but these errors were encountered:
In
src/backend/distributed/operations/worker_shard_copy.c
file contains our main COPY logic for for shard moves and shard splits. After talking with @DimCitus I realized that we could use theFREEZE
option ofCOPY
to reduce the the need for heavy vacuuming after the copy is done. To benefit from the freeze option it's required to truncate the target table in the same transaction as the copy. This is fine for all our (current) use cases, because the target shard has just been created and is thus empty.What I think is needed to achieve this:
FREEZE
option to the COPY command that we generate inConstructShardCopyStatement
FREEZE
option to the list of options created inLocalCopyToShard
ConnectToRemoteAndStartCopy
ConnectToRemoteAndStartCopy
(before starting the COPY)ShardCopyDestReceiverShutdown
Apart from the actual implementation this needs tests to see that indeed vacuum is not necessary on the new table after a shard move.
The text was updated successfully, but these errors were encountered: