From c07b284e039ea7fa776505f0fceab7c8c73ddcdd Mon Sep 17 00:00:00 2001 From: Yasuo Honda Date: Wed, 10 Apr 2024 16:46:45 +0900 Subject: [PATCH 1/4] TiDB 7.1+ checkpoint restore requires `s3:DeleteObject` permission Fix #16961 --- br/backup-and-restore-storages.md | 4 ++-- br/backup-and-restore-use-cases.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/br/backup-and-restore-storages.md b/br/backup-and-restore-storages.md index 4c808bec5bbb7..06d160bd50d99 100644 --- a/br/backup-and-restore-storages.md +++ b/br/backup-and-restore-storages.md @@ -106,8 +106,8 @@ When storing backup data in a cloud storage system, you need to configure authen Before backup, configure the following privileges to access the backup directory on S3. -- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:PutObject`, and `s3:AbortMultipartUpload` -- Minimum privileges for TiKV and BR to access the backup directories during restore: `s3:ListBucket`, `s3:GetObject`, and `s3:PutObject`. BR writes checkpoint information to the `./checkpoints` subdirectory under the backup directory. When restoring log backup data, BR writes the table ID mapping relationship of the restored cluster to the `./pitr_id_maps` subdirectory under the backup directory. +- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:PutObject`, `s3:DeleteObject`, and `s3:AbortMultipartUpload` +- Minimum privileges for TiKV and BR to access the backup directories during restore: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, and `s3:PutObject`. BR writes checkpoint information to the `./checkpoints` subdirectory under the backup directory. When restoring log backup data, BR writes the table ID mapping relationship of the restored cluster to the `./pitr_id_maps` subdirectory under the backup directory. If you have not yet created a backup directory, refer to [Create a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) to create an S3 bucket in the specified region. If necessary, you can also create a folder in the bucket by referring to [Create a folder](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html). diff --git a/br/backup-and-restore-use-cases.md b/br/backup-and-restore-use-cases.md index 9fe9e583a09e2..f5ee844f2b6f1 100644 --- a/br/backup-and-restore-use-cases.md +++ b/br/backup-and-restore-use-cases.md @@ -69,8 +69,8 @@ The detailed steps are as follows: 2. Configure permissions for BR and TiKV to access the S3 directory. It is recommended to grant permissions using the IAM method, which is the most secure way to access the S3 bucket. For detailed steps, refer to [AWS documentation: Controlling access to a bucket with user policies](https://docs.aws.amazon.com/AmazonS3/latest/userguide/walkthrough1.html). The required permissions are as follows: - - TiKV and BR in the backup cluster need `s3:ListBucket`, `s3:PutObject`, and `s3:AbortMultipartUpload` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. - - TiKV and BR in the restore cluster need `s3:ListBucket`, `s3:GetObject`, and `s3:PutObject` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. + - TiKV and BR in the backup cluster need `s3:ListBucket`, `s3:PutObject`, `s3:DeleteObject`, and `s3:AbortMultipartUpload` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. + - TiKV and BR in the restore cluster need `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, and `s3:PutObject` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. 3. Plan the directory structure that stores the backup data, including the snapshot (full) backup and the log backup. From 5b10a5120e5a7a623649471afdb6a4927816e38f Mon Sep 17 00:00:00 2001 From: Yasuo Honda Date: Thu, 11 Apr 2024 11:25:28 +0900 Subject: [PATCH 2/4] Update br/backup-and-restore-storages.md Co-authored-by: xixirangrang --- br/backup-and-restore-storages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/backup-and-restore-storages.md b/br/backup-and-restore-storages.md index 06d160bd50d99..6d9ffc8774eb0 100644 --- a/br/backup-and-restore-storages.md +++ b/br/backup-and-restore-storages.md @@ -106,7 +106,7 @@ When storing backup data in a cloud storage system, you need to configure authen Before backup, configure the following privileges to access the backup directory on S3. -- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:PutObject`, `s3:DeleteObject`, and `s3:AbortMultipartUpload` +- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, `s3:PutObject` and `s3:AbortMultipartUpload` - Minimum privileges for TiKV and BR to access the backup directories during restore: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, and `s3:PutObject`. BR writes checkpoint information to the `./checkpoints` subdirectory under the backup directory. When restoring log backup data, BR writes the table ID mapping relationship of the restored cluster to the `./pitr_id_maps` subdirectory under the backup directory. If you have not yet created a backup directory, refer to [Create a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) to create an S3 bucket in the specified region. If necessary, you can also create a folder in the bucket by referring to [Create a folder](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html). From 8b279ced5433342cfac0c7ee6be0b4b5e5eced1a Mon Sep 17 00:00:00 2001 From: Yasuo Honda Date: Thu, 11 Apr 2024 12:05:32 +0900 Subject: [PATCH 3/4] Update br/backup-and-restore-use-cases.md Co-authored-by: xixirangrang --- br/backup-and-restore-use-cases.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/backup-and-restore-use-cases.md b/br/backup-and-restore-use-cases.md index f5ee844f2b6f1..9da76cf987a74 100644 --- a/br/backup-and-restore-use-cases.md +++ b/br/backup-and-restore-use-cases.md @@ -69,7 +69,7 @@ The detailed steps are as follows: 2. Configure permissions for BR and TiKV to access the S3 directory. It is recommended to grant permissions using the IAM method, which is the most secure way to access the S3 bucket. For detailed steps, refer to [AWS documentation: Controlling access to a bucket with user policies](https://docs.aws.amazon.com/AmazonS3/latest/userguide/walkthrough1.html). The required permissions are as follows: - - TiKV and BR in the backup cluster need `s3:ListBucket`, `s3:PutObject`, `s3:DeleteObject`, and `s3:AbortMultipartUpload` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. + - TiKV and BR in the backup cluster need `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, `s3:PutObject`, and `s3:AbortMultipartUpload` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. - TiKV and BR in the restore cluster need `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, and `s3:PutObject` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. 3. Plan the directory structure that stores the backup data, including the snapshot (full) backup and the log backup. From 488208a16e8ee17eb8929ef70389912c5126d10b Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 11 Apr 2024 16:37:24 +0800 Subject: [PATCH 4/4] Update br/backup-and-restore-storages.md Co-authored-by: Aolin --- br/backup-and-restore-storages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/backup-and-restore-storages.md b/br/backup-and-restore-storages.md index 6d9ffc8774eb0..4ca7a259875c3 100644 --- a/br/backup-and-restore-storages.md +++ b/br/backup-and-restore-storages.md @@ -106,7 +106,7 @@ When storing backup data in a cloud storage system, you need to configure authen Before backup, configure the following privileges to access the backup directory on S3. -- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, `s3:PutObject` and `s3:AbortMultipartUpload` +- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, `s3:PutObject`, and `s3:AbortMultipartUpload` - Minimum privileges for TiKV and BR to access the backup directories during restore: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, and `s3:PutObject`. BR writes checkpoint information to the `./checkpoints` subdirectory under the backup directory. When restoring log backup data, BR writes the table ID mapping relationship of the restored cluster to the `./pitr_id_maps` subdirectory under the backup directory. If you have not yet created a backup directory, refer to [Create a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) to create an S3 bucket in the specified region. If necessary, you can also create a folder in the bucket by referring to [Create a folder](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html).