From 209abd883eee2917d70c8a1e7fda88d44b4270be Mon Sep 17 00:00:00 2001 From: Yasuo Honda Date: Wed, 10 Apr 2024 16:46:45 +0900 Subject: [PATCH 1/4] TiDB 7.1+ checkpoint restore requires `s3:DeleteObject` permission Fix #16961 --- br/backup-and-restore-storages.md | 4 ++-- br/backup-and-restore-use-cases.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/br/backup-and-restore-storages.md b/br/backup-and-restore-storages.md index 891769137e6e6..10a468924fb04 100644 --- a/br/backup-and-restore-storages.md +++ b/br/backup-and-restore-storages.md @@ -107,8 +107,8 @@ When storing backup data in a cloud storage system, you need to configure authen Before backup, configure the following privileges to access the backup directory on S3. -- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:PutObject`, and `s3:AbortMultipartUpload` -- Minimum privileges for TiKV and BR to access the backup directories during restore: `s3:ListBucket`, `s3:GetObject`, and `s3:PutObject`. BR writes checkpoint information to the `./checkpoints` subdirectory under the backup directory. When restoring log backup data, BR writes the table ID mapping relationship of the restored cluster to the `./pitr_id_maps` subdirectory under the backup directory. +- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:PutObject`, `s3:DeleteObject`, and `s3:AbortMultipartUpload` +- Minimum privileges for TiKV and BR to access the backup directories during restore: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, and `s3:PutObject`. BR writes checkpoint information to the `./checkpoints` subdirectory under the backup directory. When restoring log backup data, BR writes the table ID mapping relationship of the restored cluster to the `./pitr_id_maps` subdirectory under the backup directory. If you have not yet created a backup directory, refer to [Create a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) to create an S3 bucket in the specified region. If necessary, you can also create a folder in the bucket by referring to [Create a folder](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html). diff --git a/br/backup-and-restore-use-cases.md b/br/backup-and-restore-use-cases.md index 4f99d71de9f3b..4d5a101cdd215 100644 --- a/br/backup-and-restore-use-cases.md +++ b/br/backup-and-restore-use-cases.md @@ -70,8 +70,8 @@ The detailed steps are as follows: 2. Configure permissions for BR and TiKV to access the S3 directory. It is recommended to grant permissions using the IAM method, which is the most secure way to access the S3 bucket. For detailed steps, refer to [AWS documentation: Controlling access to a bucket with user policies](https://docs.aws.amazon.com/AmazonS3/latest/userguide/walkthrough1.html). The required permissions are as follows: - - TiKV and BR in the backup cluster need `s3:ListBucket`, `s3:PutObject`, and `s3:AbortMultipartUpload` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. - - TiKV and BR in the restore cluster need `s3:ListBucket`, `s3:GetObject`, and `s3:PutObject` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. + - TiKV and BR in the backup cluster need `s3:ListBucket`, `s3:PutObject`, `s3:DeleteObject`, and `s3:AbortMultipartUpload` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. + - TiKV and BR in the restore cluster need `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, and `s3:PutObject` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. 3. Plan the directory structure that stores the backup data, including the snapshot (full) backup and the log backup. From 7ae3cf1f936076a8269fc86323d50ed471cd8700 Mon Sep 17 00:00:00 2001 From: Yasuo Honda Date: Thu, 11 Apr 2024 11:25:28 +0900 Subject: [PATCH 2/4] Update br/backup-and-restore-storages.md Co-authored-by: xixirangrang --- br/backup-and-restore-storages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/backup-and-restore-storages.md b/br/backup-and-restore-storages.md index 10a468924fb04..52d6eac7914e1 100644 --- a/br/backup-and-restore-storages.md +++ b/br/backup-and-restore-storages.md @@ -107,7 +107,7 @@ When storing backup data in a cloud storage system, you need to configure authen Before backup, configure the following privileges to access the backup directory on S3. -- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:PutObject`, `s3:DeleteObject`, and `s3:AbortMultipartUpload` +- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, `s3:PutObject` and `s3:AbortMultipartUpload` - Minimum privileges for TiKV and BR to access the backup directories during restore: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, and `s3:PutObject`. BR writes checkpoint information to the `./checkpoints` subdirectory under the backup directory. When restoring log backup data, BR writes the table ID mapping relationship of the restored cluster to the `./pitr_id_maps` subdirectory under the backup directory. If you have not yet created a backup directory, refer to [Create a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) to create an S3 bucket in the specified region. If necessary, you can also create a folder in the bucket by referring to [Create a folder](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html). From df8d5355a1ece2afa1993b66f16c2dc0a54730d6 Mon Sep 17 00:00:00 2001 From: Yasuo Honda Date: Thu, 11 Apr 2024 12:05:32 +0900 Subject: [PATCH 3/4] Update br/backup-and-restore-use-cases.md Co-authored-by: xixirangrang --- br/backup-and-restore-use-cases.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/backup-and-restore-use-cases.md b/br/backup-and-restore-use-cases.md index 4d5a101cdd215..12fedc828d464 100644 --- a/br/backup-and-restore-use-cases.md +++ b/br/backup-and-restore-use-cases.md @@ -70,7 +70,7 @@ The detailed steps are as follows: 2. Configure permissions for BR and TiKV to access the S3 directory. It is recommended to grant permissions using the IAM method, which is the most secure way to access the S3 bucket. For detailed steps, refer to [AWS documentation: Controlling access to a bucket with user policies](https://docs.aws.amazon.com/AmazonS3/latest/userguide/walkthrough1.html). The required permissions are as follows: - - TiKV and BR in the backup cluster need `s3:ListBucket`, `s3:PutObject`, `s3:DeleteObject`, and `s3:AbortMultipartUpload` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. + - TiKV and BR in the backup cluster need `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, `s3:PutObject`, and `s3:AbortMultipartUpload` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. - TiKV and BR in the restore cluster need `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, and `s3:PutObject` permissions of the `s3://tidb-pitr-bucket/backup-data` directory. 3. Plan the directory structure that stores the backup data, including the snapshot (full) backup and the log backup. From 3024bc2b964442ba7f39e9bf9f36acd8b8e47107 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Thu, 11 Apr 2024 16:37:24 +0800 Subject: [PATCH 4/4] Update br/backup-and-restore-storages.md Co-authored-by: Aolin --- br/backup-and-restore-storages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/backup-and-restore-storages.md b/br/backup-and-restore-storages.md index 52d6eac7914e1..ceaed93ddbd88 100644 --- a/br/backup-and-restore-storages.md +++ b/br/backup-and-restore-storages.md @@ -107,7 +107,7 @@ When storing backup data in a cloud storage system, you need to configure authen Before backup, configure the following privileges to access the backup directory on S3. -- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, `s3:PutObject` and `s3:AbortMultipartUpload` +- Minimum privileges for TiKV and Backup & Restore (BR) to access the backup directories during backup: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, `s3:PutObject`, and `s3:AbortMultipartUpload` - Minimum privileges for TiKV and BR to access the backup directories during restore: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, and `s3:PutObject`. BR writes checkpoint information to the `./checkpoints` subdirectory under the backup directory. When restoring log backup data, BR writes the table ID mapping relationship of the restored cluster to the `./pitr_id_maps` subdirectory under the backup directory. If you have not yet created a backup directory, refer to [Create a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) to create an S3 bucket in the specified region. If necessary, you can also create a folder in the bucket by referring to [Create a folder](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html).