From f59d72f2ac025013356077fc8b88270d609e9808 Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Sun, 7 Apr 2024 15:40:54 +0800 Subject: [PATCH 01/18] add export --- TOC-tidb-cloud.md | 1 + tidb-cloud/serverless-export.md | 120 ++++++++++++++++++++++++++++++++ 2 files changed, 121 insertions(+) create mode 100644 tidb-cloud/serverless-export.md diff --git a/TOC-tidb-cloud.md b/TOC-tidb-cloud.md index b78eed22c45ec..ee133209ec36b 100644 --- a/TOC-tidb-cloud.md +++ b/TOC-tidb-cloud.md @@ -124,6 +124,7 @@ - [GitHub Integration](/tidb-cloud/branch-github-integration.md) - [Manage Spending Limit](/tidb-cloud/manage-serverless-spend-limit.md) - [Back Up and Restore TiDB Serverless Data](/tidb-cloud/backup-and-restore-serverless.md) + - [Export Data from TiDB Serverless](/tidb-cloud/serverless-export.md) - Manage TiDB Dedicated Clusters - [Create a TiDB Dedicated Cluster](/tidb-cloud/create-tidb-cluster.md) - Connect to Your TiDB Dedicated Cluster diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md new file mode 100644 index 0000000000000..b0cbe61c1a49e --- /dev/null +++ b/tidb-cloud/serverless-export.md @@ -0,0 +1,120 @@ +--- +title: TiDB Serverless Export (Beta) +summary: Learn how to export data from TiDB Serverless clusters. +--- + +# TiDB Serverless Export (Beta) + +TiDB Serverless Export (Beta) is a service that enables you to export data from a TiDB Serverless cluster to local storage or an external storage service. You can use the exported data for backup, migration, data analysis, or other purposes. + +While you can also export data using `mysqldump`, TiDB Dumpling, or other tools, TiDB Serverless Export offers a more convenient and efficient way to export data from a TiDB Serverless cluster. It brings the following benefits: + +- Convenience: the export service provides a simple and easy-to-use way to export data from a TiDB Serverless cluster, eliminating the need for additional tools or resources. +- Isolation: the export service uses separate computing resources, ensuring isolation from the resources used by your online services. +- Consistency: the export service ensures the consistency of the exported data without causing locks, which does not affect your online services. + +## Features + +### Location of files + +You can export data to the local storage or an external storage service. + +**Local storage** + +There are some limitations when you export data to local storage: + +1. You are not allowed to export multiple databases at the same time. +2. The exported data will be expired after two days, please download the data in time. +3. The exported data will be saved in the stashing area, which offers 250 GB storage space for each organization per region. If the storage space is full, you will not be able to export data to local. + +**[Amazon S3](https://aws.amazon.com/s3/)** + +You need to provide the credentials of the S3 bucket. Supported credentials include: + +- [Access Key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html): The long-term credentials for an IAM user or the AWS account root user. Please make sure the access key has the necessary permissions to your S3 bucket, we recommend you create a new bucket with full s3 access. + +> **Note:** +> +> We recommend you export to the external storage service such as S3 when you want to export a large amount of data. + +### Data Filtering + +You can filter data by specifying the database and table you want to export. If you do not specify the table, we will export all tables in the specified database. If you do not specify the database, we will export all databases in the cluster. + +> **Note:** +> +> You must specify the database when you export data to local storage. + +### Data Formats + +You can export data in the following formats: + +- SQL(default): export data in SQL format. +- CSV: export data in CSV format. + +### Data Compression + +You can compress the exported data in the following algorithms: + +- gzip(default): compress the exported data with gzip +- snappy: compress the exported data with snappy. +- zstd: compress the exported data with zstd. +- none: do not compress the exported data. + +### Cancel Export + +You can cancel an export job that is in running state. + +## Examples + +Now, you can manage exports with [TiDB Cloud CLI](/tidb-cloud/cli-reference.md). + +### Export to local + +First, create an export job which specifies the database and table you want to export. It will output the export ID. + + ```sh + ticloud serverless export create -c --database --table + ``` + +Then, download the exported data after the export is succeeded. + + ```sh + ticloud serverless export download -c -e + ``` + +### Export to S3 + + ```sh + ticloud serverless export create -c --bucket-uri --access-key-id --secret-access-key + ``` + +### Export with CSV format + + ```sh + ticloud serverless export create -c --file-type CSV + ``` + +### Export the whole database + + ```sh + ticloud serverless export create -c --database + ``` + +### Export with snappy compression + + ```sh + ticloud serverless export create -c --compress snappy + ``` + +### Cancel an export job + + ```sh + ticloud serverless export cancel -c -e + ``` + +## Pricing + +You will only be charged for a successful or canceled export. + +The export service is free during the beta period. You only need to pay for the [Request Units (RUs)](/tidb-cloud/tidb-cloud-glossary.md#request-unit) generated during the export process. \ No newline at end of file From 83160e29113beb097329271e69ed2a8639bd97c4 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Sun, 7 Apr 2024 17:25:38 +0800 Subject: [PATCH 02/18] refine descriptions --- tidb-cloud/serverless-export.md | 86 ++++++++++++++++----------------- 1 file changed, 43 insertions(+), 43 deletions(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index b0cbe61c1a49e..a54bd6120b192 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -7,7 +7,7 @@ summary: Learn how to export data from TiDB Serverless clusters. TiDB Serverless Export (Beta) is a service that enables you to export data from a TiDB Serverless cluster to local storage or an external storage service. You can use the exported data for backup, migration, data analysis, or other purposes. -While you can also export data using `mysqldump`, TiDB Dumpling, or other tools, TiDB Serverless Export offers a more convenient and efficient way to export data from a TiDB Serverless cluster. It brings the following benefits: +While you can also export data using tools such as [mysqldump](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html) and TiDB [Dumpling](https://docs.pingcap.com/tidb/dev/dumpling-overview), TiDB Serverless Export offers a more convenient and efficient way to export data from a TiDB Serverless cluster. It brings the following benefits: - Convenience: the export service provides a simple and easy-to-use way to export data from a TiDB Serverless cluster, eliminating the need for additional tools or resources. - Isolation: the export service uses separate computing resources, ensuring isolation from the resources used by your online services. @@ -15,106 +15,106 @@ While you can also export data using `mysqldump`, TiDB Dumpling, or other tools, ## Features -### Location of files +This section describes the features of TiDB Serverless Export. -You can export data to the local storage or an external storage service. +### Export location -**Local storage** +You can export data to local storage or [Amazon S3](https://aws.amazon.com/s3/). -There are some limitations when you export data to local storage: +> **Note:** +> +> If the size of the data to be exported is large, it is recommended that you export it to Amazon S3. -1. You are not allowed to export multiple databases at the same time. -2. The exported data will be expired after two days, please download the data in time. -3. The exported data will be saved in the stashing area, which offers 250 GB storage space for each organization per region. If the storage space is full, you will not be able to export data to local. +**Local storage** -**[Amazon S3](https://aws.amazon.com/s3/)** +Exporting data to local storage has the following limitations: -You need to provide the credentials of the S3 bucket. Supported credentials include: +- Exporting multiple databases to local storage at the same time is not supported. +- Exported data is saved in the stashing area and will expire after two days. You need to download the exported data in time. +- TiDB Cloud offers 250 GiB of storage space in the stashing area for each organization per region. If the storage space is full, you will not be able to export data to local storage. -- [Access Key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html): The long-term credentials for an IAM user or the AWS account root user. Please make sure the access key has the necessary permissions to your S3 bucket, we recommend you create a new bucket with full s3 access. +**Amazon S3** -> **Note:** -> -> We recommend you export to the external storage service such as S3 when you want to export a large amount of data. +To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket and make sure the access key has the necessary permissions for your S3 bucket. It is recommended that you create a new bucket with full S3 access. -### Data Filtering +### Data filtering -You can filter data by specifying the database and table you want to export. If you do not specify the table, we will export all tables in the specified database. If you do not specify the database, we will export all databases in the cluster. +You can filter data by specifying the database and table you want to export. If you specify a database without specifying a table, all tables in that specified database will be exported. If you do not specify a database when you export data to Amazon S3, all databases in the cluster will be exported. > **Note:** > > You must specify the database when you export data to local storage. -### Data Formats +### Data formats You can export data in the following formats: -- SQL(default): export data in SQL format. -- CSV: export data in CSV format. +- `SQL` (default): export data in SQL format. +- `CSV`: export data in CSV format. -### Data Compression +### Data compression -You can compress the exported data in the following algorithms: +You can compress the exported data using the following algorithms: -- gzip(default): compress the exported data with gzip -- snappy: compress the exported data with snappy. -- zstd: compress the exported data with zstd. -- none: do not compress the exported data. +- `gzip` (default): compress the exported data with gzip. +- `snappy`: compress the exported data with snappy. +- `zstd`: compress the exported data with zstd. +- `none`: do not compress the exported data. -### Cancel Export +### Cancel export -You can cancel an export job that is in running state. +You can cancel an export job that is in the running state. ## Examples -Now, you can manage exports with [TiDB Cloud CLI](/tidb-cloud/cli-reference.md). +Currently, you can manage export jobs using [TiDB Cloud CLI](/tidb-cloud/cli-reference.md). -### Export to local +### Export data to local storage -First, create an export job which specifies the database and table you want to export. It will output the export ID. +1. Create an export job that specifies the database and table you want to export: - ```sh + ```shell ticloud serverless export create -c --database --table
``` -Then, download the exported data after the export is succeeded. + You will get an export ID from the output. - ```sh +2. After the export is successful, download the exported data to your local storage: + + ```shell ticloud serverless export download -c -e ``` -### Export to S3 +### Export data to Amazon S3 - ```sh + ```shell ticloud serverless export create -c --bucket-uri --access-key-id --secret-access-key ``` -### Export with CSV format +### Export with the CSV format - ```sh + ```shell ticloud serverless export create -c --file-type CSV ``` ### Export the whole database - ```sh + ```shell ticloud serverless export create -c --database ``` ### Export with snappy compression - ```sh + ```shell ticloud serverless export create -c --compress snappy ``` ### Cancel an export job - ```sh + ```shell ticloud serverless export cancel -c -e ``` ## Pricing -You will only be charged for a successful or canceled export. - -The export service is free during the beta period. You only need to pay for the [Request Units (RUs)](/tidb-cloud/tidb-cloud-glossary.md#request-unit) generated during the export process. \ No newline at end of file +The export service is free during the beta period. You only need to pay for the [Request Units (RUs)](/tidb-cloud/tidb-cloud-glossary.md#request-unit) generated during the export process of successful or canceled jobs. For failed export jobs, you will not be charged. \ No newline at end of file From da8fc5e524818d73d19eaadec557c01b30ca5c97 Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Sun, 7 Apr 2024 18:33:25 +0800 Subject: [PATCH 03/18] add permissions --- tidb-cloud/serverless-export.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index a54bd6120b192..2bbacc28d1da9 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -23,7 +23,7 @@ You can export data to local storage or [Amazon S3](https://aws.amazon.com/s3/). > **Note:** > -> If the size of the data to be exported is large, it is recommended that you export it to Amazon S3. +> If the size of the data to be exported is large(more than 100 GiB), it is recommended that you export it to Amazon S3. **Local storage** @@ -35,7 +35,8 @@ Exporting data to local storage has the following limitations: **Amazon S3** -To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket and make sure the access key has the necessary permissions for your S3 bucket. It is recommended that you create a new bucket with full S3 access. +To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket. Please make sure the access key has the read and write access for your S3 bucket, including at least these permissions: `s3:PutObject`, and `s3:ListBucket`. + ### Data filtering From e6ad63d08d871ff6329451100f028c43d33043e8 Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Sun, 7 Apr 2024 18:33:25 +0800 Subject: [PATCH 04/18] add permissions --- tidb-cloud/serverless-export.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index a54bd6120b192..d6e00690a563a 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -23,7 +23,7 @@ You can export data to local storage or [Amazon S3](https://aws.amazon.com/s3/). > **Note:** > -> If the size of the data to be exported is large, it is recommended that you export it to Amazon S3. +> If the size of the data to be exported is large(more than 100 GiB), it is recommended that you export it to Amazon S3. **Local storage** @@ -35,7 +35,7 @@ Exporting data to local storage has the following limitations: **Amazon S3** -To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket and make sure the access key has the necessary permissions for your S3 bucket. It is recommended that you create a new bucket with full S3 access. +To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket. Please make sure the access key has the read and write access for your S3 bucket, including at least these permissions: `s3:PutObject`, and `s3:ListBucket`. ### Data filtering From 6cf227fb02e5bb942895da714d0799bbd349952f Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 11 Apr 2024 17:49:00 +0800 Subject: [PATCH 05/18] Apply suggestions from code review --- tidb-cloud/serverless-export.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index d6e00690a563a..1513065daf4bb 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -63,15 +63,15 @@ You can compress the exported data using the following algorithms: ### Cancel export -You can cancel an export job that is in the running state. +You can cancel an export task that is in the running state. ## Examples -Currently, you can manage export jobs using [TiDB Cloud CLI](/tidb-cloud/cli-reference.md). +Currently, you can manage export tasks using [TiDB Cloud CLI](/tidb-cloud/cli-reference.md). ### Export data to local storage -1. Create an export job that specifies the database and table you want to export: +1. Create an export task that specifies the database and table you want to export: ```shell ticloud serverless export create -c --database --table
@@ -109,7 +109,7 @@ Currently, you can manage export jobs using [TiDB Cloud CLI](/tidb-cloud/cli-ref ticloud serverless export create -c --compress snappy ``` -### Cancel an export job +### Cancel an export task ```shell ticloud serverless export cancel -c -e @@ -117,4 +117,4 @@ Currently, you can manage export jobs using [TiDB Cloud CLI](/tidb-cloud/cli-ref ## Pricing -The export service is free during the beta period. You only need to pay for the [Request Units (RUs)](/tidb-cloud/tidb-cloud-glossary.md#request-unit) generated during the export process of successful or canceled jobs. For failed export jobs, you will not be charged. \ No newline at end of file +The export service is free during the beta period. You only need to pay for the [Request Units (RUs)](/tidb-cloud/tidb-cloud-glossary.md#request-unit) generated during the export process of successful or canceled tasks. For failed export tasks, you will not be charged. \ No newline at end of file From 2653751f33571eb2dc8545349acd846b94f90609 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 12 Apr 2024 15:24:19 +0800 Subject: [PATCH 06/18] Apply suggestions from code review Co-authored-by: Lilian Lee --- tidb-cloud/serverless-export.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index 1513065daf4bb..b35dceb522626 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -23,7 +23,7 @@ You can export data to local storage or [Amazon S3](https://aws.amazon.com/s3/). > **Note:** > -> If the size of the data to be exported is large(more than 100 GiB), it is recommended that you export it to Amazon S3. +> If the size of the data to be exported is large (more than 100 GiB), it is recommended that you export it to Amazon S3. **Local storage** @@ -35,7 +35,7 @@ Exporting data to local storage has the following limitations: **Amazon S3** -To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket. Please make sure the access key has the read and write access for your S3 bucket, including at least these permissions: `s3:PutObject`, and `s3:ListBucket`. +To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket. Make sure the access key has the read and write access to your S3 bucket, including at least these permissions: `s3:PutObject`, and `s3:ListBucket`. ### Data filtering @@ -73,17 +73,17 @@ Currently, you can manage export tasks using [TiDB Cloud CLI](/tidb-cloud/cli-re 1. Create an export task that specifies the database and table you want to export: - ```shell - ticloud serverless export create -c --database --table
- ``` + ```shell + ticloud serverless export create -c --database --table
+ ``` You will get an export ID from the output. 2. After the export is successful, download the exported data to your local storage: - ```shell - ticloud serverless export download -c -e - ``` + ```shell + ticloud serverless export download -c -e + ``` ### Export data to Amazon S3 From acb17482c234dd0b7d3bcb1e00659f236c75eecd Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 12 Apr 2024 15:26:15 +0800 Subject: [PATCH 07/18] Update tidb-cloud/serverless-export.md --- tidb-cloud/serverless-export.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index b35dceb522626..65cdcf27fc194 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -35,7 +35,7 @@ Exporting data to local storage has the following limitations: **Amazon S3** -To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket. Make sure the access key has the read and write access to your S3 bucket, including at least these permissions: `s3:PutObject`, and `s3:ListBucket`. +To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket. Make sure the access key has read and write access to your S3 bucket, including at least these permissions: `s3:PutObject` and `s3:ListBucket`. ### Data filtering From 627d6973af6400c2f4de66b361e8d3d547d2c13e Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 12 Apr 2024 15:32:16 +0800 Subject: [PATCH 08/18] fix the indent issues --- tidb-cloud/serverless-export.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index 2bbacc28d1da9..a0a9a75b52be9 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -88,33 +88,33 @@ Currently, you can manage export jobs using [TiDB Cloud CLI](/tidb-cloud/cli-ref ### Export data to Amazon S3 - ```shell - ticloud serverless export create -c --bucket-uri --access-key-id --secret-access-key - ``` +```shell +ticloud serverless export create -c --bucket-uri --access-key-id --secret-access-key +``` ### Export with the CSV format - ```shell - ticloud serverless export create -c --file-type CSV - ``` +```shell +ticloud serverless export create -c --file-type CSV +``` ### Export the whole database - ```shell - ticloud serverless export create -c --database - ``` +```shell +ticloud serverless export create -c --database +``` ### Export with snappy compression - ```shell - ticloud serverless export create -c --compress snappy - ``` +```shell +ticloud serverless export create -c --compress snappy +``` ### Cancel an export job - ```shell - ticloud serverless export cancel -c -e - ``` +```shell +ticloud serverless export cancel -c -e +``` ## Pricing From 47f2348994f683dcdef57b87bea079c046fa3ee5 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 12 Apr 2024 15:37:23 +0800 Subject: [PATCH 09/18] job -> task --- tidb-cloud/serverless-export.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index a0a9a75b52be9..f791a44fa4ca0 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -64,15 +64,15 @@ You can compress the exported data using the following algorithms: ### Cancel export -You can cancel an export job that is in the running state. +You can cancel an export task that is in the running state. ## Examples -Currently, you can manage export jobs using [TiDB Cloud CLI](/tidb-cloud/cli-reference.md). +Currently, you can manage export tasks using [TiDB Cloud CLI](/tidb-cloud/cli-reference.md). ### Export data to local storage -1. Create an export job that specifies the database and table you want to export: +1. Create an export task that specifies the database and table you want to export: ```shell ticloud serverless export create -c --database --table
@@ -110,7 +110,7 @@ ticloud serverless export create -c --database ticloud serverless export create -c --compress snappy ``` -### Cancel an export job +### Cancel an export task ```shell ticloud serverless export cancel -c -e @@ -118,4 +118,4 @@ ticloud serverless export cancel -c -e ## Pricing -The export service is free during the beta period. You only need to pay for the [Request Units (RUs)](/tidb-cloud/tidb-cloud-glossary.md#request-unit) generated during the export process of successful or canceled jobs. For failed export jobs, you will not be charged. \ No newline at end of file +The export service is free during the beta period. You only need to pay for the [Request Units (RUs)](/tidb-cloud/tidb-cloud-glossary.md#request-unit) generated during the export process of successful or canceled tasks. For failed export tasks, you will not be charged. \ No newline at end of file From 91b240dc80b6b756b002eac6ea05f4c91a582015 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 12 Apr 2024 15:39:32 +0800 Subject: [PATCH 10/18] Apply suggestions from code review --- tidb-cloud/serverless-export.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index f791a44fa4ca0..09002323dbef0 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -35,7 +35,7 @@ Exporting data to local storage has the following limitations: **Amazon S3** -To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket. Please make sure the access key has the read and write access for your S3 bucket, including at least these permissions: `s3:PutObject`, and `s3:ListBucket`. +To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket. Make sure the access key has read and write access for your S3 bucket, including at least these permissions: `s3:PutObject` and `s3:ListBucket`. ### Data filtering From 06df57f3eb92a43a81c5c31ce981190338d56ea8 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 12 Apr 2024 15:41:38 +0800 Subject: [PATCH 11/18] Apply suggestions from code review --- tidb-cloud/serverless-export.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index 09002323dbef0..6b26184cf6b3c 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -23,7 +23,7 @@ You can export data to local storage or [Amazon S3](https://aws.amazon.com/s3/). > **Note:** > -> If the size of the data to be exported is large(more than 100 GiB), it is recommended that you export it to Amazon S3. +> If the size of the data to be exported is large (more than 100 GiB), it is recommended that you export it to Amazon S3. **Local storage** From 3ad43b5e77096101d1fdf9b6ff0510f3f5d8baf5 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Fri, 12 Apr 2024 15:57:41 +0800 Subject: [PATCH 12/18] Apply suggestions from code review Co-authored-by: Lilian Lee --- tidb-cloud/serverless-export.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index 6b26184cf6b3c..882ec53ab3150 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -1,9 +1,9 @@ --- -title: TiDB Serverless Export (Beta) +title: Export Data from TiDB Serverless summary: Learn how to export data from TiDB Serverless clusters. --- -# TiDB Serverless Export (Beta) +# Export Data from TiDB Serverless TiDB Serverless Export (Beta) is a service that enables you to export data from a TiDB Serverless cluster to local storage or an external storage service. You can use the exported data for backup, migration, data analysis, or other purposes. From c7b9472b7699762d13272437a648d678ac0adb7a Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Fri, 12 Apr 2024 16:07:05 +0800 Subject: [PATCH 13/18] fix verify --- tidb-cloud/serverless-export.md | 1 - 1 file changed, 1 deletion(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index 882ec53ab3150..62bf5ff5cb9c7 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -37,7 +37,6 @@ Exporting data to local storage has the following limitations: To export data to Amazon S3, you need to provide an [access key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) for your S3 bucket. Make sure the access key has read and write access for your S3 bucket, including at least these permissions: `s3:PutObject` and `s3:ListBucket`. - ### Data filtering You can filter data by specifying the database and table you want to export. If you specify a database without specifying a table, all tables in that specified database will be exported. If you do not specify a database when you export data to Amazon S3, all databases in the cluster will be exported. From de9e7236dedfd938cb5cab1ebdcae07e0f65f11e Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Mon, 15 Apr 2024 16:03:42 +0800 Subject: [PATCH 14/18] fix according to the review --- tidb-cloud/serverless-export.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index 62bf5ff5cb9c7..3d06f19069163 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -31,7 +31,7 @@ Exporting data to local storage has the following limitations: - Exporting multiple databases to local storage at the same time is not supported. - Exported data is saved in the stashing area and will expire after two days. You need to download the exported data in time. -- TiDB Cloud offers 250 GiB of storage space in the stashing area for each organization per region. If the storage space is full, you will not be able to export data to local storage. +- If the storage space of stashing area is full, you will not be able to export data to local storage. **Amazon S3** @@ -52,6 +52,12 @@ You can export data in the following formats: - `SQL` (default): export data in SQL format. - `CSV`: export data in CSV format. +Then the schema and data will be exported in the following naming rules: + +- Database schema: {schema}-schema-create.sql +- Table schema: {schema}.{table}-schema.sql +- Data: {schema}.{table}.{0001}.{sql|csv} + ### Data compression You can compress the exported data using the following algorithms: From c5bc63b9118bcd4a72294edd2f518b0d662c5742 Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Mon, 15 Apr 2024 16:29:10 +0800 Subject: [PATCH 15/18] fix according to the review --- tidb-cloud/serverless-export.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index 3d06f19069163..ad84751bb1b72 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -54,9 +54,10 @@ You can export data in the following formats: Then the schema and data will be exported in the following naming rules: -- Database schema: {schema}-schema-create.sql -- Table schema: {schema}.{table}-schema.sql -- Data: {schema}.{table}.{0001}.{sql|csv} +- Database schema: {database}-schema-create.sql +- Table schema: {database}.{table}-schema.sql +- Data: {database}.{table}.{0001}.{sql|csv} +- Compressed data: {database}.{table}.{0001}.{sql|csv}.{compress} ### Data compression From 109325dfc1f7b2bb87e94ca9ac64fa34e4c71faf Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 15 Apr 2024 18:21:39 +0800 Subject: [PATCH 16/18] wording updates --- tidb-cloud/serverless-export.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index ad84751bb1b72..9958f0630476f 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -52,7 +52,7 @@ You can export data in the following formats: - `SQL` (default): export data in SQL format. - `CSV`: export data in CSV format. -Then the schema and data will be exported in the following naming rules: +The schema and data are exported according to the following naming conventions: - Database schema: {database}-schema-create.sql - Table schema: {database}.{table}-schema.sql From 9504bd42b17d2366c5c6a0abb58c90f47da0a800 Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Tue, 16 Apr 2024 16:35:12 +0800 Subject: [PATCH 17/18] alter formats --- tidb-cloud/serverless-export.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index 9958f0630476f..54f2ef987929c 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -54,10 +54,11 @@ You can export data in the following formats: The schema and data are exported according to the following naming conventions: -- Database schema: {database}-schema-create.sql -- Table schema: {database}.{table}-schema.sql -- Data: {database}.{table}.{0001}.{sql|csv} -- Compressed data: {database}.{table}.{0001}.{sql|csv}.{compress} +| item | not compressed | compressed | +|-----------------|------------------------------------------|-----------------------------------------------------| +| database schema | {database}-schema-create.sql | {database}-schema-create.sql.{compress} | +| table schema | {database}.{table}-schema.sql | {database}.{table}-schema.sql.{compress} | +| data | {database}.{table}.{0001}.{sql|csv} | {database}.{table}.{0001}.{sql|csv}.{compress} | ### Data compression From 1e8d9c9ba42b3253115daceb6e16937b05eea9f3 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Tue, 16 Apr 2024 17:19:10 +0800 Subject: [PATCH 18/18] wording updates --- tidb-cloud/serverless-export.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tidb-cloud/serverless-export.md b/tidb-cloud/serverless-export.md index 54f2ef987929c..199fd7c93c0d4 100644 --- a/tidb-cloud/serverless-export.md +++ b/tidb-cloud/serverless-export.md @@ -54,11 +54,11 @@ You can export data in the following formats: The schema and data are exported according to the following naming conventions: -| item | not compressed | compressed | +| Item | Not compressed | Compressed | |-----------------|------------------------------------------|-----------------------------------------------------| -| database schema | {database}-schema-create.sql | {database}-schema-create.sql.{compress} | -| table schema | {database}.{table}-schema.sql | {database}.{table}-schema.sql.{compress} | -| data | {database}.{table}.{0001}.{sql|csv} | {database}.{table}.{0001}.{sql|csv}.{compress} | +| Database schema | {database}-schema-create.sql | {database}-schema-create.sql.{compression-type} | +| Table schema | {database}.{table}-schema.sql | {database}.{table}-schema.sql.{compression-type} | +| Data | {database}.{table}.{0001}.{sql|csv} | {database}.{table}.{0001}.{sql|csv}.{compression-type} | ### Data compression