Skip to content

Commit

Permalink
Replaces Zookeeper with ClickHouse Keeper
Browse files Browse the repository at this point in the history
  • Loading branch information
linghengqian committed Nov 26, 2024
1 parent e6571ae commit c50437b
Show file tree
Hide file tree
Showing 22 changed files with 448 additions and 235 deletions.
1 change: 1 addition & 0 deletions RELEASE-NOTES.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
1. Infra: Support connecting to HiveServer2 through database connection pools other than HikariCP - [#33762](https://github.com/apache/shardingsphere/pull/33762)
1. Infra: Partial support for connecting to embedded ClickHouse `chDB` - [#33786](https://github.com/apache/shardingsphere/pull/33786)
1. Proxy Native: Support connecting to HiveServer2 with ZooKeeper Service Discovery enabled in GraalVM Native Image - [#33768](https://github.com/apache/shardingsphere/pull/33768)
1. Proxy Native: Support local transactions of ClickHouse under GraalVM Native Image - [#33801](https://github.com/apache/shardingsphere/pull/33801)
1. Doc: Adds documentation for ClickHouse support - [#33779](https://github.com/apache/shardingsphere/pull/33779)

### Bug Fixes
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ create table IF NOT EXISTS t_order (
order_type Int32,
user_id Int32 NOT NULL,
address_id Int64 NOT NULL,
status String
status VARCHAR(50)
) engine = MergeTree
primary key (order_id)
order by (order_id);
Expand Down Expand Up @@ -158,7 +158,7 @@ public class ExampleUtils {

### SQL 限制

ShardingSphere JDBC DataSource 尚不支持执行 ClickHouse 的 `CREATE TABLE` 语句和 `TRUNCATE TABLE` 语句。
ShardingSphere JDBC DataSource 尚不支持执行 ClickHouse 的 `create table``truncate table``drop table` 语句。
用户应考虑为 ShardingSphere 提交包含单元测试的 PR。

### 分布式序列限制
Expand Down Expand Up @@ -207,9 +207,178 @@ public class ExampleTest {

### 事务限制

ClickHouse 不支持 ShardingSphere 集成级别的本地事务,XA 事务和 Seata AT 模式事务,
ClickHouse 不支持 ShardingSphere 集成级别的 XA 事务,
因为 https://github.com/ClickHouse/clickhouse-java 未实现 `javax.sql.XADataSource` 的相关 Java 接口。

ClickHouse 不支持 ShardingSphere 集成级别的 Seata AT 模式事务,
因为 https://github.com/apache/incubator-seata 未实现 ClickHouse 的 SQL 方言解析。

ClickHouse 支持 ShardingSphere 集成级别的本地事务,但需要对 ClickHouse 进行额外配置,
更多讨论位于 https://github.com/ClickHouse/clickhouse-docs/issues/2300

引入讨论,编写 Docker Compose 文件来启动 ClickHouse 和 ClickHouse Keeper。

```yaml
services:
clickhouse-keeper-01:
image: clickhouse/clickhouse-keeper:24.10.2.80
volumes:
- ./keeper_config.xml:/etc/clickhouse-keeper/keeper_config.xml
clickhouse-server:
image: clickhouse/clickhouse-server:24.10.2.80
depends_on:
- clickhouse-keeper-01
ports:
- "8123:8123"
volumes:
- ./transactions.xml:/etc/clickhouse-server/config.d/transactions.xml
```
`./keeper_config.xml` 的内容如下,

```xml
<clickhouse replace="true">
<listen_host>0.0.0.0</listen_host>
<keeper_server>
<tcp_port>9181</tcp_port>
<server_id>1</server_id>
<snapshot_storage_path>/var/lib/clickhouse/coordination/snapshots</snapshot_storage_path>
<raft_configuration>
<server>
<id>1</id>
<hostname>clickhouse-keeper-01</hostname>
<port>9234</port>
</server>
</raft_configuration>
</keeper_server>
</clickhouse>
```

`./transactions.xml` 的内容如下,

```xml
<clickhouse>
<allow_experimental_transactions>1</allow_experimental_transactions>
<zookeeper>
<node index="1">
<host>clickhouse-keeper-01</host>
<port>9181</port>
</node>
</zookeeper>
</clickhouse>
```

在 DBeaver Community 内,使用 `jdbc:ch://localhost:8123/default` 的 `jdbcUrl`,`default` 的`username` 连接至 ClickHouse,
`password` 留空。
执行如下 SQL,

```sql
-- noinspection SqlNoDataSourceInspectionForFile
CREATE DATABASE demo_ds_0;
CREATE DATABASE demo_ds_1;
CREATE DATABASE demo_ds_2;
```

分别使用 `jdbc:ch://localhost:8123/demo_ds_0` ,
`jdbc:ch://localhost:8123/demo_ds_1` 和 `jdbc:ch://localhost:8123/demo_ds_2` 的 `jdbcUrl` 连接至 ClickHouse 来执行如下 SQL,

```sql
-- noinspection SqlNoDataSourceInspectionForFile
create table IF NOT EXISTS t_order (
order_id Int64 NOT NULL,
order_type Int32,
user_id Int32 NOT NULL,
address_id Int64 NOT NULL,
status VARCHAR(50)
) engine = MergeTree
primary key (order_id)
order by (order_id);
TRUNCATE TABLE t_order;
```

在业务项目引入`前提条件`涉及的依赖后,在业务项目的 classpath 上编写 ShardingSphere 数据源的配置文件`demo.yaml`,

```yaml
dataSources:
ds_0:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.clickhouse.jdbc.ClickHouseDriver
jdbcUrl: jdbc:ch://localhost:8123/demo_ds_0?transactionSupport=true
ds_1:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.clickhouse.jdbc.ClickHouseDriver
jdbcUrl: jdbc:ch://localhost:8123/demo_ds_1?transactionSupport=true
ds_2:
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
driverClassName: com.clickhouse.jdbc.ClickHouseDriver
jdbcUrl: jdbc:ch://localhost:8123/demo_ds_2?transactionSupport=true
rules:
- !SHARDING
tables:
t_order:
actualDataNodes:
keyGenerateStrategy:
column: order_id
keyGeneratorName: snowflake
defaultDatabaseStrategy:
standard:
shardingColumn: user_id
shardingAlgorithmName: inline
shardingAlgorithms:
inline:
type: INLINE
props:
algorithm-expression: ds_${user_id % 2}
keyGenerators:
snowflake:
type: SNOWFLAKE
```

创建 ShardingSphere 的数据源后可正常使用本地事务,

```java
import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import java.sql.Connection;
import java.sql.SQLException;
@SuppressWarnings({"SqlNoDataSourceInspection", "AssertWithSideEffects"})
public class ExampleUtils {
void test() throws SQLException {
HikariConfig config = new HikariConfig();
config.setJdbcUrl("jdbc:shardingsphere:classpath:demo.yaml");
config.setDriverClassName("org.apache.shardingsphere.driver.ShardingSphereDriver");
try (HikariDataSource dataSource = new HikariDataSource(config); Connection connection = dataSource.getConnection()) {
try {
connection.setAutoCommit(false);
connection.createStatement().executeUpdate("INSERT INTO t_order (user_id, order_type, address_id, status) VALUES (2024, 0, 2024, 'INSERT_TEST')");
connection.createStatement().executeUpdate("INSERT INTO t_order_does_not_exist (test_id_does_not_exist) VALUES (2024)");
connection.commit();
} catch (final SQLException ignored) {
connection.rollback();
} finally {
connection.setAutoCommit(true);
}
try (Connection conn = dataSource.getConnection()) {
assert !conn.createStatement().executeQuery("SELECT * FROM t_order WHERE user_id = 2024").next();
}
}
}
}
```

一旦在 ShardingSphere 的配置文件为 ClickHouse JDBC Driver 的 jdbcUrl 设置 `transactionSupport=true`,
**ClickHouse 的 DML SQL 执行速度将大幅下降**。此时,用户在执行 `insert` 和 `alter table` 等 DML 语句的前后可能需要进行等待,
以避免如下 Error 的发生,

```shell
java.sql.BatchUpdateException: Code: 341. DB::Exception: Exception happened during execution of mutation 'mutation_6.txt' with part 'all_1_1_0' reason: 'Serialization error: part all_1_1_0 is locked by transaction 5672402456378293316'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. (UNFINISHED) (version 24.10.2.80 (official build))
at com.clickhouse.jdbc.SqlExceptionUtils.batchUpdateError(SqlExceptionUtils.java:107)
at com.clickhouse.jdbc.internal.SqlBasedPreparedStatement.executeAny(SqlBasedPreparedStatement.java:223)
at com.clickhouse.jdbc.internal.SqlBasedPreparedStatement.executeLargeUpdate(SqlBasedPreparedStatement.java:302)
at com.clickhouse.jdbc.internal.AbstractPreparedStatement.executeUpdate(AbstractPreparedStatement.java:135)
```

### 嵌入式 ClickHouse 限制

嵌入式 ClickHouse `chDB` 尚未发布 Java 客户端,
Expand Down
Loading

0 comments on commit c50437b

Please sign in to comment.