Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GraalVM Reachability Metadata and corresponding nativeTest for Seata integration #30138

Merged
merged 1 commit into from
Feb 22, 2024
Merged

Add GraalVM Reachability Metadata and corresponding nativeTest for Seata integration #30138

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 48 additions & 6 deletions ...ument/content/user-manual/shardingsphere-jdbc/graalvm-native-image/_index.cn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ plugins {

dependencies {
implementation 'org.apache.shardingsphere:shardingsphere-jdbc:${shardingsphere.version}'
implementation(group: 'org.graalvm.buildtools', name: 'graalvm-reachability-metadata', version: '0.10.0', classifier: 'repository', ext: 'zip')
}

graalvmNative {
Expand All @@ -89,6 +90,9 @@ graalvmNative {
buildArgs.add('-H:+AddAllCharsets')
}
}
metadataRepository {
enabled.set(false)
}
}
```

Expand Down Expand Up @@ -181,7 +185,7 @@ Image 下使用。
```

2. 对于 `读写分离` 的功能，你需要使用 `行表达式` SPI 的其他实现，以在配置 `logic database name`，`writeDataSourceName` 和 `readDataSourceNames`
时绕开对 GroovyShell 的调用。一个可能的配置是使用 `LITERAL` 的 `行表达式` SPI 的实现。对于 `数据分片` 的功能的 `actualDataNodes` 同理。
时绕开对 GroovyShell 的调用。一个可能的配置是使用 `LITERAL` 的 `行表达式` SPI 的实现。
```yaml
rules:
- !READWRITE_SPLITTING
Expand All @@ -193,6 +197,18 @@ rules:
- <LITERAL>ds_2
```

对于 `数据分片` 的功能的 `actualDataNodes` 同理。

```yaml
- !SHARDING
tables:
t_order:
actualDataNodes: <LITERAL>ds_0.t_order_0, ds_0.t_order_1, ds_1.t_order_0, ds_1.t_order_1
keyGenerateStrategy:
column: order_id
keyGeneratorName: snowflake
```

3. 使用者依然需要在 `src/main/resources/META-INF/native-image` 文件夹或 `src/test/resources/META-INF/native-image` 文件夹配置独立
文件的 GraalVM Reachability Metadata。使用者可通过 GraalVM Native Build Tools 的 GraalVM Tracing Agent 来快速采集 GraalVM
Reachability Metadata。
Expand All @@ -201,18 +217,44 @@ Reachability Metadata。
当遇到如下 Error，使用者需要添加 `-H:+AddAllCharsets` 的 `buildArg` 到 GraalVM Native Build Tools 的配置中。

```shell
Caused by: java.io.UnsupportedEncodingException: SQL Server collation SQL_Latin1_General_CP1_CI_AS is not supported by this driver.
com.microsoft.sqlserver.jdbc.SQLCollation.encodingFromSortId(SQLCollation.java:506)
com.microsoft.sqlserver.jdbc.SQLCollation.<init>(SQLCollation.java:63)
com.microsoft.sqlserver.jdbc.SQLServerConnection.processEnvChange(SQLServerConnection.java:3174)
[...]
Caused by: java.io.UnsupportedEncodingException: Codepage Cp1252 is not supported by the Java environment.
com.microsoft.sqlserver.jdbc.Encoding.checkSupported(SQLCollation.java:572)
com.microsoft.sqlserver.jdbc.SQLCollation$SortOrder.getEncoding(SQLCollation.java:473)
com.microsoft.sqlserver.jdbc.SQLCollation.encodingFromSortId(SQLCollation.java:501)
[...]
```

5. 当使用 Seata 的 BASE 集成时，用户需要使用特定的 `io.seata:seata-all:1.8.0` 版本以避开对 ByteBuddy Java API 的使用，
并排除 `io.seata:seata-all:1.8.0` 中过时的 `org.antlr:antlr4-runtime:4.8` 的 Maven 依赖。可能的配置例子如下，

```xml
<project>
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-transaction-base-seata-at</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>io.seata</groupId>
<artifactId>seata-all</artifactId>
<version>1.8.0</version>
<exclusions>
<exclusion>
<groupId>org.antlr</groupId>
<artifactId>antlr4-runtime</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</project>
```

## 贡献 GraalVM Reachability Metadata

ShardingSphere 对在 GraalVM Native Image 下的可用性的验证，是通过 GraalVM Native Build Tools 的 Maven Plugin 子项目来完成的。
Expand Down
63 changes: 53 additions & 10 deletions ...ument/content/user-manual/shardingsphere-jdbc/graalvm-native-image/_index.en.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,12 @@ and the documentation of GraalVM Native Build Tools shall prevail.

```groovy
plugins {
id 'org.graalvm.buildtools.native' version '0.10.0'
id 'org.graalvm.buildtools.native' version '0.10.0'
}

dependencies {
implementation 'org.apache.shardingsphere:shardingsphere-jdbc:${shardingsphere.version}'
implementation 'org.apache.shardingsphere:shardingsphere-jdbc:${shardingsphere.version}'
implementation(group: 'org.graalvm.buildtools', name: 'graalvm-reachability-metadata', version: '0.10.0', classifier: 'repository', ext: 'zip')
}

graalvmNative {
Expand All @@ -92,6 +93,9 @@ graalvmNative {
buildArgs.add('-H:+AddAllCharsets')
}
}
metadataRepository {
enabled.set(false)
}
}
```

Expand Down Expand Up @@ -186,9 +190,8 @@ normally under GraalVM Native Image.
```

2. For the `ReadWrite Splitting` feature, you need to use other implementations of `Row Value Expressions` SPI to configure
`logic database name`, `writeDataSourceName` and `readDataSourceNames` when bypassing calls to GroovyShell. One possible
configuration is to use the `Row Value Expressions` SPI implementation of `LITERAL`. The same applies to `actualDataNodes`
for the `Sharding` feature.
`logic database name`, `writeDataSourceName` and `readDataSourceNames` when bypassing calls to GroovyShell.
One possible configuration is to use the `Row Value Expressions` SPI implementation of `LITERAL`.

```yaml
rules:
Expand All @@ -201,6 +204,18 @@ rules:
- <LITERAL>ds_2
```

The same applies to `actualDataNodes` for the `Sharding` feature.

```yaml
- !SHARDING
tables:
t_order:
actualDataNodes: <LITERAL>ds_0.t_order_0, ds_0.t_order_1, ds_1.t_order_0, ds_1.t_order_1
keyGenerateStrategy:
column: order_id
keyGeneratorName: snowflake
```

3. Users still need to configure GraalVM Reachability Metadata for independent files in the `src/main/resources/META-INF/native-image`
folder or `src/test/resources/META-INF/native-image` folder. Users can quickly collect GraalVM Reachability Metadata through
the GraalVM Tracing Agent of GraalVM Native Build Tools.
Expand All @@ -210,18 +225,46 @@ will dynamically load different character sets based on the encoding used in the
When encountering the following Error, users need to add the `buildArg` of `-H:+AddAllCharsets` to the configuration of GraalVM Native Build Tools.

```shell
Caused by: java.io.UnsupportedEncodingException: SQL Server collation SQL_Latin1_General_CP1_CI_AS is not supported by this driver.
com.microsoft.sqlserver.jdbc.SQLCollation.encodingFromSortId(SQLCollation.java:506)
com.microsoft.sqlserver.jdbc.SQLCollation.<init>(SQLCollation.java:63)
com.microsoft.sqlserver.jdbc.SQLServerConnection.processEnvChange(SQLServerConnection.java:3174)
[...]
Caused by: java.io.UnsupportedEncodingException: Codepage Cp1252 is not supported by the Java environment.
com.microsoft.sqlserver.jdbc.Encoding.checkSupported(SQLCollation.java:572)
com.microsoft.sqlserver.jdbc.SQLCollation$SortOrder.getEncoding(SQLCollation.java:473)
com.microsoft.sqlserver.jdbc.SQLCollation.encodingFromSortId(SQLCollation.java:501)
[...]
```

5. When using Seata's BASE integration,
users need to use a specific `io.seata:seata-all:1.8.0` version to avoid using the ByteBuddy Java API,
and exclude the outdated Maven dependency of `org.antlr:antlr4-runtime:4.8` in `io.seata:seata-all:1.8.0`.
Possible configuration examples are as follows,

```xml
<project>
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-transaction-base-seata-at</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>io.seata</groupId>
<artifactId>seata-all</artifactId>
<version>1.8.0</version>
<exclusions>
<exclusion>
<groupId>org.antlr</groupId>
<artifactId>antlr4-runtime</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</project>
```

## Contribute GraalVM Reachability Metadata

The verification of ShardingSphere's availability under GraalVM Native Image is completed through the Maven Plugin subproject
Expand Down
8 changes: 5 additions & 3 deletions ...ent/content/user-manual/shardingsphere-jdbc/special-api/transaction/seata.cn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ ALTER TABLE `undo_log` ADD INDEX `ix_log_created` (`log_created`);

```conf
sharding.transaction.seata.at.enable = true
sharding.transaction.seata.tx.timeout = 30
sharding.transaction.seata.tx.timeout = 60

client {
application.id = example
Expand Down Expand Up @@ -117,9 +117,11 @@ ShardingSphere 的 Seata 集成将获取到的 Seata 全局事务置入线程的

3. 在函数上使用 Jakarta EE 9/10 的 `jakarta.transaction.Transactional` 注解，这是被允许的。

4. 在函数上使用 `io.seata.spring.annotation.GlobalTransactional` 注解，这是不被允许的。
4. 在函数上使用 Spring Framework 的 `org.springframework.transaction.annotation.Transactional` 注解，这是被允许的。

5. 手动从 `io.seata.tm.api.GlobalTransactionContext ` 创建 `io.seata.tm.api.GlobalTransaction` 实例，
5. 在函数上使用 `io.seata.spring.annotation.GlobalTransactional` 注解，这是不被允许的。

6. 手动从 `io.seata.tm.api.GlobalTransactionContext ` 创建 `io.seata.tm.api.GlobalTransaction` 实例，
调用 `io.seata.tm.api.GlobalTransaction` 实例的 `begin()`, `commit()` 和 `rollback()` 方法，这是不被允许的。

长话短说，在使用 ShardingSphere 的 Seata 集成时，你不应该使用 Seata Java API。
8 changes: 5 additions & 3 deletions ...ent/content/user-manual/shardingsphere-jdbc/special-api/transaction/seata.en.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ A fully configured `seata.conf` is as follows,

```conf
sharding.transaction.seata.at.enable = true
sharding.transaction.seata.tx.timeout = 30
sharding.transaction.seata.tx.timeout = 60

client {
application.id = example
Expand Down Expand Up @@ -118,9 +118,11 @@ For ShardingSphere data source, discuss 5 situations,

3. Using Jakarta EE 9/10’s `jakarta.transaction.Transactional` annotation on functions is allowed.

4. Using the `io.seata.spring.annotation.GlobalTransactional` annotation on the function is not allowed.
4. Using Spring Framework’s `org.springframework.transaction.annotation.Transactional` annotation on functions is allowed.

5. Manually create `io.seata.tm.api.GlobalTransaction` instance from `io.seata.tm.api.GlobalTransactionContext`,
5. Using the `io.seata.spring.annotation.GlobalTransactional` annotation on the function is not allowed.

6. Manually create `io.seata.tm.api.GlobalTransaction` instance from `io.seata.tm.api.GlobalTransactionContext`,
calling the `begin()`, `commit()` and `rollback()` methods of an `io.seata.tm.api.GlobalTransaction` instance is not allowed.

Long story short, you should not use the Seata Java API when using ShardingSphere's Seata integration.
21 changes: 18 additions & 3 deletions ...es/META-INF/native-image/com.github.ben-manes.caffeine/caffeine/2.9.3/reflect-config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,15 +40,25 @@
"fields":[{"name":"value"}]
},
{
"condition":{"typeReachable":"com.github.benmanes.caffeine.cache.NodeFactory"},
"name":"com.github.benmanes.caffeine.cache.PDMS",
"condition":{"typeReachable":"com.github.benmanes.caffeine.cache.PDW"},
"name":"com.github.benmanes.caffeine.cache.PDW",
"fields":[{"name":"writeTime"}]
},
{
"condition":{"typeReachable":"com.github.benmanes.caffeine.cache.SIMSW"},
"name":"com.github.benmanes.caffeine.cache.PDWMS",
"methods":[{"name":"<init>","parameterTypes":[] }]
},
{
"condition":{"typeReachable":"com.github.benmanes.caffeine.cache.LocalCacheFactory"},
"condition":{"typeReachable":"com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalLoadingCache"},
"name":"com.github.benmanes.caffeine.cache.SIMS",
"methods":[{"name":"<init>","parameterTypes":["com.github.benmanes.caffeine.cache.Caffeine","com.github.benmanes.caffeine.cache.CacheLoader","boolean"] }]
},
{
"condition":{"typeReachable":"com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalManualCache"},
"name":"com.github.benmanes.caffeine.cache.SIMSW",
"methods":[{"name":"<init>","parameterTypes":["com.github.benmanes.caffeine.cache.Caffeine","com.github.benmanes.caffeine.cache.CacheLoader","boolean"] }]
},
{
"condition":{"typeReachable":"com.github.benmanes.caffeine.cache.StripedBuffer"},
"name":"com.github.benmanes.caffeine.cache.StripedBuffer",
Expand All @@ -58,5 +68,10 @@
"condition":{"typeReachable":"com.github.benmanes.caffeine.cache.StripedBuffer"},
"name":"java.lang.Thread",
"fields":[{"name":"threadLocalRandomProbe"}]
},
{
"condition":{"typeReachable":"com.github.benmanes.caffeine.cache.NodeFactory"},
"name":"com.github.benmanes.caffeine.cache.PDMS",
"methods":[{"name":"<init>","parameterTypes":[] }]
}
]
10 changes: 10 additions & 0 deletions .../org.apache.shardingsphere/shardingsphere-infra-reachability-metadata/reflect-config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1166,6 +1166,16 @@
"condition":{"typeReachable":"org.apache.shardingsphere.infra.yaml.config.shortcut.YamlRuleConfigurationShortcuts"},
"name":"org.apache.shardingsphere.traffic.yaml.config.YamlTrafficRuleConfiguration"
},
{
"condition":{"typeReachable":"org.apache.shardingsphere.transaction.ShardingSphereTransactionManagerEngine"},
"name":"org.apache.shardingsphere.transaction.base.seata.at.SeataATShardingSphereTransactionManager",
"methods":[{"name":"<init>","parameterTypes":[] }]
},
{
"condition":{"typeReachable":"org.apache.shardingsphere.infra.executor.sql.hook.SPISQLExecutionHook"},
"name":"org.apache.shardingsphere.transaction.base.seata.at.SeataTransactionalSQLExecutionHook",
"methods":[{"name":"<init>","parameterTypes":[] }]
},
{
"condition":{"typeReachable":"org.apache.shardingsphere.transaction.ShardingSphereTransactionManagerEngine"},
"name":"org.apache.shardingsphere.transaction.xa.XAShardingSphereTransactionManager",
Expand Down
4 changes: 4 additions & 0 deletions test/native/native-image-filter/extra-filter.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
{"excludeClasses": "android.app.**"},
{"excludeClasses": "com.arjuna.**"},
{"excludeClasses": "com.atomikos.logging.**"},
{"excludeClasses": "com.alibaba.druid.**"},
{"excludeClasses": "com.fasterxml.jackson.databind.**"},
{"excludeClasses": "com.github.benmanes.caffeine.cache.**"},
{"excludeClasses": "com.github.dockerjava.api.**"},
Expand All @@ -30,10 +31,13 @@
{"excludeClasses": "ch.qos.logback.classic.**"},
{"excludeClasses": "io.grpc.**"},
{"excludeClasses": "io.netty.**"},
{"excludeClasses": "io.seata.**"},
{"excludeClasses": "io.vertx.core.**"},
{"excludeClasses": "groovy.**"},
{"excludeClasses": "libcore.io.**"},
{"excludeClasses": "net.bytebuddy.**"},
{"excludeClasses": "org.apache.calcite.**"},
{"excludeClasses": "org.apache.commons.logging.**"},
{"excludeClasses": "org.apache.logging.log4j.**"},
{"excludeClasses": "org.apache.log4j.**"},
{"excludeClasses": "org.apache.zookeeper.**"},
Expand Down
30 changes: 30 additions & 0 deletions test/native/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,12 @@
<artifactId>shardingsphere-test-native</artifactId>
<name>${project.artifactId}</name>

<properties>
<!--Before merging https://github.com/apache/incubator-seata/pull/5234,
using Seata Client under GraalVM Native Image requires using the version released under its 1.x branch-->
<seata.version>1.8.0</seata.version>
</properties>

<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
Expand Down Expand Up @@ -61,31 +67,55 @@
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-transaction-xa-core</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-transaction-xa-narayana</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.narayana.jta</groupId>
<artifactId>jta</artifactId>
<version>${narayana.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.narayana.jts</groupId>
<artifactId>narayana-jts-integration</artifactId>
<version>${narayana.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss</groupId>
<artifactId>jboss-transaction-spi</artifactId>
<version>${jboss-transaction-spi.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.logging</groupId>
<artifactId>jboss-logging</artifactId>
<version>${jboss-logging.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-transaction-base-seata-at</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.seata</groupId>
<artifactId>seata-all</artifactId>
<version>${seata.version}</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.antlr</groupId>
<artifactId>antlr4-runtime</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
Expand Down
Loading
Loading