-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Provide built-in GraalVM Reachability Metadata
- Loading branch information
1 parent
de93e07
commit 471e302
Showing
24 changed files
with
1,168 additions
and
44 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
# | ||
# Licensed to the Apache Software Foundation (ASF) under one or more | ||
# contributor license agreements. See the NOTICE file distributed with | ||
# this work for additional information regarding copyright ownership. | ||
# The ASF licenses this file to You under the Apache License, Version 2.0 | ||
# (the "License"); you may not use this file except in compliance with | ||
# the License. You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
# | ||
|
||
name: NativeTest by GraalVM CE with Maven | ||
|
||
on: | ||
push: | ||
branches: [ master ] | ||
pull_request: | ||
branches: [ master ] | ||
|
||
jobs: | ||
build: | ||
strategy: | ||
matrix: | ||
java: [ '17.0.8' ] | ||
os: [ 'ubuntu-latest' ] | ||
runs-on: ${{ matrix.os }} | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- name: Set up GraalVM CE ${{ matrix.java }} | ||
uses: graalvm/setup-graalvm@v1 | ||
with: | ||
java-version: ${{ matrix.java }} | ||
distribution: 'graalvm-community' | ||
github-token: ${{ secrets.GITHUB_TOKEN }} | ||
cache: 'maven' | ||
- name: Build NativeTest with Maven | ||
continue-on-error: true | ||
run: | | ||
./mvnw -PnativeTestInElasticJob -T1C -B -e clean test |
82 changes: 82 additions & 0 deletions
82
docs/content/user-manual/elasticjob-lite/configuration/graalvm-native-image.cn.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,82 @@ | ||
+++ | ||
title = "GraalVM Native Image 支持" | ||
weight = 6 | ||
chapter = true | ||
+++ | ||
|
||
## 对 GraalVM Native Image 的支持 | ||
|
||
ElasticJob Lite 提供了构建 GraalVM Native Image 所需要的 GraalVM Reachability Metadata。在 `pom.xml` 引入 `elasticjob-lite-core` 后无需额外处理。 | ||
|
||
```xml | ||
<project> | ||
<dependencies> | ||
<dependency> | ||
<groupId>org.apache.shardingsphere.elasticjob</groupId> | ||
<artifactId>elasticjob-lite-core</artifactId> | ||
<version>${project.version}</version> | ||
</dependency> | ||
</dependencies> | ||
</project> | ||
``` | ||
|
||
## 内部的 GraalVM Reachability Metadata 条目 | ||
|
||
`org.apache.shardingsphere.elasticjob:elasticjob-infra-reachability-metadata` 托管了如下 Maven 库的 GraalVM Reachability | ||
Metadata,一旦 https://github.com/oracle/graalvm-reachability-metadata 对如下库的 GraalVM Reachability Metadata 进行了 | ||
Release 流程, 下列第三方库的相关 JSON 文件将在 ElasticJob 一侧被删除。 | ||
|
||
- `org.apache.zookeeper:zookeeper:3.9.0` | ||
- `org.apache.curator:curator-client:5.5.0` | ||
- `org.apache.curator:curator-framework:5.5.0` | ||
- `org.apache.curator:curator-recipes:5.5.0` | ||
- `org.apache.shardingsphere.elasticjob:elasticjob-lite-core:${project.version}` | ||
|
||
用户可以通过在自有项目的资源根目录或测试资源文件夹,通过新建对应库信息的 `/META-INF/native-image/${project.groupId}/${project.artifactId}/${project.version}` | ||
文件夹来覆写被传入项目的 GraalVM Reachability Metadata 。 | ||
|
||
用户依然可以自己编写 GraalVM Reachability Metadata 的相关文件来为缺少 GraalVM Reachability Metadata 的类库提供 GraalVM Native Image 支持。 | ||
|
||
## 已知限制 | ||
|
||
1. 用户无法在 GraalVM Native Image 使用 elasticJobType 为 SCRIPT 的 Job。参考 https://github.com/oracle/graal/issues/7390 。 | ||
|
||
2. 用户无法在 GraalVM Native Image 下使用与 `org.apache.shardingsphere.elasticjob.tracing.api.TracingConfiguration` 相关的 Tracing 功能。 | ||
|
||
3. ElasticJob 的 Spring Boot Starter 尚未在 GraalVM Native Image 下可用。 | ||
|
||
## 贡献 GraalVM Reachability Metadata | ||
|
||
此节文本针对贡献者。假设贡献者位于新的 Ubuntu 22.04.3 实例下,可通过 `SDKMAN!` 初始化 `GraalVM CE` 环境。 | ||
|
||
```bash | ||
sdk install java 17.0.8-graalce | ||
sdk use java 17.0.8-graalce | ||
sudo apt-get install build-essential libz-dev zlib1g-dev -y | ||
``` | ||
|
||
对于与 ElasticJob 无关的 GraalVM Reachability Metadata,相关的 issue 和 PR 应当首先创建在 https://github.com/oracle/graalvm-reachability-metadata 。 | ||
该存储库使 GraalVM Native Image 的用户能够共享和重用 Java 生态系统中的库和框架的元数据。 | ||
|
||
ElasticJob 维护了一组单元测试子集,此子集避开对 Mockito 相关的 | ||
类的使用,以服务于项目 CI 对 Native Image 的测试。参考 https://github.com/mockito/mockito/issues/2862 。 | ||
|
||
- `org.apache.shardingsphere.elasticjob.lite.fixture.reachability.metadata.**` | ||
|
||
在 ElasticJob 一侧,存在 Maven Profile 为 `generateMetadata`,用户可在 ElasticJob 项目根目录执行如下命令来采集 GraalVM Reachability Metadata。 | ||
请手动删除无任何具体条目的 JSON 文件。 | ||
|
||
```bash | ||
./mvnw -PgenerateMetadata -DskipNativeTests -e -T1C clean test native:metadata-copy | ||
``` | ||
|
||
在 ElasticJob 一侧,存在 Maven Profile 为 `nativeTestInElasticJob`,用户可在 ElasticJob 项目根目录执行如下命令来执行特定于 GraalVM | ||
Native Build Tools 的 nativeTest,以验证 GraalVM Native Image 中的单元测试覆盖率。 | ||
|
||
```bash | ||
./mvnw -PnativeTestInElasticJob -T1C -e clean test | ||
``` | ||
|
||
对于特定于单元测试的 GraalVM Reachability Metadata,请放置在 `${project.basedir}/src/test/resources/META-INF/native-image/${project.artifactId}-test-metadata/` | ||
文件夹下,`${project.basedir}` 和 `${project.artifactId}` 为对应单元测试涉及的子模块。如有需要,请使用 `org.junit.jupiter.api.condition.DisabledInNativeImage` | ||
注解或 `org.graalvm.nativeimage.imagecode` 的 System Property 屏蔽部分单元测试在 GraalVM Native Image 下运行。 |
92 changes: 92 additions & 0 deletions
92
docs/content/user-manual/elasticjob-lite/configuration/graalvm-native-image.en.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
+++ | ||
title = "GraalVM Native Image Support" | ||
weight = 6 | ||
chapter = true | ||
+++ | ||
|
||
## Support for GraalVM Native Image | ||
|
||
ElasticJob Lite provides the GraalVM Reachability Metadata needed to build GraalVM Native Images. No additional | ||
processing is required after the introduction of `elasticjob-lite-core` in `pom.xml`. | ||
|
||
```xml | ||
<project> | ||
<dependencies> | ||
<dependency> | ||
<groupId>org.apache.shardingsphere.elasticjob</groupId> | ||
<artifactId>elasticjob-lite-core</artifactId> | ||
<version>${project.version}</version> | ||
</dependency> | ||
</dependencies> | ||
</project> | ||
``` | ||
|
||
## Internal GraalVM Reachability Metadata entry | ||
|
||
`org.apache.shardingsphere.elasticjob:elasticjob-infra-reachability-metadata` hosts GraalVM Reachability Metadata for | ||
the following Maven library, once https://github.com/oracle/graalvm-reachability-metadata is done Release process with | ||
the GraalVM Reachability Metadata for the following libraries, the relevant JSON files for the following third-party | ||
libraries will be deleted on the ElasticJob side. | ||
|
||
- `org.apache.zookeeper:zookeeper:3.9.0` | ||
- `org.apache.curator:curator-client:5.5.0` | ||
- `org.apache.curator:curator-framework:5.5.0` | ||
- `org.apache.curator:curator-recipes:5.5.0` | ||
- `org.apache.shardingsphere.elasticjob:elasticjob-lite-core:${project.version}` | ||
|
||
Users can create a new corresponding library folder called `/META-INF/native-image/${project.groupId}/${project.artifactId}/${project.version}` | ||
in the resource root directory of their own project or test resource folder to override the GraalVM Reachability | ||
Metadata of the incoming project. | ||
|
||
Users can still write their own GraalVM Reachability Metadata files to provide GraalVM Native Image support for | ||
libraries that lack GraalVM Reachability Metadata. | ||
|
||
## Known limitations | ||
|
||
1. Users cannot use jobs with `elasticJobType` as `SCRIPT` in GraalVM Native Image. Refer to https://github.com/oracle/graal/issues/7390 . | ||
|
||
2. Users cannot use the tracing function related to `org.apache.shardingsphere.elasticjob.tracing.api.TracingConfiguration` | ||
under GraalVM Native Image. | ||
|
||
3. ElasticJob's Spring Boot Starter is not yet available under GraalVM Native Image. | ||
|
||
## Contribute GraalVM Reachability Metadata | ||
|
||
This section text is for contributors. Assuming the contributor is under a new Ubuntu 22.04.3 instance, the `GraalVM CE` | ||
environment can be initialized via `SDKMAN!`. | ||
|
||
```bash | ||
sdk install java 17.0.8-graalce | ||
sdk use java 17.0.8-graalce | ||
sudo apt-get install build-essential libz-dev zlib1g-dev -y | ||
``` | ||
|
||
For GraalVM Reachability Metadata that is not related to ElasticJob, issues and PRs should first be created in the https://github.com/oracle/graalvm-reachability-metadata . | ||
The repository enables users of GraalVM Native Image to share and reuse metadata for libraries and frameworks in the | ||
Java ecosystem. | ||
|
||
ElasticJob maintains a subset of unit tests that eschews the use of Mockito-related classes to serve the project CI's | ||
testing of Native Images. Refer to https://github.com/mockito/mockito/issues/2862 . | ||
|
||
- `org.apache.shardingsphere.elasticjob.lite.fixture.reachability.metadata.**` | ||
|
||
On the ElasticJob side, there is a Maven Profile as `generateMetadata`, and users can execute the following command in | ||
the root directory of the ElasticJob project to capture GraalVM Reachability Metadata. Manually delete the JSON file | ||
without any specific entries. | ||
|
||
```bash | ||
./mvnw -PgenerateMetadata -DskipNativeTests -e -T1C clean test native:metadata-copy | ||
``` | ||
|
||
On the ElasticJob side, there is a Maven Profile as `nativeTestInElasticJob`, and users can execute the following | ||
command at the root of the ElasticJob project to execute GraalVM-specific nativeTest of Native Build Tools to verify | ||
unit test coverage in GraalVM Native Image. | ||
|
||
```bash | ||
./mvnw -PnativeTestInElasticJob -T1C -e clean test | ||
``` | ||
|
||
For GraalVM Reachability Metadata specific to unit tests, be placed in `${project.basedir}/src/test/resources/META-INF/native-image/${project.artifactId}-test-metadata/` | ||
folder, `${project.basedir}` and `${project.artifactId}` are submodules involved in the corresponding unit test. If | ||
needed, use `org.junit.jupiter.api.condition.DisabledInNativeImage` annotations or `org.graalvm.nativeimage.imagecode` | ||
System Property masked unit tests to run under GraalVM Native Image. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
30 changes: 30 additions & 0 deletions
30
elasticjob-infra/elasticjob-infra-reachability-metadata/pom.xml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
<?xml version="1.0" encoding="UTF-8"?> | ||
<!-- | ||
~ Licensed to the Apache Software Foundation (ASF) under one or more | ||
~ contributor license agreements. See the NOTICE file distributed with | ||
~ this work for additional information regarding copyright ownership. | ||
~ The ASF licenses this file to You under the Apache License, Version 2.0 | ||
~ (the "License"); you may not use this file except in compliance with | ||
~ the License. You may obtain a copy of the License at | ||
~ | ||
~ http://www.apache.org/licenses/LICENSE-2.0 | ||
~ | ||
~ Unless required by applicable law or agreed to in writing, software | ||
~ distributed under the License is distributed on an "AS IS" BASIS, | ||
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
~ See the License for the specific language governing permissions and | ||
~ limitations under the License. | ||
--> | ||
|
||
<project xmlns="http://maven.apache.org/POM/4.0.0" | ||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
<modelVersion>4.0.0</modelVersion> | ||
<parent> | ||
<groupId>org.apache.shardingsphere.elasticjob</groupId> | ||
<artifactId>elasticjob-infra</artifactId> | ||
<version>3.1.0-SNAPSHOT</version> | ||
</parent> | ||
<artifactId>elasticjob-infra-reachability-metadata</artifactId> | ||
<name>${project.artifactId}</name> | ||
</project> |
86 changes: 86 additions & 0 deletions
86
...sources/META-INF/native-image/org.apache.curator/curator-client/5.5.0/reflect-config.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
[ | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"java.io.FilePermission" | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.retry.RetryForever"}, | ||
"name":"java.lang.Object", | ||
"methods":[{"name":"toString","parameterTypes":[] }] | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"java.lang.RuntimePermission" | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.retry.RetryForever"}, | ||
"name":"java.lang.StackWalker", | ||
"methods":[{"name":"getInstance","parameterTypes":["java.util.Set","int"] }, {"name":"walk","parameterTypes":["java.util.function.Function"] }] | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.retry.RetryForever"}, | ||
"name":"java.lang.StackWalker$Option" | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.retry.RetryForever"}, | ||
"name":"java.lang.StackWalker$StackFrame", | ||
"methods":[{"name":"getClassName","parameterTypes":[] }, {"name":"getFileName","parameterTypes":[] }, {"name":"getLineNumber","parameterTypes":[] }, {"name":"getMethodName","parameterTypes":[] }] | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"java.net.NetPermission" | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"java.net.SocketPermission" | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"java.net.URLPermission", | ||
"methods":[{"name":"<init>","parameterTypes":["java.lang.String","java.lang.String"] }] | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"java.security.AllPermission" | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"java.security.SecureRandomParameters" | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.QuorumConfigBuilder"}, | ||
"name":"java.security.SecureRandomParameters" | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"java.security.SecurityPermission" | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"java.util.PropertyPermission" | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.ZooKeeperServerEmbeddedAdapter"}, | ||
"name":"java.util.concurrent.ForkJoinTask", | ||
"fields":[{"name":"aux"}, {"name":"status"}] | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"javax.smartcardio.CardPermission" | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"sun.security.provider.NativePRNG", | ||
"methods":[{"name":"<init>","parameterTypes":[] }, {"name":"<init>","parameterTypes":["java.security.SecureRandomParameters"] }] | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.QuorumConfigBuilder"}, | ||
"name":"sun.security.provider.NativePRNG", | ||
"methods":[{"name":"<init>","parameterTypes":[] }, {"name":"<init>","parameterTypes":["java.security.SecureRandomParameters"] }] | ||
}, | ||
{ | ||
"condition":{"typeReachable":"org.apache.curator.test.DirectoryUtils"}, | ||
"name":"sun.security.provider.SHA", | ||
"methods":[{"name":"<init>","parameterTypes":[] }] | ||
} | ||
] |
Oops, something went wrong.