Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
  • Loading branch information
@shaomengwang
shaomengwang committed Jan 7, 2021
1 parent 34f0146 commit 8efd044
Show file tree
Hide file tree
Showing 2 changed files with 70 additions and 23 deletions.
47 changes: 35 additions & 12 deletions README.en-US.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,16 +32,16 @@ Welcome everyone to join the Alink open source user group to communicate.

#### About package names and versions:
- PyAlink provides different Python packages for Flink versions that Alink supports:
package `pyalink` always maintains Alink Python API against the latest Flink version, which is 1.11,
while `pyalink-flink-***` support old-version Flink, which are `pyalink-flink-1.10` and `pyalink-flink-1.9` for now.
- The version of python packages always follows Alink Java version, like `1.3.0`.
package `pyalink` always maintains Alink Python API against the latest Flink version, which is 1.12,
while `pyalink-flink-***` support old-version Flink, which are `pyalink-flink-1.11`, `pyalink-flink-1.10` and `pyalink-flink-1.9` for now.
- The version of python packages always follows Alink Java version, like `1.3.1`.

#### Installation steps:

1. Make sure the version of python3 on your computer is 3.6 or 3.7.
2. Make sure Java 8 is installed on your computer.
3. Use pip to install:
`pip install pyalink`, `pip install pyalink-flink-1.10` or `pip install pyalink-flink-1.9`.
`pip install pyalink`, `pip install pyalink-flink-1.11`, `pip install pyalink-flink-1.10` or `pip install pyalink-flink-1.9`.


#### Potential issues:
Expand All @@ -50,9 +50,10 @@ Welcome everyone to join the Alink open source user group to communicate.
If `pyalink` or `pyalink-flink-***` was/were installed, please use `pip uninstall pyalink` or `pip uninstall pyalink-flink-***` to remove them.

2. If `pip install` is slow of failed, refer to [this article](https://segmentfault.com/a/1190000006111096) to change the pip source, or use the following download links:
- Flink 1.11：[Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink-1.3.0-py3-none-any.whl) (MD5: 1e5fb63c798a4aafe4461505521ac79a)
- Flink 1.10：[Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink_flink_1.10-1.3.0-py3-none-any.whl) (MD5: f0d35a4c3500db0e52c390ed1ab830c5)
- Flink 1.9: [Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink_flink_1.9-1.3.0-py3-none-any.whl) (MD5: 3bfbef09e5d5147d2db2aeba785f3ba6)
- Flink 1.12：[Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.1/pyalink-1.3.1-py3-none-any.whl) (MD5: a7c793b1bb38045c5d1ef4c50285562f)
- Flink 1.11：[Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.1/pyalink_flink_1.11-1.3.1-py3-none-any.whl) (MD5: f71779fb6d3afe99bab593d8c91f540f)
- Flink 1.10：[Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.1/pyalink_flink_1.10-1.3.1-py3-none-any.whl) (MD5: 4950fc5cafac27d3062a047ab2b7bb34)
- Flink 1.9: [Link](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.1/pyalink_flink_1.9-1.3.1-py3-none-any.whl) (MD5: f6071a4e9f6b41a3558ed97bb235346e)
3. If multiple version of Python exist, you may need to use a special version of `pip`, like `pip3`;
If Anaconda is used, the command should be run in Anaconda prompt.

Expand All @@ -71,6 +72,9 @@ The following dependencies and their versions of jars are supported:
- MySQL: 5.1.27
- Derby: 10.6.1.0
- SQLite: 3.19.3
- S3-hadoop: 1.11.788
- S3-presto: 1.11.788
- odps: 0.36.4-public

These jars will be installed to the ```lib/plugins``` folder of PyAlink.
Note that these command require the access for the folder.
Expand Down Expand Up @@ -149,12 +153,31 @@ Pipeline pipeline = new Pipeline().add(va).add(kMeans);
pipeline.fit(data).transform(data).print();
```

### With Flink-1.12
```xml
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.12_2.11</artifactId>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-scala_2.11</artifactId>
<version>1.12.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner_2.11</artifactId>
<version>1.12.0</version>
</dependency>
```

### With Flink-1.11
```xml
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.11_2.11</artifactId>
<version>1.3.0</version>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand All @@ -173,7 +196,7 @@ pipeline.fit(data).transform(data).print();
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.10_2.11</artifactId>
<version>1.3.0</version>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand All @@ -193,7 +216,7 @@ pipeline.fit(data).transform(data).print();
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.9_2.11</artifactId>
<version>1.3.0</version>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand All @@ -213,8 +236,8 @@ Get started to run Alink Algorithm with a Flink Cluster

1. Prepare a Flink Cluster:
```shell
wget https://archive.apache.org/dist/flink/flink-1.11.0/flink-1.11.0-bin-scala_2.11.tgz
tar -xf flink-1.11.0-bin-scala_2.11.tgz && cd flink-1.11.0
wget https://archive.apache.org/dist/flink/flink-1.12.0/flink-1.12.0-bin-scala_2.11.tgz
tar -xf flink-1.12.0-bin-scala_2.11.tgz && cd flink-1.12.0
./bin/start-cluster.sh
```

Expand Down
46 changes: 35 additions & 11 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,23 +31,24 @@
#### 包名和版本说明：

- PyAlink 根据 Alink 所支持的 Flink 版本提供不同的 Python 包：
其中，`pyalink` 包对应为 Alink 所支持的最新 Flink 版本，当前为 1.11，而 `pyalink-flink-***` 为旧版本的 Flink 版本，当前提供 `pyalink-flink-1.10` 和 `pyalink-flink-1.9`。
- Python 包的版本号与 Alink 的版本号一致，例如`1.3.0`。
其中，`pyalink` 包对应为 Alink 所支持的最新 Flink 版本，当前为 1.12，而 `pyalink-flink-***` 为旧版本的 Flink 版本，当前提供 `pyalink-flink-1.11`, `pyalink-flink-1.10` 和 `pyalink-flink-1.9`。
- Python 包的版本号与 Alink 的版本号一致，例如`1.3.1`。

####安装步骤：
1. 确保使用环境中有Python3，版本限于 3.6 和 3.7。
2. 确保使用环境中安装有 Java 8。
3. 使用 pip 命令进行安装：
`pip install pyalink`、`pip install pyalink-flink-1.10` 或者 `pip install pyalink-flink-1.9`。
`pip install pyalink`、`pip install pyalink-flink-1.11`、`pip install pyalink-flink-1.10` 或者 `pip install pyalink-flink-1.9`。

#### 安装注意事项：

1. `pyalink` 和 `pyalink-flink-***` 不能同时安装，也不能与旧版本同时安装。
如果之前安装过 `pyalink` 或者 `pyalink-flink-***`，请使用`pip uninstall pyalink` 或者 `pip uninstall pyalink-flink-***` 卸载之前的版本。
2. 出现`pip`安装缓慢或不成功的情况，可以参考[这篇文章](https://segmentfault.com/a/1190000006111096)修改pip源，或者直接使用下面的链接下载 whl 包，然后使用 `pip` 安装：
- Flink 1.11：[链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink-1.3.0-py3-none-any.whl) (MD5: 1e5fb63c798a4aafe4461505521ac79a)
- Flink 1.10：[链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink_flink_1.10-1.3.0-py3-none-any.whl) (MD5: f0d35a4c3500db0e52c390ed1ab830c5)
- Flink 1.9: [链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.0/pyalink_flink_1.9-1.3.0-py3-none-any.whl) (MD5: 3bfbef09e5d5147d2db2aeba785f3ba6)
- Flink 1.12：[链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.1/pyalink-1.3.1-py3-none-any.whl) (MD5: a7c793b1bb38045c5d1ef4c50285562f)
- Flink 1.11：[链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.1/pyalink_flink_1.11-1.3.1-py3-none-any.whl) (MD5: f71779fb6d3afe99bab593d8c91f540f)
- Flink 1.10：[链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.1/pyalink_flink_1.10-1.3.1-py3-none-any.whl) (MD5: 4950fc5cafac27d3062a047ab2b7bb34)
- Flink 1.9: [链接](https://alink-release.oss-cn-beijing.aliyuncs.com/v1.3.1/pyalink_flink_1.9-1.3.1-py3-none-any.whl) (MD5: f6071a4e9f6b41a3558ed97bb235346e)
3. 如果有多个版本的 Python，可能需要使用特定版本的 `pip`，比如 `pip3`；如果使用 Anaconda，则需要在 Anaconda 命令行中进行安装。

#### 下载安装文件系统或 Catalog 依赖 jar 包：
Expand All @@ -63,6 +64,9 @@
- MySQL: 5.1.27
- Derby: 10.6.1.0
- SQLite: 3.19.3
- S3-hadoop: 1.11.788
- S3-presto: 1.11.788
- odps: 0.36.4-public

这些 jar 包将被下载到 PyAlink 安装路径的 ```lib/plugins``` 目录下，所以要求运行命令时有 PyAlink 安装目录的权限。

Expand Down Expand Up @@ -134,12 +138,32 @@ Pipeline pipeline = new Pipeline().add(va).add(kMeans);
pipeline.fit(data).transform(data).print();
```


### Flink-1.12 的 Maven 依赖
```xml
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.12_2.11</artifactId>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-scala_2.11</artifactId>
<version>1.12.0</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner_2.11</artifactId>
<version>1.12.0</version>
</dependency>
```

### Flink-1.11 的 Maven 依赖
```xml
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.11_2.11</artifactId>
<version>1.3.0</version>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand All @@ -158,7 +182,7 @@ pipeline.fit(data).transform(data).print();
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.10_2.11</artifactId>
<version>1.3.0</version>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand All @@ -178,7 +202,7 @@ pipeline.fit(data).transform(data).print();
<dependency>
<groupId>com.alibaba.alink</groupId>
<artifactId>alink_core_flink-1.9_2.11</artifactId>
<version>1.3.0</version>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
Expand All @@ -199,8 +223,8 @@ pipeline.fit(data).transform(data).print();

1. 准备Flink集群
```shell
wget https://archive.apache.org/dist/flink/flink-1.11.0/flink-1.10.0-bin-scala_2.11.tgz
tar -xf flink-1.11.0-bin-scala_2.11.tgz && cd flink-1.11.0
wget https://archive.apache.org/dist/flink/flink-1.12.0/flink-1.12.0-bin-scala_2.11.tgz
tar -xf flink-1.12.0-bin-scala_2.11.tgz && cd flink-1.12.0
./bin/start-cluster.sh
```

Expand Down

0 comments on commit 8efd044

Please sign in to comment.