Skip to content
This repository has been archived by the owner on Nov 22, 2024. It is now read-only.

Commit

Permalink
update flink to 1.18.1 (#173)
Browse files Browse the repository at this point in the history
* update flink to 1.18.1
* upgrade flink-connector-kafka to  3.1.0-1.18
  • Loading branch information
fieldera-da authored Feb 21, 2024
1 parent 66341bb commit 050a837
Show file tree
Hide file tree
Showing 5 changed files with 10 additions and 9 deletions.
3 changes: 2 additions & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ lazy val flinkTests =
crossScalaVersions := Vector(Dependencies.Scala212),
scalafmtOnCompile := true,
(Test / sourceGenerators) += (Test / avroScalaGenerateSpecific).taskValue,
Test / parallelExecution := false)
Test / parallelExecution := false,
Test / fork := true)

lazy val flinkSbtPlugin =
Project(id = "cloudflow-sbt-flink", base = file("cloudflow-sbt-flink"))
Expand Down
8 changes: 4 additions & 4 deletions cloudflow-it/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,10 @@ prepare-clis:
mv spark-3.3.4-bin-hadoop2 spark
rm spark-3.3.4-bin-hadoop2.tgz
rm -rf flink
wget https://archive.apache.org/dist/flink/flink-1.17.2/flink-1.17.2-bin-scala_2.12.tgz
tar -xf flink-1.17.2-bin-scala_2.12.tgz
mv flink-1.17.2 flink
rm flink-1.17.2-bin-scala_2.12.tgz
wget https://archive.apache.org/dist/flink/flink-1.18.1/flink-1.18.1-bin-scala_2.12.tgz
tar -xf flink-1.18.1-bin-scala_2.12.tgz
mv flink-1.18.1 flink
rm flink-1.18.1-bin-scala_2.12.tgz

.PHONY: prepare-cluster
prepare-cluster:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ import cloudflow.sbt.CloudflowBasePlugin._

object CloudflowNativeFlinkPlugin extends AutoPlugin {
val FlinkHome = "/opt/flink"
val FlinkVersion = "1.17.2"
val FlinkVersion = "1.18.1"
val FlinkUsrLib = s"$FlinkHome/usrlib"

val AppJarsDir: String = "app-jars"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ The following scripts are expected to be present on the `PATH`:
- bash
- jq (minimum version required is > 1.5, because of https://github.com/stedolan/jq/issues/1408[this] issue)
- kubectl
- flink cli (version 1.17.2)
- flink cli (version 1.18.1)

In the `flink` sub-folder the first script available is `setup-example-rbac.sh` and this first step needs to be performed once on any cluster you want to deploy Flink streamlets, refer to https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#rbac[the upstream documentation] for further details. This script will setup service account and cluster role binding. Take a note of the service account name that it prints in stdout - you will need it to run subsequent scripts.

Expand Down
4 changes: 2 additions & 2 deletions project/Dependencies.scala
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ object Dependencies {
object Versions {
val cloudflowVersion = "2.3.2"

val flinkVersion = "1.17.2"
val flinkKafkaVersion = "3.0.2-1.17"
val flinkVersion = "1.18.1"
val flinkKafkaVersion = "3.1.0-1.18"
val sparkVersion = "3.3.4"
val akka = "2.6.19"
val jackson = "2.12.6"
Expand Down

0 comments on commit 050a837

Please sign in to comment.