Skip to content

Latest commit

 

History

History
74 lines (50 loc) · 2.84 KB

developer-guide.md

File metadata and controls

74 lines (50 loc) · 2.84 KB
Raw

Developer Guide

Build the Operator

In case you want to build the operator from the source code, e.g., to test a fix or a feature you write, you can do so following the instructions below.

The easiest way to build without worrying about dependencies is to just build the Dockerfile.

$ docker build -t <image-tag> .

The operator image is built upon a base Spark image that defaults to gcr.io/spark-operator/spark:v2.3.1. If you want to use your own Spark image (e.g., an image with a different version of Spark or some custom dependencies), specify the argument SPARK_IMAGE as the following example shows:

$ docker build --build-arg SPARK_IMAGE=<your Spark image> -t <image-tag> .

If you'd like to build/test the spark-operator locally, follow the instructions below:

$ mkdir -p $GOPATH/src/github.com/GoogleCloudPlatform
$ cd $GOPATH/src/github.com/GoogleCloudPlatform
$ git clone [email protected]:GoogleCloudPlatform/spark-on-k8s-operator.git
$ cd spark-on-k8s-operator

The operator uses dep for dependency management. Please install dep following the instruction on the website if you don't have it available locally. To install the dependencies, run the following command:

$ dep ensure

To update the dependencies, run the following command. (You can skip this unless you know there's a dependency that needs updating):

$ dep ensure -update

Before building the operator the first time, run the following commands to get the required Kubernetes code generators:

$ go get -u k8s.io/code-generator/cmd/client-gen
$ go get -u k8s.io/code-generator/cmd/deepcopy-gen
$ go get -u k8s.io/code-generator/cmd/defaulter-gen

To update the auto-generated code, run the following command. (This step is only required if the CRD types have been changed):

$ go generate

Due to an issue with Kubernetes code generation that lower-cases generated import paths, some of the client code files generated by running go generate have import paths starting with the lower-case prefix github.com/googlecloudplatform, which is inconsistent with the camel case prefix github.com/GoogleCloudPlatform in other files. This will cause go build to fail. This PR fixes the issue, but the fix won't be available until its merged and released in a future Kubernetes version. As a workaround, revert the lower-casing changes made by the code generator under pkg/client/clientset/versioned.

You can verify the current auto-generated code is up to date with:

$ hack/verify-codegen.sh

To build the operator, run the following command:

$ GOOS=linux go build -o spark-operator

To run unit tests, run the following command:

$ go test ./...