Fixes to code generators

- Moved boilerplate headers to external files - Update-codegen.sh improvements: - Add header argument to update-codegen.sh - required for k8s 1.18 - Ensure GOPATH is set - required for k8s 1.18 - Use script root instead of git root, make code-gen path relative to script root - generate-fakes improvements - Call counterfieter from GOPATH - Add header text from boilerplate - Add generate Makefile target to create client, deepcopies, fakes, and copyright. Updated travis to install counterfeiter and verify generated code. - Update docs/development/testing.md to inform contributors of the verification step in Travis.
shipwright-io · Oct 8, 2020 · 2b4cece · 2b4cece
1 parent 5b7de61
commit 2b4cece
Show file tree

Hide file tree

Showing 11 changed files with 88 additions and 89 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -9,7 +9,7 @@ install:
 script:
   - set -e
   - make
-  - make verify-copyright
+  - make verify-codegen
   # https://github.com/shipwright-io/build/issues/123
   - make test-unit-coverage
   - make test-integration

diff --git a/Makefile b/Makefile
@@ -96,12 +96,14 @@ push-image:
 release:
 	hack/release.sh
 
-.PHONY: gen-copyright
-gen-copyright:
+.PHONY: generate
+generate:
+	hack/generate-client.sh
+	hack/generate-fakes.sh
 	hack/generate-copyright.sh
 
-.PHONY: verify-copyright
-verify-copyright: gen-copyright
+.PHONY: verify-codegen
+verify-codegen: generate
 	# TODO: Fix travis issue with ginkgo install updating go.mod and go.sum
 	# TODO: Verify vendor tree is accurate
 	git diff --quiet -- ':(exclude)go.mod' ':(exclude)go.sum' ':(exclude)vendor/*'
@@ -114,6 +116,9 @@ install-ginkgo:
 install-gocov:
 	cd && GO111MODULE=on go get github.com/axw/gocov/[email protected]
 
+install-counterfeiter:
+	hack/install-counterfeiter.sh
+
 # https://github.com/shipwright-io/build/issues/123
 test: test-unit
 
@@ -201,6 +206,6 @@ kind:
 	./hack/install-kind.sh
 	./hack/install-registry.sh
 
-travis: install-ginkgo install-gocov kubectl kind
+travis: install-counterfeiter install-ginkgo install-gocov kubectl kind
 	./hack/install-tekton.sh
 	./hack/install-operator-sdk.sh
diff --git a/docs/development/testing.md b/docs/development/testing.md
@@ -8,9 +8,9 @@ SPDX-License-Identifier: Apache-2.0
 
 - [Overview](#overview)
 - [Ginkgo](#ginkgo)
-- [Unit Tests](#unit-tests)
+- [Verifying your code](#verifying-your-code)
   - [Counterfeiter](#counterfeiter)
-  - [Running unit tests](#running-unit-tests)
+- [Unit Tests](#unit-tests)
 - [Integration Tests](#integration-tests)
   - [Running integration tests](#running-integration-tests)
 - [E2E Tests](#e2e-tests)
@@ -24,13 +24,27 @@ SPDX-License-Identifier: Apache-2.0
 
 ## Overview
 
-Before opening a pull requests, please ensure that your changes are passing unit and integration tests. In the following sections, the three levels of tests we cover are explained in detail.
+Before opening a pull requests, please ensure that your changes are passing unit, integration, and verification tests.
+In the following sections, the three levels of tests we cover are explained in detail.
 Our testing implementation follows the Kubernetes community recommendations, see the [community docs](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/testing.md) for more information.
 
 ## Ginkgo
 
 For all the testing levels, we rely on Gingko as the testing framework for defining test cases and executing them. Ginkgo is a go testing framework that use the Go´s **testing** package. The framework have many [features](https://github.com/onsi/ginkgo#feature-list) on top of Go´s built-in testing primitives.
 
+## Verifying your code
+
+Our Travis builds verify that your code conforms to project standards and that all generated code is up to date.
+When adding files or updating APIs, please run `make generate` before submitting your code.
+
+### Counterfeiter
+
+Counterfeiter is used to generate and update fake implementations of objects. Currently only used for the `manager` and `client` package interface of the `sigs.k8s.io/controller-runtime`.
+
+This allow us to use test doubles in the unit tests, from where we can instantiate the fakes and then stub return values. This is very useful, for example we can mock all **client** calls that happened during the k8s controllers reconciliation and stub the result. Same case for the **manager** when creating controllers.
+
+Counterfeiter is required by the code generator scripts. Run `make install-counterfeiter` to add counterfeiter to your `GOPATH`.
+
 ## Unit Tests
 
 We use unit tests to provide coverage and ensure that our functions are behaving as expected, but also to assert the behaviour of the controllers during Reconciliations.
@@ -53,17 +67,7 @@ When building unit-tests, try to follow:
 - Assert all errors.
 - Assert that function invocations generate the expected values.
 
-### Counterfeiter
-
-Counterfeiter is used to generate and update fake implementations of objects. Currently only used for the `manager` and `client` package interface of the `sigs.k8s.io/controller-runtime`.
-
-This allow us to use test doubles in the unit tests, from where we can instantiate the fakes and then stub return values. This is very useful, for example we can mock all **client** calls that happened during the k8s controllers reconciliation and stub the result. Same case for the **manager** when creating controllers. For installing the binary, refer to [installing counterfeiter](https://github.com/maxbrunsfeld/counterfeiter#installing-counterfeiter-to-gopathbin).
-
-### Running unit tests
-
-```sh
-make test
-```
+To run the unit tests, run `make test` from the command line.
 
 ## Integration Tests
 

diff --git a/hack/boilerplate.go.txt b/hack/boilerplate.go.txt
@@ -0,0 +1,3 @@
+// Copyright The Shipwright Contributors
+//
+// SPDX-License-Identifier: Apache-2.0
diff --git a/hack/boilerplate.html.txt b/hack/boilerplate.html.txt
@@ -0,0 +1,5 @@
+<!--
+Copyright The Shipwright Contributors
+
+SPDX-License-Identifier: Apache-2.0
+-->
diff --git a/hack/boilerplate.sh.txt b/hack/boilerplate.sh.txt
@@ -0,0 +1,3 @@
+# Copyright The Shipwright Contributors
+#
+# SPDX-License-Identifier: Apache-2.0
diff --git a/hack/generate-client.sh b/hack/generate-client.sh
@@ -0,0 +1,28 @@
+#!/bin/bash
+
+# Copyright The Shipwright Contributors
+#
+# SPDX-License-Identifier: Apache-2.0
+
+#
+# Generates the typed client for Kubernetes CRDs
+# From https://www.openshift.com/blog/kubernetes-deep-dive-code-generation-customresources
+#
+
+set -euo pipefail
+
+GOPATH=${GOPATH:-$(go env GOPATH)}
+SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
+CODEGEN_PKG=${CODEGEN_PKG:-$(cd ${SCRIPT_ROOT}; ls -d -1 ./vendor/k8s.io/code-generator 2>/dev/null || echo ../../../k8s.io/code-generator)}
+
+echo ""
+echo "Using code-generator package version, as instructed in the go.mod file"
+echo "The code-generator package is imported via the pkg/kubecodegen dir"
+echo "To modify the current version, please modify this in the go.mod"
+echo ""
+
+GOFLAGS="" GOPATH=${GOPATH} /bin/bash ${CODEGEN_PKG}/generate-groups.sh "deepcopy,client" \
+  github.com/shipwright-io/build/pkg/client/build \
+  github.com/shipwright-io/build/pkg/apis \
+  build:v1alpha1 \
+  --go-header-file "${SCRIPT_ROOT}/hack/boilerplate.go.txt"
diff --git a/hack/generate-copyright.sh b/hack/generate-copyright.sh
@@ -6,8 +6,7 @@
 
 set -e
 
-copyrightTxt="Copyright The Shipwright Contributors"
-spdxTxt="SPDX-License-Identifier: Apache-2.0"
+SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
 
 function listPkgDirs() {
 	go list -f '{{.Dir}}' ./cmd/... ./pkg/... ./test/... ./version/...
@@ -18,7 +17,7 @@ function listGoFiles() {
 	# pipeline is much faster than for loop
 	listPkgDirs | xargs -I {} find {} \( -name '*.go' -a ! -name "zz_generated*.go" \)
   local goFiles=$?
-  echo $PWD/tools.go
+  echo "${SCRIPT_ROOT}/tools.go"
   goFiles="$goFiles $?"
 }
 
@@ -43,10 +42,7 @@ function generateGoCopyright() {
   for file in $allFiles ; do
     if ! head -n3 "${file}" | grep -Eq "(Copyright|SPDX-License-Identifier)" ; then
       cp "${file}" "${file}.bak"
-      echo "// ${copyrightTxt}" > "${file}"
-      echo "//" >> "${file}"
-      echo "// ${spdxTxt}" >> "${file}"
-      echo "" >> "${file}"
+      cat "${SCRIPT_ROOT}/hack/boilerplate.go.txt" > "${file}"
       cat "${file}.bak" >> "${file}"
       rm "${file}.bak"
     fi
@@ -58,10 +54,7 @@ function generateDockerfileCopyright() {
   for file in $dockerfiles ; do
     if ! head -n3 "${file}" | grep -Eq "(Copyright|SPDX-License-Identifier)" ; then
       cp "${file}" "${file}.bak"
-      echo "# ${copyrightTxt}" > "${file}"
-      echo "# " >> "${file}"
-      echo "# ${spdxTxt}" >> "${file}"
-      echo "" >> "${file}"
+      cat "${SCRIPT_ROOT}/hack/boilerplate.sh.txt" > "${file}"
       cat "${file}.bak" >> "${file}"
       rm "${file}.bak"
     fi
@@ -75,12 +68,10 @@ function generateBashCopyright() {
       cp "${file}" "${file}.bak"
       # Copy the shebang first - this is assumed to be the first line
       head -n1 "${file}.bak" > "${file}"
-      echo "" >> "${file}"
-      echo "# ${copyrightTxt}" >> "${file}"
-      echo "#" >> "${file}"
-      echo "# ${spdxTxt}" >> "${file}"
-      echo "" >> "${file}"
-      tail -n +2 "${file}.bak" >> "${file}"
+      {
+        cat "${SCRIPT_ROOT}/hack/boilerplate.sh.txt"
+        tail -n +2 "${file}.bak"
+      } >> "${file}"
       rm "${file}.bak"
     fi
   done
@@ -91,12 +82,7 @@ function generateMarkdownCopyright() {
   for file in $mdFiles ; do
     if ! head -n4 "${file}" | grep -Eq "(Copyright|SPDX-License-Identifier)" ; then
       cp "${file}" "${file}.bak"
-      echo "<!--" > "${file}"
-      echo "${copyrightTxt}" >> "${file}"
-      echo "" >> "${file}"
-      echo "${spdxTxt}" >> "${file}"
-      echo "-->" >> "${file}"
-      echo "" >> "${file}"
+      cat "${SCRIPT_ROOT}/hack/boilerplate.html.txt" > "${file}"
       cat "${file}.bak" >> "${file}"
       rm "${file}.bak"
     fi

diff --git a/hack/generate-fakes.sh b/hack/generate-fakes.sh
@@ -9,6 +9,9 @@ set -euo pipefail
 
 [ ! -d "vendor" ] && echo "$0 requires vendor/ folder, run 'go mod vendor'"
 
-counterfeiter -o pkg/controller/fakes/manager.go vendor/sigs.k8s.io/controller-runtime/pkg/manager Manager
-counterfeiter -o pkg/controller/fakes/client.go vendor/sigs.k8s.io/controller-runtime/pkg/client Client
-counterfeiter -o pkg/controller/fakes/status_writer.go vendor/sigs.k8s.io/controller-runtime/pkg/client StatusWriter
+SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
+BIN=$(go env GOPATH)/bin
+
+"${BIN}/counterfeiter" -header "${SCRIPT_ROOT}/hack/boilerplate.go.txt" -o pkg/controller/fakes/manager.go vendor/sigs.k8s.io/controller-runtime/pkg/manager Manager
+"${BIN}/counterfeiter" -header "${SCRIPT_ROOT}/hack/boilerplate.go.txt" -o pkg/controller/fakes/client.go vendor/sigs.k8s.io/controller-runtime/pkg/client Client
+"${BIN}/counterfeiter" -header "${SCRIPT_ROOT}/hack/boilerplate.go.txt" -o pkg/controller/fakes/status_writer.go vendor/sigs.k8s.io/controller-runtime/pkg/client StatusWriter
diff --git a/hack/install-counterfeiter.sh b/hack/install-counterfeiter.sh
@@ -0,0 +1,5 @@
+#!/bin/bash
+
+set -eu
+
+GO111MODULE=off go get -u github.com/maxbrunsfeld/counterfeiter
diff --git a/hack/update-codegen.sh b/hack/update-codegen.sh