Skip to content

Commit

Permalink
Merge pull request #435 from adambkaplan/codgegen-boilerplate
Browse files Browse the repository at this point in the history
Fixes to code generators
  • Loading branch information
openshift-merge-robot authored Oct 15, 2020
2 parents 10f42da + 797e7d1 commit 5d89593
Show file tree
Hide file tree
Showing 35 changed files with 154 additions and 407 deletions.
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ install:
script:
- set -e
- make
- make verify-copyright
- make verify-codegen
# https://github.com/shipwright-io/build/issues/123
- make test-unit-coverage
- make test-integration
Expand Down
15 changes: 10 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -96,12 +96,14 @@ push-image:
release:
hack/release.sh

.PHONY: gen-copyright
gen-copyright:
.PHONY: generate
generate:
hack/generate-client.sh
hack/generate-fakes.sh
hack/generate-copyright.sh

.PHONY: verify-copyright
verify-copyright: gen-copyright
.PHONY: verify-codegen
verify-codegen: generate
# TODO: Fix travis issue with ginkgo install updating go.mod and go.sum
# TODO: Verify vendor tree is accurate
git diff --quiet -- ':(exclude)go.mod' ':(exclude)go.sum' ':(exclude)vendor/*'
Expand All @@ -114,6 +116,9 @@ install-ginkgo:
install-gocov:
cd && GO111MODULE=on go get github.com/axw/gocov/[email protected]

install-counterfeiter:
hack/install-counterfeiter.sh

# https://github.com/shipwright-io/build/issues/123
test: test-unit

Expand Down Expand Up @@ -207,6 +212,6 @@ kind:
./hack/install-kind.sh
./hack/install-registry.sh

travis: install-ginkgo install-gocov kubectl kind
travis: install-counterfeiter install-ginkgo install-gocov kubectl kind
./hack/install-tekton.sh
./hack/install-operator-sdk.sh
32 changes: 18 additions & 14 deletions docs/development/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ SPDX-License-Identifier: Apache-2.0

- [Overview](#overview)
- [Ginkgo](#ginkgo)
- [Unit Tests](#unit-tests)
- [Verifying your code](#verifying-your-code)
- [Counterfeiter](#counterfeiter)
- [Running unit tests](#running-unit-tests)
- [Unit Tests](#unit-tests)
- [Integration Tests](#integration-tests)
- [Running integration tests](#running-integration-tests)
- [E2E Tests](#e2e-tests)
Expand All @@ -24,13 +24,27 @@ SPDX-License-Identifier: Apache-2.0

## Overview

Before opening a pull requests, please ensure that your changes are passing unit and integration tests. In the following sections, the three levels of tests we cover are explained in detail.
Before opening a pull requests, please ensure that your changes are passing unit, integration, and verification tests.
In the following sections, the three levels of tests we cover are explained in detail.
Our testing implementation follows the Kubernetes community recommendations, see the [community docs](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/testing.md) for more information.

## Ginkgo

For all the testing levels, we rely on Gingko as the testing framework for defining test cases and executing them. Ginkgo is a go testing framework that use the Go´s **testing** package. The framework have many [features](https://github.com/onsi/ginkgo#feature-list) on top of Go´s built-in testing primitives.

## Verifying your code

Our Travis builds verify that your code conforms to project standards and that all generated code is up to date.
When adding files or updating APIs, please run `make generate` before submitting your code.

### Counterfeiter

Counterfeiter is used to generate and update fake implementations of objects. Currently only used for the `manager` and `client` package interface of the `sigs.k8s.io/controller-runtime`.

This allow us to use test doubles in the unit tests, from where we can instantiate the fakes and then stub return values. This is very useful, for example we can mock all **client** calls that happened during the k8s controllers reconciliation and stub the result. Same case for the **manager** when creating controllers.

Counterfeiter is required by the code generator scripts. Run `make install-counterfeiter` to add counterfeiter to your `GOPATH`.

## Unit Tests

We use unit tests to provide coverage and ensure that our functions are behaving as expected, but also to assert the behaviour of the controllers during Reconciliations.
Expand All @@ -53,17 +67,7 @@ When building unit-tests, try to follow:
- Assert all errors.
- Assert that function invocations generate the expected values.

### Counterfeiter

Counterfeiter is used to generate and update fake implementations of objects. Currently only used for the `manager` and `client` package interface of the `sigs.k8s.io/controller-runtime`.

This allow us to use test doubles in the unit tests, from where we can instantiate the fakes and then stub return values. This is very useful, for example we can mock all **client** calls that happened during the k8s controllers reconciliation and stub the result. Same case for the **manager** when creating controllers. For installing the binary, refer to [installing counterfeiter](https://github.com/maxbrunsfeld/counterfeiter#installing-counterfeiter-to-gopathbin).

### Running unit tests

```sh
make test
```
To run the unit tests, run `make test` from the command line.

## Integration Tests

Expand Down
3 changes: 3 additions & 0 deletions hack/boilerplate.go.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
// Copyright The Shipwright Contributors
//
// SPDX-License-Identifier: Apache-2.0
5 changes: 5 additions & 0 deletions hack/boilerplate.html.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
<!--
Copyright The Shipwright Contributors

SPDX-License-Identifier: Apache-2.0
-->
3 changes: 3 additions & 0 deletions hack/boilerplate.sh.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Copyright The Shipwright Contributors
#
# SPDX-License-Identifier: Apache-2.0
28 changes: 28 additions & 0 deletions hack/generate-client.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/bin/bash

# Copyright The Shipwright Contributors
#
# SPDX-License-Identifier: Apache-2.0

#
# Generates the typed client for Kubernetes CRDs
# From https://www.openshift.com/blog/kubernetes-deep-dive-code-generation-customresources
#

set -euo pipefail

GOPATH=${GOPATH:-$(go env GOPATH)}
SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
CODEGEN_PKG=${CODEGEN_PKG:-$(cd ${SCRIPT_ROOT}; ls -d -1 ./vendor/k8s.io/code-generator 2>/dev/null || echo ../../../k8s.io/code-generator)}

echo ""
echo "Using code-generator package version, as instructed in the go.mod file"
echo "The code-generator package is imported via the pkg/kubecodegen dir"
echo "To modify the current version, please modify this in the go.mod"
echo ""

GOFLAGS="" GOPATH=${GOPATH} /bin/bash ${CODEGEN_PKG}/generate-groups.sh "deepcopy,client" \
github.com/shipwright-io/build/pkg/client/build \
github.com/shipwright-io/build/pkg/apis \
build:v1alpha1 \
--go-header-file "${SCRIPT_ROOT}/hack/boilerplate.go.txt"
32 changes: 9 additions & 23 deletions hack/generate-copyright.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@

set -e

copyrightTxt="Copyright The Shipwright Contributors"
spdxTxt="SPDX-License-Identifier: Apache-2.0"
SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..

function listPkgDirs() {
go list -f '{{.Dir}}' ./cmd/... ./pkg/... ./test/... ./version/...
Expand All @@ -18,7 +17,7 @@ function listGoFiles() {
# pipeline is much faster than for loop
listPkgDirs | xargs -I {} find {} \( -name '*.go' -a ! -name "zz_generated*.go" \)
local goFiles=$?
echo $PWD/tools.go
echo "${SCRIPT_ROOT}/tools.go"
goFiles="$goFiles $?"
}

Expand All @@ -43,10 +42,7 @@ function generateGoCopyright() {
for file in $allFiles ; do
if ! head -n3 "${file}" | grep -Eq "(Copyright|SPDX-License-Identifier)" ; then
cp "${file}" "${file}.bak"
echo "// ${copyrightTxt}" > "${file}"
echo "//" >> "${file}"
echo "// ${spdxTxt}" >> "${file}"
echo "" >> "${file}"
cat "${SCRIPT_ROOT}/hack/boilerplate.go.txt" > "${file}"
cat "${file}.bak" >> "${file}"
rm "${file}.bak"
fi
Expand All @@ -58,10 +54,7 @@ function generateDockerfileCopyright() {
for file in $dockerfiles ; do
if ! head -n3 "${file}" | grep -Eq "(Copyright|SPDX-License-Identifier)" ; then
cp "${file}" "${file}.bak"
echo "# ${copyrightTxt}" > "${file}"
echo "# " >> "${file}"
echo "# ${spdxTxt}" >> "${file}"
echo "" >> "${file}"
cat "${SCRIPT_ROOT}/hack/boilerplate.sh.txt" > "${file}"
cat "${file}.bak" >> "${file}"
rm "${file}.bak"
fi
Expand All @@ -75,12 +68,10 @@ function generateBashCopyright() {
cp "${file}" "${file}.bak"
# Copy the shebang first - this is assumed to be the first line
head -n1 "${file}.bak" > "${file}"
echo "" >> "${file}"
echo "# ${copyrightTxt}" >> "${file}"
echo "#" >> "${file}"
echo "# ${spdxTxt}" >> "${file}"
echo "" >> "${file}"
tail -n +2 "${file}.bak" >> "${file}"
{
cat "${SCRIPT_ROOT}/hack/boilerplate.sh.txt"
tail -n +2 "${file}.bak"
} >> "${file}"
rm "${file}.bak"
fi
done
Expand All @@ -91,12 +82,7 @@ function generateMarkdownCopyright() {
for file in $mdFiles ; do
if ! head -n4 "${file}" | grep -Eq "(Copyright|SPDX-License-Identifier)" ; then
cp "${file}" "${file}.bak"
echo "<!--" > "${file}"
echo "${copyrightTxt}" >> "${file}"
echo "" >> "${file}"
echo "${spdxTxt}" >> "${file}"
echo "-->" >> "${file}"
echo "" >> "${file}"
cat "${SCRIPT_ROOT}/hack/boilerplate.html.txt" > "${file}"
cat "${file}.bak" >> "${file}"
rm "${file}.bak"
fi
Expand Down
9 changes: 6 additions & 3 deletions hack/generate-fakes.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ set -euo pipefail

[ ! -d "vendor" ] && echo "$0 requires vendor/ folder, run 'go mod vendor'"

counterfeiter -o pkg/controller/fakes/manager.go vendor/sigs.k8s.io/controller-runtime/pkg/manager Manager
counterfeiter -o pkg/controller/fakes/client.go vendor/sigs.k8s.io/controller-runtime/pkg/client Client
counterfeiter -o pkg/controller/fakes/status_writer.go vendor/sigs.k8s.io/controller-runtime/pkg/client StatusWriter
SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
BIN=$(go env GOPATH)/bin

"${BIN}/counterfeiter" -header "${SCRIPT_ROOT}/hack/boilerplate.go.txt" -o pkg/controller/fakes/manager.go vendor/sigs.k8s.io/controller-runtime/pkg/manager Manager
"${BIN}/counterfeiter" -header "${SCRIPT_ROOT}/hack/boilerplate.go.txt" -o pkg/controller/fakes/client.go vendor/sigs.k8s.io/controller-runtime/pkg/client Client
"${BIN}/counterfeiter" -header "${SCRIPT_ROOT}/hack/boilerplate.go.txt" -o pkg/controller/fakes/status_writer.go vendor/sigs.k8s.io/controller-runtime/pkg/client StatusWriter
8 changes: 8 additions & 0 deletions hack/install-counterfeiter.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash
# Copyright The Shipwright Contributors
#
# SPDX-License-Identifier: Apache-2.0

set -eu

GO111MODULE=off go get -u github.com/maxbrunsfeld/counterfeiter
43 changes: 0 additions & 43 deletions hack/update-codegen.sh

This file was deleted.

18 changes: 3 additions & 15 deletions pkg/apis/build/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 3 additions & 15 deletions pkg/client/build/clientset/versioned/clientset.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 3 additions & 15 deletions pkg/client/build/clientset/versioned/doc.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 3 additions & 15 deletions pkg/client/build/clientset/versioned/fake/clientset_generated.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 5d89593

Please sign in to comment.