From bbb40457cfa57fb57ba30c7f1ebac3974eaf5b57 Mon Sep 17 00:00:00 2001 From: Rolfe Dlugy-Hegwer Date: Mon, 9 May 2022 10:24:59 -0400 Subject: [PATCH] RHDEVDOCS-3863 Prep two Shipwright docs for downstream --- docs/README.adoc | 34 ++ docs/build.adoc | 660 ++++++++++++++++++++++++++ docs/buildrun.adoc | 596 +++++++++++++++++++++++ docs/buildstrategies.adoc | 965 ++++++++++++++++++++++++++++++++++++++ docs/configuration.adoc | 86 ++++ docs/metrics.adoc | 161 +++++++ docs/profiling.adoc | 46 ++ 7 files changed, 2548 insertions(+) create mode 100644 docs/README.adoc create mode 100644 docs/build.adoc create mode 100644 docs/buildrun.adoc create mode 100644 docs/buildstrategies.adoc create mode 100644 docs/configuration.adoc create mode 100644 docs/metrics.adoc create mode 100644 docs/profiling.adoc diff --git a/docs/README.adoc b/docs/README.adoc new file mode 100644 index 0000000000..7ac212f8cd --- /dev/null +++ b/docs/README.adoc @@ -0,0 +1,34 @@ +//// +Copyright The Shipwright Contributors + +SPDX-License-Identifier: Apache-2.0 +//// += Build Controllers + +Build or codenamed *build-v2* is an API open-source implementation that build container-images on Kubernetes from a _dockerfile-based_ or a _source-based_ approach. + +The whole idea of *Build* is to hide the details of image construction from an application developer, by defining https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/[Custom Resources] that the *Build* understands. + +Through the *Build* CRD´s the user can specify a desired popular strategy, like *source-to-image*, *buildpack-v3*, *kaniko*, *jib* and *buildah*, in order to build an image. + +From a high-level perspective: + +* xref:build.adoc[`Build`] hosts the user provide information. This defines the strategy, source input and the desired output(_e.g. container registry_). +* xref:buildrun.adoc[`BuildRun`] hosts the details of an image construction, abstracting this from the user and taking advantage of the Tekton Pipelines task to build the image. +* xref:buildstrategies.adoc[`BuildStrategy`] hosts a list of steps to execute in the Tekton Task definition during the *BuildRun* execution. +* xref:buildstrategies.adoc[`ClusterBuildStrategy`] similar to the *BuildStrategy* but it is _cluster-scoped_. + +== Learn more + +See the following docs referencing each of the Kubernetes resources currently supported: + +* xref:build.adoc[`Build`] +* xref:buildrun.adoc[`BuildRun`] +* xref:buildstrategies.adoc[`BuildStrategy`] +* xref:buildstrategies.adoc[`ClusterBuildStrategy`] + +== Controllers Flow + +The following image illustrate the interactions between the `Build`, `BuildRun` controller and the Tekton `Pipeline` controller. + +image::controllers_flow.png[controllers flow] diff --git a/docs/build.adoc b/docs/build.adoc new file mode 100644 index 0000000000..b2516d76c4 --- /dev/null +++ b/docs/build.adoc @@ -0,0 +1,660 @@ +//// +Copyright The Shipwright Contributors + +SPDX-License-Identifier: Apache-2.0 +//// += Build + +* <> +* <> +* <> +* <> + ** <> + ** <> + ** <> + ** <> + ** <> + ** <> +* <> + +== Overview + +A `Build` resource allows the user to define: + +* source +* sources +* strategy +* params +* builder +* dockerfile +* output +* env +* retention + +A `Build` is available within a namespace. + +== Build Controller + +The controller watches for: + +* Updates on the `Build` resource (_CRD instance_) + +When the controller reconciles it: + +* Validates if the referenced `StrategyRef` exists. +* Validates if the specified `params` exist on the referenced strategy parameters. It also validates if the `params` names collide with the Shipwright reserved names. +* Validates if the container `registry` output secret exists. +* Validates if the referenced `spec.source.url` endpoint exists. + +== Build Validations + +To prevent users from triggering `BuildRuns` (_execution of a Build_) that will eventually fail because of wrong or missing dependencies or configuration settings, the Build controller will validate them in advance. If all validations are successful, users can expect a `Succeeded` `status.reason`. However, if any validations fail, users can rely on the `status.reason` and `status.message` fields to understand the root cause. + +|=== +| Status.Reason | Description + +| BuildStrategyNotFound +| The referenced namespace-scope strategy doesn't exist. + +| ClusterBuildStrategyNotFound +| The referenced cluster-scope strategy doesn't exist. + +| SetOwnerReferenceFailed +| Setting ownerreferences between a Build and a BuildRun failed. This status is triggered when using the `build.shipwright.io/build-run-deletion` annotation in a Build. + +| SpecSourceSecretRefNotFound +| The secret used to authenticate to git doesn't exist. + +| SpecOutputSecretRefNotFound +| The secret used to authenticate to the container registry doesn't exist. + +| SpecBuilderSecretRefNotFound +| The secret used to authenticate the container registry doesn't exist. + +| MultipleSecretRefNotFound +| More than one secret is missing. At the moment, only three paths on a Build can specify a secret. + +| RestrictedParametersInUse +| One or many defined `params` are colliding with Shipwright reserved parameters. See <> for more information. + +| UndefinedParameter +| One or many defined `params` are not defined in the referenced strategy. Please ensure that the strategy defines them under its `spec.parameters` list. + +| RemoteRepositoryUnreachable +| The defined `spec.source.url` was not found. This validation only takes place for HTTP/HTTPS protocols. + +| BuildNameInvalid +| The defined `Build` name (`metadata.name`) is invalid. The `Build` name should be a https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set[valid label value]. + +| SpecEnvNameCanNotBeBlank +| Indicates that the name for a user-provided environment variable is blank. + +| SpecEnvValueCanNotBeBlank +| Indicates that the value for a user-provided environment variable is blank. +|=== + +== Configuring a Build + +The `Build` definition supports the following fields: + +* Required: + ** https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/#required-fields[`apiVersion`] - Specifies the API version, for example `shipwright.io/v1alpha1`. + ** https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/#required-fields[`kind`] - Specifies the Kind type, for example `Build`. + ** https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/#required-fields[`metadata`] - Metadata that identify the CRD instance, for example the name of the `Build`. + ** `spec.source` - Refers to the location of the source code, for example a Git repository or source bundle image. + ** `spec.strategy` - Refers to the `BuildStrategy` to be used, see the link:../samples/buildstrategy[examples] + ** `spec.builder.image` - Refers to the image containing the build tools to build the source code. (_Use this path for Dockerless strategies, this is just required for `source-to-image` buildStrategy_) + ** `spec.output`- Refers to the location where the generated image would be pushed. + ** `spec.output.credentials.name`- Reference an existing secret to get access to the container registry. +* Optional: + ** `spec.paramValues` - Refers to a name-value(s) list to specify values for `parameters` defined in the `BuildStrategy`. + ** `spec.dockerfile` - Path to a Dockerfile to be used for building an image. (_Use this path for strategies that require a Dockerfile_) + ** `spec.sources` - <> describes a slice of artifacts that will be imported into the project context before the actual build process starts. + ** `spec.timeout` - Defines a custom timeout. The value needs to be parsable by https://golang.org/pkg/time/#ParseDuration[ParseDuration], for example, `5m`. The default is ten minutes. You can overwrite the value in the `BuildRun`. + ** `metadata.annotations[build.shipwright.io/build-run-deletion]` - Defines if delete all related BuildRuns when deleting the Build. The default is `false`. + ** `spec.output.annotations` - Refers to a list of `key/value` that could be used to https://github.com/opencontainers/image-spec/blob/main/annotations.md[annotate] the output image. + ** `spec.output.labels` - Refers to a list of `key/value` that could be used to label the output image. + ** `spec.env` - Specifies additional environment variables that should be passed to the build container. The available variables depend on the tool that is being used by the chosen build strategy. + ** `spec.retention.ttlAfterFailed` - Specifies the duration for which a failed buildrun can exist. + ** `spec.retention.ttlAfterSucceeded` - Specifies the duration for which a successful buildrun can exist. + ** `spec.retention.failedLimit` - Specifies the number of failed buildrun that can exist. + ** `spec.retention.succeededLimit` - Specifies the number of successful buildrun can exist. + +=== Defining the Source + +A `Build` resource can specify a Git repository or bundle image source, together with other parameters like: + +* `source.url` - Specify the source location using a Git repository. +* `source.bundleContainer.image` - Specify a source bundle container image to be used as the source. +* `source.bundleContainer.prune` - Configure whether the source bundle image should be deleted after the source was obtained (defaults to `Never`, other option is `AfterPull` to delete the image after a successful image pull). +* `source.credentials.name` - For private repositories or registries, the name references a secret in the namespace that contains the SSH private key or Docker access credentials, respectively. +* `source.revision` - A specific revision to select from the source repository, this can be a commit, tag or branch name. If not defined, it will fallback to the Git repository default branch. +* `source.contextDir` - For repositories where the source code is not located at the root folder, you can specify this path here. + +By default, the Build controller does not validate that the Git repository exists. If the validation is desired, users can explicitly define the `build.shipwright.io/verify.repository` annotation with `true`. For example: + +Example of a `Build` with the *build.shipwright.io/verify.repository* annotation to enable the `spec.source.url` validation. + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: buildah-golang-build + annotations: + build.shipwright.io/verify.repository: "true" +spec: + source: + url: https://github.com/shipwright-io/sample-go + contextDir: docker-build +---- + +NOTE: The Build controller only validates two scenarios. The first one is when the endpoint uses an `http/https` protocol. The second one is when an `ssh` protocol such as `git@` has been defined but a referenced secret, such as `source.credentials.name`, has not been provided. + +Example of a `Build` with a source with *credentials* defined by the user. + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: buildpack-nodejs-build +spec: + source: + url: https://github.com/sclorg/nodejs-ex + credentials: + name: source-repository-credentials +---- + +Example of a `Build` with a source that specifies a specific subfolder on the repository. + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: buildah-custom-context-dockerfile +spec: + source: + url: https://github.com/SaschaSchwarze0/npm-simple + contextDir: renamed +---- + +Example of a `Build` that specifies the tag `v.0.1.0` for the git repository: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: buildah-golang-build +spec: + source: + url: https://github.com/shipwright-io/sample-go + contextDir: docker-build + revision: v0.1.0 +---- + +Example of a `Build` that specifies environment variables: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: buildah-golang-build +spec: + source: + url: https://github.com/shipwright-io/sample-go + contextDir: docker-build + env: + - name: EXAMPLE_VAR_1 + value: "example-value-1" + - name: EXAMPLE_VAR_2 + value: "example-value-2" +---- + +Example of a `Build` that uses the Kubernetes Downward API to +expose a `Pod` field as an environment variable: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: buildah-golang-build +spec: + source: + url: https://github.com/shipwright-io/sample-go + contextDir: docker-build + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name +---- + +Example of a `Build` that uses the Kubernetes Downward API to +expose a `Container` field as an environment variable: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: buildah-golang-build +spec: + source: + url: https://github.com/shipwright-io/sample-go + contextDir: docker-build + env: + - name: MEMORY_LIMIT + valueFrom: + resourceFieldRef: + containerName: my-container + resource: limits.memory +---- + +=== Defining the Strategy + +A `Build` resource can specify the `BuildStrategy` to use, these are: + +* link:buildstrategies.md#buildah[Buildah] +* link:buildstrategies.md#buildpacks-v3[Buildpacks-v3] +* link:buildstrategies.md#buildkit[BuildKit] +* link:buildstrategies.md#kaniko[Kaniko] +* link:buildstrategies.md#ko[ko] +* link:buildstrategies.md#source-to-image[Source-to-Image] + +Defining the strategy is straightforward. You define the `name` and the `kind`. For example: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: buildpack-nodejs-build +spec: + strategy: + name: buildpacks-v3 + kind: ClusterBuildStrategy +---- + +=== Defining ParamValues + +A `Build` resource can specify _paramValues_ for parameters that are defined in the referenced `BuildStrategy`. You specify these parameter values to control how the steps of the build strategy behave. You can overwrite values in the `BuildRun` resource. See the related link:./buildrun.md#defining-params[documentation] for more information. + +The build strategy author can define a parameter as either a simple string or an array. Depending on that, you must specify the value accordingly. The build strategy parameter can be specified with a default value. You must specify a value in the `Build` or `BuildRun` for parameters without a default. + +You can either specify values directly or reference keys from https://kubernetes.io/docs/concepts/configuration/configmap/[ConfigMaps] and https://kubernetes.io/docs/concepts/configuration/secret/[Secrets]. *Note*: the usage of ConfigMaps and Secrets is limited by the usage of the parameter in the build strategy steps. You can only use them if the parameter is used in the command, arguments, or environment variable values. + +When using _paramValues_, users should avoid: + +* Defining a `spec.paramValues` name that doesn't match one of the `spec.parameters` defined in the `BuildStrategy`. +* Defining a `spec.paramValues` name that collides with the Shipwright reserved parameters. These are _BUILDER_IMAGE_, _DOCKERFILE_, _CONTEXT_DIR_, and any name starting with _shp-_. + +In general, _paramValues_ are tightly bound to Strategy _parameters_. Please make sure you understand the contents of your strategy of choice before defining _paramValues_ in the _Build_. + +==== Example + +The link:../samples/buildstrategy/buildkit/buildstrategy_buildkit_cr.yaml[BuildKit sample `BuildStrategy`] contains various parameters. Two of them are outlined here: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: ClusterBuildStrategy +metadata: + name: buildkit + ... +spec: + parameters: + - name: build-args + description: "The ARG values in the Dockerfile. Values must be in the format KEY=VALUE." + type: array + defaults: [] + - name: cache + description: "Configure BuildKit's cache usage. Allowed values are 'disabled' and 'registry'. The default is 'registry'." + type: string + default: registry + ... + buildSteps: + ... +---- + +The `cache` parameter is a simple string. You can provide it like this in your Build: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: a-build + namespace: a-namespace +spec: + paramValues: + - name: cache + value: disabled + strategy: + name: buildkit + kind: ClusterBuildStrategy + source: + ... + output: + ... +---- + +If you have multiple Builds and want to control this parameter centrally, then you can create a ConfigMap: + +[,yaml] +---- +apiVersion: v1 +kind: ConfigMap +metadata: + name: buildkit-configuration + namespace: a-namespace +data: + cache: disabled +---- + +You reference the ConfigMap as a parameter value like this: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: a-build + namespace: a-namespace +spec: + paramValues: + - name: cache + configMapValue: + name: buildkit-configuration + key: cache + strategy: + name: buildkit + kind: ClusterBuildStrategy + source: + ... + output: + ... +---- + +The `build-args` parameter is defined as an array. In the BuildKit strategy, you use `build-args` to set the https://docs.docker.com/engine/reference/builder/#arg[`ARG` values in the Dockerfile], specified as key-value pairs separated by an equals sign, for example, `NODE_VERSION=16`. Your Build then looks like this (the value for `cache` is retained to outline how multiple _paramValue_ can be set): + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: a-build + namespace: a-namespace +spec: + paramValues: + - name: cache + configMapValue: + name: buildkit-configuration + key: cache + - name: build-args + values: + - value: NODE_VERSION=16 + strategy: + name: buildkit + kind: ClusterBuildStrategy + source: + ... + output: + ... +---- + +Like simple values, you can also reference ConfigMaps and Secrets for every item in the array. Example: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: a-build + namespace: a-namespace +spec: + paramValues: + - name: cache + configMapValue: + name: buildkit-configuration + key: cache + - name: build-args + values: + - configMapValue: + name: project-configuration + key: node-version + format: NODE_VERSION=${CONFIGMAP_VALUE} + - value: DEBUG_MODE=true + - secretValue: + name: npm-registry-access + key: npm-auth-token + format: NPM_AUTH_TOKEN=${SECRET_VALUE} + strategy: + name: buildkit + kind: ClusterBuildStrategy + source: + ... + output: + ... +---- + +Here, we pass three items in the `build-args` array: + +. The first item references a ConfigMap. Because the ConfigMap just contains the value (for example `"16"`) as the data of the `node-version` key, the `format` setting is used to prepend `NODE_VERSION=` to make it a complete key-value pair. +. The second item is just a hard-coded value. +. The third item references a Secret, the same as with ConfigMaps. + +*NOTE*: The logging output of BuildKit contains expanded ``ARG``s in `RUN` commands. Also, such information ends up in the final container image if you use such args in the https://docs.docker.com/develop/develop-images/multistage-build/[final stage of your Dockerfile]. An alternative approach to pass secrets is using https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-secret-information[secret mounts]. The BuildKit sample strategy supports them using the `secrets` parameter. + +=== Defining the Builder or Dockerfile + +In the `Build` resource, you use the `spec.builder` or `spec.dockerfile` parameters to specify the image that contains the tools to build the final image. For example, the following Build definition specifies a `Dockerfile` image. + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: buildah-golang-build +spec: + source: + url: https://github.com/shipwright-io/sample-go + contextDir: docker-build + strategy: + name: buildah + kind: ClusterBuildStrategy + dockerfile: Dockerfile +---- + +Another example is when the user chooses the `builder` image for a specific language as part of the `source-to-image` buildStrategy: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: s2i-nodejs-build +spec: + source: + url: https://github.com/shipwright-io/sample-nodejs + contextDir: source-build/ + strategy: + name: source-to-image + kind: ClusterBuildStrategy + builder: + image: docker.io/centos/nodejs-10-centos7 +---- + +=== Defining the Output + +A `Build` resource can specify the output where it should push the image. For external private registries, it is recommended to specify a secret with the related data to access it. An option is available to specify the annotation and labels for the output image. The annotations and labels mentioned here are specific to the container image and do not relate to the `Build` annotations. + +*NOTE*: When you specify annotations or labels, the output image will get pushed twice. The first push comes from the build strategy. Then, a follow-on update changes the image configuration to add the annotations and labels. If you have automation based on push events in your container registry, be aware of this behavior. + +For example, the user specifies a public registry: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: s2i-nodejs-build +spec: + source: + url: https://github.com/shipwright-io/sample-nodejs + contextDir: source-build/ + strategy: + name: source-to-image + kind: ClusterBuildStrategy + builder: + image: docker.io/centos/nodejs-10-centos7 + output: + image: image-registry.openshift-image-registry.svc:5000/build-examples/nodejs-ex +---- + +Another example is when the user specifies a private registry: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: s2i-nodejs-build +spec: + source: + url: https://github.com/shipwright-io/sample-nodejs + contextDir: source-build/ + strategy: + name: source-to-image + kind: ClusterBuildStrategy + builder: + image: docker.io/centos/nodejs-10-centos7 + output: + image: us.icr.io/source-to-image-build/nodejs-ex + credentials: + name: icr-knbuild +---- + +Example of user specifies image annotations and labels: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: s2i-nodejs-build +spec: + source: + url: https://github.com/shipwright-io/sample-nodejs + contextDir: source-build/ + strategy: + name: source-to-image + kind: ClusterBuildStrategy + builder: + image: docker.io/centos/nodejs-10-centos7 + output: + image: us.icr.io/source-to-image-build/nodejs-ex + credentials: + name: icr-knbuild + annotations: + "org.opencontainers.image.source": "https://github.com/org/repo" + "org.opencontainers.image.url": "https://my-company.com/images" + labels: + "maintainer": "team@my-company.com" + "description": "This is my cool image" +---- + +Annotations added to the output image can be verified by running the command: + +[,sh] +---- + docker manifest inspect us.icr.io/source-to-image-build/nodejs-ex | jq ".annotations" +---- + +You can verify which labels were added to the output image that is available on the host machine by running the command: + +[,sh] +---- + docker inspect us.icr.io/source-to-image-build/nodejs-ex | jq ".[].Config.Labels" +---- + +=== Defining Retention Parameters + +A `Build` resource can specify how long a completed BuildRun can exist and the number of buildruns that have failed or succeeded that should exist. Instead of manually cleaning up old BuildRuns, retention parameters provide an alternate method for cleaning up BuildRuns automatically. + +As part of the retention parameters, we have the following fields: + +* `retention.succeededLimit` - Defines number of succeeded BuildRuns for a Build that can exist. +* `retention.failedLimit` - Defines number of failed BuildRuns for a Build that can exist. +* `retention.ttlAfterFailed` - Specifies the duration for which a failed buildrun can exist. +* `retention.ttlAfterSucceeded` - Specifies the duration for which a successful buildrun can exist. + +An example of a user using both TTL and Limit retention fields. In case of such a configuration, BuildRun will get deleted once the first criteria is met. + +[,yaml] +---- + apiVersion: shipwright.io/v1alpha1 + kind: Build + metadata: + name: build-retention-ttl + spec: + source: + url: "https://github.com/shipwright-io/sample-go" + contextDir: docker-build + strategy: + kind: ClusterBuildStrategy + output: + ... + retention: + ttlAfterFailed: 30m + ttlAfterSucceeded: 1h + failedLimit: 10 + succeededLimit: 20 +---- + +*NOTE*: When changes are made to `retention.failedLimit` and `retention.succeededLimit` values, they come into effect as soon as the build is applied, thereby enforcing the new limits. On the other hand, changing the `retention.ttlAfterFailed` and `retention.ttlAfterSucceeded` values will only affect new buildruns. Old buildruns will adhere to the old TTL retention values. In case TTL values are defined in buildrun specifications as well as build specifications, priority will be given to the values defined in the buildrun specifications. + +=== Sources + +Sources represent remote artifacts, as in external entities added to the build context before the actual Build starts. Therefore, you may employ `.spec.sources` to download artifacts from external repositories. + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: nodejs-ex +spec: + sources: + - name: project-logo + url: https://gist.github.com/project/image.png +---- + +Under `.spec.sources` are the following attributes: + +* `.name`: represents the name of the resource, required attribute. +* `.url`: universal resource location (URL), required attribute. + +When downloading artifacts, the process is executed in the same directory where the application source-code is located, by default `/workspace/source`. + +Additionally, we plan to keep evolving `.spec.sources` by adding more types of remote data declaration. This API field works as an extension point to support external and internal resource locations. + +At this initial stage, authentication is not supported; therefore, you can only download from sources without this mechanism in place. + +== BuildRun deletion + +A `Build` can automatically delete a related `BuildRun`. To enable this feature set the `build.shipwright.io/build-run-deletion` annotation to `true` in the `Build` instance. This annotation is not present in a `Build` definition by default. See an example of how to define this annotation: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: kaniko-golang-build + annotations: + build.shipwright.io/build-run-deletion: "true" +---- diff --git a/docs/buildrun.adoc b/docs/buildrun.adoc new file mode 100644 index 0000000000..09cc512a86 --- /dev/null +++ b/docs/buildrun.adoc @@ -0,0 +1,596 @@ +//// +Copyright The Shipwright Contributors + +SPDX-License-Identifier: Apache-2.0 +//// += BuildRun + +* <> +* <> +* <> + ** <> + ** <> + ** <> + ** <> + ** <> +* <> +* <> +* <> +* <> + ** <> + ** <> + *** <> + ** <> + ** <> +* <> + +== Overview + +The resource `BuildRun` (`buildruns.shipwright.io/v1alpha1`) is the build process of a `Build` resource definition executed in Kubernetes. + +A `BuildRun` resource allows the user to define: + +* The `BuildRun` name, through which the user can monitor the status of the image construction. +* A referenced `Build` instance to use during the build construction. +* A service account for hosting all related secrets to build the image. + +A `BuildRun` is available within a namespace. + +== BuildRun Controller + +The controller watches for: + +* Updates on a `Build` resource (_CRD instance_) +* Updates on a `TaskRun` resource (_CRD instance_) + +When the controller reconciles it: + +* Looks for any existing owned `TaskRuns` and updates its parent `BuildRun` status. +* Retrieves the specified `SA` and sets this with the specify output secret on the `Build` resource. +* If one does not exist, it generates a new tekton `TaskRun` and sets a reference to this resource(_as a child of the controller_). +* On any subsequent updates on the `TaskRun`, the controller will update the parent `BuildRun` resource instance. + +== Configuring a BuildRun + +The `BuildRun` definition supports the following fields: + +* Required: + ** https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/#required-fields[`apiVersion`] - Specifies the API version, for example `shipwright.io/v1alpha1`. + ** https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/#required-fields[`kind`] - Specifies the Kind type, for example `BuildRun`. + ** https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/#required-fields[`metadata`] - Metadata that identify the CRD instance, for example the name of the `BuildRun`. +* Optional: + ** `spec.buildRef` - Specifies an existing `Build` resource instance to use. It cannot be used together with `buildSpec`. + ** `spec.buildSpec` - Specifies an embedded (transient) Build resource to use. It cannot be used together with `buildRef`. + ** `spec.serviceAccount` - Refers to the SA to use when building the image. (_defaults to the `default` SA_) + ** `spec.timeout` - Defines a custom timeout. The value needs to be parsable by https://golang.org/pkg/time/#ParseDuration[ParseDuration], for example, `5m`. The value overwrites the value that is defined in the `Build`. + ** `spec.paramValues` - Refers to a name-value(s) list to specify values for `parameters` defined in the `BuildStrategy`. This value overwrites values defined with the same name in the Build. + ** `spec.output.image` - Refers to a custom location where the generated image would be pushed. The value will overwrite the `output.image` value defined in `Build`. ( Note: other properties of the output, for example, the credentials, cannot be specified in the buildRun spec. ) + ** `spec.output.credentials.name` - Reference an existing secret to get access to the container registry. This secret will be added to the service account along with the ones requested by the `Build`. + ** `spec.env` - Specifies additional environment variables that should be passed to the build container. Overrides any environment variables that are specified in the `Build` resource. The available variables depend on the tool used by the chosen build strategy. + +NOTE: The `BuildRef` and `BuildSpec` are mutually exclusive. Furthermore, the overrides for `timeout`, `paramValues`, `output`, and `env` can only be combined with `buildRef`, but *not* with `buildSpec`. + +=== Defining the BuildRef + +A `BuildRun` resource can reference a `Build` resource, that indicates what image to build. For example: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: BuildRun +metadata: + name: buildpack-nodejs-buildrun-namespaced +spec: + buildRef: + name: buildpack-nodejs-build-namespaced +---- + +=== Defining the BuildSpec + +Alternatively to `BuildRef`, a complete `BuildSpec` can be embedded into the `BuildRun` for the build. + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: BuildRun +metadata: + name: standalone-buildrun +spec: + buildSpec: + source: + url: https://github.com/shipwright-io/sample-go.git + contextDir: source-build + strategy: + kind: ClusterBuildStrategy + name: buildpacks-v3 + output: + image: foo/bar:latest +---- + +=== Defining ParamValues + +A `BuildRun` resource can define _paramValues_ for parameters specified in the build strategy. If a value has been provided for a parameter with the same name in the `Build` already, then the value from the `BuildRun` will have precedence. + +For example, the following `BuildRun` overrides the value for _sleep-time_ param, which is defined in the _a-build_ `Build` resource. + +[,yaml] +---- +--- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: a-build + namespace: a-namespace +spec: + paramValues: + - name: cache + value: disabled + strategy: + name: buildkit + kind: ClusterBuildStrategy + source: + ... + output: + ... + +--- +apiVersion: shipwright.io/v1alpha1 +kind: BuildRun +metadata: + name: a-buildrun + namespace: a-namespace +spec: + buildRef: + name: a-build + paramValues: + - name: cache + value: registry +---- + +See more about _paramValues_ usage in the related link:./build.md#defining-paramvalues[Build] resource docs. + +=== Defining the ServiceAccount + +A `BuildRun` resource can define a serviceaccount to use. Usually this SA will host all related secrets referenced on the `Build` resource, for example: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: BuildRun +metadata: + name: buildpack-nodejs-buildrun-namespaced +spec: + buildRef: + name: buildpack-nodejs-build-namespaced + serviceAccount: + name: pipeline +---- + +You can also use set the `spec.serviceAccount.generate` path to `true`. This will generate the service account during runtime for you. The name of the generated service account is the name of the BuildRun. + +_*Note*_: When the service account is not defined, the `BuildRun` uses the `pipeline` service account if it exists in the namespace, and falls back to the `default` service account. + +=== Defining Retention Parameters + +A `Buildrun` resource can specify how long a completed BuildRun can exist. Instead of manually cleaning up old BuildRuns, retention parameters provide an alternate method for cleaning up BuildRuns automatically. + +As part of the buildrun retention parameters, we have the following fields: + +* `retention.ttlAfterFailed` - Specifies the duration for which a failed buildrun can exist. +* `retention.ttlAfterSucceeded` - Specifies the duration for which a successful buildrun can exist. + +An example of a user using buildrun TTL parameters. + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: BuildRun +metadata: + name: buidrun-retention-ttl +spec: + buildRef: + name: build-retention-ttl + retention: + ttlAfterFailed: 10m + ttlAfterSucceeded: 10m +---- + +*NOTE* In case TTL values are defined in buildrun specifications as well as build specifications, priority will be given to the values defined in the buildrun specifications. + +== Canceling a `BuildRun` + +To cancel a `BuildRun` that's currently executing, update its status to mark it as canceled. + +When you cancel a `BuildRun`, the underlying `TaskRun` is marked as canceled per the https://github.com/tektoncd/pipeline/blob/main/docs/taskruns.md[Tekton cancel `TaskRun` feature]. + +Example of canceling a `BuildRun`: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: BuildRun +metadata: + name: buildpack-nodejs-buildrun-namespaced +spec: + # [...] + state: "BuildRunCanceled" +---- + +== Automatic `BuildRun` deletion + +We have two controllers that ensure that buildruns can be deleted automatically if required. This is ensured by adding `retention` parameters in either the build specifications or the buildrun specifications. + +* Buildrun TTL parameters: These are used to make sure that buildruns exist for a fixed duration of time after completiion. + ** `buildrun.spec.retention.ttlAfterFailed`: The buildrun is deleted if the mentioned duration of time has passed and the buildrun has failed. + ** `buildrun.spec.retention.ttlAfterSucceeded`: The buildrun is deleted if the mentioned duration of time has passed and the buildrun has succeeded. +* Build TTL parameters: These are used to make sure that related buildruns exist for a fixed duration of time after completiion. + ** `build.spec.retention.ttlAfterFailed`: The buildrun is deleted if the mentioned duration of time has passed and the buildrun has failed. + ** `build.spec.retention.ttlAfterSucceeded`: The buildrun is deleted if the mentioned duration of time has passed and the buildrun has succeeded. +* Build Limit parameters: These are used to make sure that related buildruns exist for a fixed duration of time after completiion. + ** `build.spec.retention.succeededLimit` - Defines number of succeeded BuildRuns for a Build that can exist. + ** `build.spec.retention.failedLimit` - Defines number of failed BuildRuns for a Build that can exist. + +== Specifying Environment Variables + +An example of a `BuildRun` that specifies environment variables: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: BuildRun +metadata: + name: buildpack-nodejs-buildrun-namespaced +spec: + buildRef: + name: buildpack-nodejs-build-namespaced + env: + - name: EXAMPLE_VAR_1 + value: "example-value-1" + - name: EXAMPLE_VAR_2 + value: "example-value-2" +---- + +Example of a `BuildRun` that uses the Kubernetes Downward API to +expose a `Pod` field as an environment variable: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: BuildRun +metadata: + name: buildpack-nodejs-buildrun-namespaced +spec: + buildRef: + name: buildpack-nodejs-build-namespaced + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name +---- + +Example of a `BuildRun` that uses the Kubernetes Downward API to +expose a `Container` field as an environment variable: + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: BuildRun +metadata: + name: buildpack-nodejs-buildrun-namespaced +spec: + buildRef: + name: buildpack-nodejs-build-namespaced + env: + - name: MEMORY_LIMIT + valueFrom: + resourceFieldRef: + containerName: my-container + resource: limits.memory +---- + +== BuildRun Status + +The `BuildRun` resource is updated as soon as the current image building status changes: + +[,sh] +---- +$ kubectl get buildrun buildpacks-v3-buildrun +NAME SUCCEEDED REASON MESSAGE STARTTIME COMPLETIONTIME +buildpacks-v3-buildrun Unknown Pending Pending 1s +---- + +And finally: + +[,sh] +---- +$ kubectl get buildrun buildpacks-v3-buildrun +NAME SUCCEEDED REASON MESSAGE STARTTIME COMPLETIONTIME +buildpacks-v3-buildrun True Succeeded All Steps have completed executing 4m28s 16s +---- + +The above allows users to get an overview of the building mechanism state. + +=== Understanding the state of a BuildRun + +A `BuildRun` resource stores the relevant information regarding the object's state under `status.conditions`. + +https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#typical-status-properties[Conditions] allow users to quickly understand the resource state without needing to understand resource-specific details. + +For the `BuildRun`, we use a Condition of the type `Succeeded`, which is a well-known type for resources that run to completion. + +The `status.conditions` hosts different fields, like `status`, `reason` and `message`. Users can expect these fields to be populated with relevant information. + +The following table illustrates the different states a BuildRun can have under its `status.conditions`: + +|=== +| Status | Reason | CompletionTime is set | Description + +| Unknown +| Pending +| No +| The BuildRun is waiting on a Pod in status Pending. + +| Unknown +| Running +| No +| The BuildRun has been validated and started to perform its work. + +| Unknown +| Running +| No +| The BuildRun has been validated and started to perform its work. + +| Unknown +| BuildRunCanceled +| No +| The user requested the BuildRun to be canceled. This results in the BuildRun controller requesting the TaskRun be canceled. Cancellation has not been done yet. + +| True +| Succeeded +| Yes +| The BuildRun Pod is done. + +| False +| Failed +| Yes +| The BuildRun failed in one of the steps. + +| False +| BuildRunTimeout +| Yes +| The BuildRun timed out. + +| False +| UnknownStrategyKind +| Yes +| The Build specified strategy Kind is unknown. (_options: ClusterBuildStrategy or BuildStrategy_) + +| False +| ClusterBuildStrategyNotFound +| Yes +| The referenced cluster strategy was not found in the cluster. + +| False +| BuildStrategyNotFound +| Yes +| The referenced namespaced strategy was not found in the cluster. + +| False +| SetOwnerReferenceFailed +| Yes +| Setting ownerreferences from the BuildRun to the related TaskRun failed. + +| False +| TaskRunIsMissing +| Yes +| The BuildRun related TaskRun was not found. + +| False +| TaskRunGenerationFailed +| Yes +| The generation of a TaskRun spec failed. + +| False +| MissingParameterValues +| Yes +| No value has been provided for some parameters that are defined in the build strategy without any default. Values for those parameters must be provided through the Build or the BuildRun. + +| False +| RestrictedParametersInUse +| Yes +| A value for a system parameter was provided. This is not allowed. + +| False +| UndefinedParameter +| Yes +| A value for a parameter was provided that is not defined in the build strategy. + +| False +| WrongParameterValueType +| Yes +| A value was provided for a build strategy parameter using the wrong type. The parameter is defined as `array` or `string` in the build strategy. Depending on that, you must provide `values` or a direct value. + +| False +| InconsistentParameterValues +| Yes +| A value for a parameter contained more than one of `value`, `configMapValue`, and `secretValue`. Any values including array items must only provide one of them. + +| False +| EmptyArrayItemParameterValues +| Yes +| An item inside the `values` of an array parameter contained none of `value`, `configMapValue`, and `secretValue`. Exactly one of them must be provided. Null array items are not allowed. + +| False +| IncompleteConfigMapValueParameterValues +| Yes +| A value for a parameter contained a `configMapValue` where the `name` or the `value` were empty. You must specify them to point to an existing ConfigMap key in your namespace. + +| False +| IncompleteSecretValueParameterValues +| Yes +| A value for a parameter contained a `secretValue` where the `name` or the `value` were empty. You must specify them to point to an existing Secret key in your namespace. + +| False +| ServiceAccountNotFound +| Yes +| The referenced service account was not found in the cluster. + +| False +| BuildRegistrationFailed +| Yes +| The related Build in the BuildRun is in a Failed state. + +| False +| BuildNotFound +| Yes +| The related Build in the BuildRun was not found. + +| False +| BuildRunCanceled +| Yes +| The BuildRun and underlying TaskRun were canceled successfully. + +| False +| BuildRunNameInvalid +| Yes +| The defined `BuildRun` name (`metadata.name`) is invalid. The `BuildRun` name should be a https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set[valid label value]. + +| False +| BuildRunNoRefOrSpec +| Yes +| BuildRun does not have either `BuildRef` or `BuildSpec` defined. There is no connection to a Build specification. + +| False +| BuildRunAmbiguousBuild +| Yes +| The defined `BuildRun` uses both `BuildRef` and `BuildSpec`. Only one of them is allowed at the same time. + +| False +| BuildRunBuildFieldOverrideForbidden +| Yes +| The defined `BuildRun` uses an override (e.g. `timeout`, `paramValues`, `output`, or `env`) in combination with `BuildSpec`, which is not allowed. Use the `BuildSpec` to directly specify the respective value. + +| False +| PodEvicted +| Yes +| The BuildRun Pod was evicted from the node it was running on. See https://kubernetes.io/docs/concepts/scheduling-eviction/api-eviction/[API-initiated Eviction] and https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/[Node-pressure Eviction] for more information. +|=== + +NOTE: We heavily rely on the Tekton TaskRun https://github.com/tektoncd/pipeline/blob/main/docs/taskruns.md#monitoring-execution-status[Conditions] for populating the BuildRun ones, with some exceptions. + +=== Understanding failed BuildRuns + +[DEPRECATED] To make it easier for users to understand why did a BuildRun failed, users can infer the pod and container where the failure took place from the `status.failedAt` field. + +In addition, the `status.conditions` hosts a compacted message under the' message' field that contains the `kubectl` command to trigger and retrieve the logs. + +Lastly, users can check the `status.failureDetails` field, which includes the same information available in the `status.failedAt` field, +as well as a human-readable error message and reason. +The message and reason are only included if the build strategy provides them. + +Example of failed BuildRun: + +[,yaml] +---- +# [...] +status: + # [...] + failureDetails: + location: + container: step-source-default + pod: baran-build-buildrun-gzmv5-b7wbf-pod-bbpqr + message: The source repository does not exist, or you have insufficient permission + to access it. + reason: GitRemotePrivate +---- + +==== Understanding failed git-source step + +All git-related operations support error reporting via `status.failureDetails`. The following table explains the possible +error reasons: + +|=== +| Reason | Description + +| `GitAuthInvalidUserOrPass` +| Basic authentication has failed. Check your username or password. Note: GitHub requires a personal access token instead of your regular password. + +| `GitAuthInvalidKey` +| The key is invalid for the specified target. Please make sure that the Git repository exists, you have sufficient permissions, and the key is in the right format. + +| `GitRevisionNotFound` +| The remote revision does not exist. Check the revision specified in your Build. + +| `GitRemoteRepositoryNotFound` +| The source repository does not exist, or you have insufficient permissions to access it. + +| `GitRemoteRepositoryPrivate` +| You are trying to access a non-existing or private repository without having sufficient permissions to access it via HTTPS. + +| `GitBasicAuthIncomplete` +| Basic Auth incomplete: Both username and password must be configured. + +| `GitSSHAuthUnexpected` +| Credential/URL inconsistency: SSH credentials were provided, but the URL is not an SSH Git URL. + +| `GitSSHAuthExpected` +| Credential/URL inconsistency: No SSH credentials provided, but the URL is an SSH Git URL. + +| `GitError` +| The specific error reason is unknown. Check the error message for more information. +|=== + +=== Step Results in BuildRun Status + +After completing a `BuildRun`, the `.status` field contains the results (`.status.taskResults`) emitted from the `TaskRun` steps generated by the `BuildRun` controller as part of processing the `BuildRun`. These results contain valuable metadata for users, like the _image digest_ or the _commit sha_ of the source code used for building. +The results from the source step will be surfaced to the `.status.sources`, and the results from +the link:buildstrategies.md#system-results[output step] will be surfaced to the `.status.output` field of a `BuildRun`. + +Example of a `BuildRun` with surfaced results for `git` source (note that the `branchName` is only included if the Build does not specify any `revision`): + +[,yaml] +---- +# [...] +status: + buildSpec: + # [...] + output: + digest: sha256:07626e3c7fdd28d5328a8d6df8d29cd3da760c7f5e2070b534f9b880ed093a53 + size: 1989004 + sources: + - name: default + git: + commitAuthor: xxx xxxxxx + commitSha: f25822b85021d02059c9ac8a211ef3804ea8fdde + branchName: main +---- + +Another example of a `BuildRun` with surfaced results for local source code(`bundle`) source: + +[,yaml] +---- +# [...] +status: + buildSpec: + # [...] + output: + digest: sha256:07626e3c7fdd28d5328a8d6df8d29cd3da760c7f5e2070b534f9b880ed093a53 + size: 1989004 + sources: + - name: default + bundle: + digest: sha256:0f5e2070b534f9b880ed093a537626e3c7fdd28d5328a8d6df8d29cd3da760c7 +---- + +NOTE: The digest and size of the output image are only included if the build strategy provides them. See link:buildstrategies.md#system-results[System results]. + +=== Build Snapshot + +For every BuildRun controller reconciliation, the `buildSpec` in the status of the `BuildRun` is updated if an existing owned `TaskRun` is present. During this update, a `Build` resource snapshot is generated and embedded into the `status.buildSpec` path of the `BuildRun`. A `buildSpec` is just a copy of the original `Build` spec, from where the `BuildRun` executed a particular image build. The snapshot approach allows developers to see the original `Build` configuration. + +== Relationship with Tekton Tasks + +The `BuildRun` resource abstracts the image construction by delegating this work to the Tekton Pipeline https://github.com/tektoncd/pipeline/blob/main/docs/taskruns.md[TaskRun]. Compared to a Tekton Pipeline https://github.com/tektoncd/pipeline/blob/main/docs/tasks.md[Task], a `TaskRun` runs all `steps` until completion of the `Task` or until a failure occurs in the `Task`. + +During the Reconcile, the `BuildRun` controller will generate a new `TaskRun`. The controller will embed in the `TaskRun` `Task` definition the requires `steps` to execute during the execution. These `steps` are defined in the strategy defined in the `Build` resource, either a `ClusterBuildStrategy` or a `BuildStrategy`. diff --git a/docs/buildstrategies.adoc b/docs/buildstrategies.adoc new file mode 100644 index 0000000000..27d5f13bbb --- /dev/null +++ b/docs/buildstrategies.adoc @@ -0,0 +1,965 @@ +//// +Copyright The Shipwright Contributors + +SPDX-License-Identifier: Apache-2.0 +//// += BuildStrategies + +* <> +* <> +* <> +* <> + ** <> +* <> + ** <> +* <> + ** <> + *** <> +* <> + ** <> + ** <> + ** <> + ** <> + ** <> + ** <> +* <> + ** <> + ** <> +* <> + ** <> + ** <> +* <> +* <> +* <> +* <> +* <> +* <> + ** <> + ** <> + ** <> +* <> +* <> + +== Overview + +There are two types of strategies, the `ClusterBuildStrategy` (`clusterbuildstrategies.shipwright.io/v1alpha1`) and the `BuildStrategy` (`buildstrategies.shipwright.io/v1alpha1`). Both strategies define a shared group of steps, needed to fullfil the application build. + +A `ClusterBuildStrategy` is available cluster-wide, while a `BuildStrategy` is available within a namespace. + +== Available ClusterBuildStrategies + +Well-known strategies can be bootstrapped from link:../samples/buildstrategy[here]. The currently supported Cluster BuildStrategy are: + +|=== +| Name | Supported platforms + +| link:../samples/buildstrategy/buildah/buildstrategy_buildah_cr.yaml[buildah] +| all + +| link:../samples/buildstrategy/buildkit/buildstrategy_buildkit_cr.yaml[BuildKit] +| all + +| link:../samples/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3-heroku_cr.yaml[buildpacks-v3-heroku] +| linux/amd64 only + +| link:../samples/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_cr.yaml[buildpacks-v3] +| linux/amd64 only + +| link:../samples/buildstrategy/kaniko/buildstrategy_kaniko_cr.yaml[kaniko] +| all + +| link:../samples/buildstrategy/ko/buildstrategy_ko_cr.yaml[ko] +| all + +| link:../samples/buildstrategy/source-to-image/buildstrategy_source-to-image_cr.yaml[source-to-image] +| linux/amd64 only +|=== + +== Available BuildStrategies + +The current supported namespaces BuildStrategy are: + +|=== +| Name | Supported platforms + +| link:../samples/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3-heroku_namespaced_cr.yaml[buildpacks-v3-heroku] +| linux/amd64 only + +| link:../samples/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_namespaced_cr.yaml[buildpacks-v3] +| linux/amd64 only +|=== + +''' + +== Buildah + +The `buildah` ClusterBuildStrategy consists of using https://github.com/containers/buildah[`buildah`] to build and push a container image, out of a `Dockerfile`. The `Dockerfile` should be specified on the `Build` resource. + +=== Installing Buildah Strategy + +To install use: + +[,sh] +---- +kubectl apply -f samples/buildstrategy/buildah/buildstrategy_buildah_cr.yaml +---- + +''' + +== Buildpacks v3 + +The https://buildpacks.io/[buildpacks-v3] BuildStrategy/ClusterBuildStrategy uses a Cloud Native Builder (https://buildpacks.io/docs/concepts/components/builder/[CNB]) container image, and is able to implement https://buildpacks.io/docs/concepts/components/lifecycle/[lifecycle commands]. The following CNB images are the most common options: + +* https://hub.docker.com/r/heroku/buildpacks/[`heroku/buildpacks:18`] +* https://hub.docker.com/r/cloudfoundry/cnb[`cloudfoundry/cnb:bionic`] +* https://hub.docker.com/r/paketobuildpacks/builder/tags[`docker.io/paketobuildpacks/builder:full`] + +=== Installing Buildpacks v3 Strategy + +You can install the `BuildStrategy` in your namespace or install the `ClusterBuildStrategy` at cluster scope so that it can be shared across namespaces. + +To install the cluster scope strategy, use (below is a heroku example, you can also use paketo sample): + +[,sh] +---- +kubectl apply -f samples/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3-heroku_cr.yaml +---- + +To install the namespaced scope strategy, use: + +[,sh] +---- +kubectl apply -f samples/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3-heroku_namespaced_cr.yaml +---- + +''' + +== Kaniko + +The `kaniko` ClusterBuildStrategy is composed by Kaniko's `executor` https://github.com/GoogleContainerTools/kaniko[kaniko], with the objective of building a container-image, out of a `Dockerfile` and context directory. The `kaniko-trivy` ClusterBuildStrategy adds https://github.com/aquasecurity/trivy[trivy] scanning and refuses to push images with critical vulnerabilities. + +=== Installing Kaniko Strategy + +To install the cluster scope strategy, use: + +[,sh] +---- +kubectl apply -f samples/buildstrategy/kaniko/buildstrategy_kaniko_cr.yaml +---- + +==== Scanning with Trivy + +You can also incorporate scanning into the ClusterBuildStrategy. The `kaniko-trivy` ClusterBuildStrategy builds the image with `kaniko`, then scans with https://github.com/aquasecurity/trivy[trivy]. The BuildRun will then exit with an error if there is a critical vulnerability, instead of pushing the vulnerable image into the container registry. + +To install the cluster scope strategy, use: + +[,sh] +---- +kubectl apply -f samples/buildstrategy/kaniko/buildstrategy_kaniko-trivy_cr.yaml +---- + +_Note: doing image scanning is not a substitute for trusting the Dockerfile you are building. The build process itself is also susceptible if the Dockerfile has a vulnerability. Frameworks/strategies such as build-packs or source-to-image (which avoid directly building a Dockerfile) should be considered if you need guardrails around the code you want to build._ + +''' + +== BuildKit + +https://github.com/moby/buildkit[BuildKit] is composed of the `buildctl` client and the `buildkitd` daemon. For the `buildkit` ClusterBuildStrategy, it runs on a https://github.com/moby/buildkit#daemonless[daemonless] mode, where both client and ephemeral daemon run in a single container. In addition, it runs without privileges (_https://github.com/moby/buildkit/blob/master/docs/rootless.md[rootless]_). + +=== Cache Exporters + +By default, the `buildkit` ClusterBuildStrategy will use caching to optimize the build times. When pushing an image to a registry, it will use the inline export cache, which appends cache information to the image that is built. Please refer to https://github.com/moby/buildkit#export-cache[export-cache docs] for more information. Caching can be disabled by setting the `cache` parameter to `"disabled"`. See link:build.md#defining-paramvalues[Defining ParamValues] for more information. + +=== Build-args and secrets + +The sample build strategy contains array parameters to set values for https://docs.docker.com/engine/reference/builder/#arg[``ARG``s in your Dockerfile], and for https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-secret-information[mounts with type=secret]. The parameter names are `build-args` and `secrets`. link:build.md#defining-paramvalues[Defining ParamValues] contains example usage. + +=== Multi-platform builds + +The sample build strategy contains a `platforms` array parameter that you can set to leverage https://github.com/moby/buildkit/blob/master/docs/multi-platform.md[BuildKit's support to build multi-platform images]. If you do not set this value, the image is built for the platform that is supported by the `FROM` image. If that image supports multiple platforms, then the image will be built for the platform of your Kubernetes node. + +=== Known Limitations + +The `buildkit` ClusterBuildStrategy currently locks the following parameters: + +* To allow running rootless, it requires both https://kubernetes.io/docs/tutorials/clusters/apparmor/[AppArmor] as well as https://kubernetes.io/docs/tutorials/clusters/seccomp/[SecComp] to be disabled using the `unconfined` profile. + +=== Usage in Clusters with Pod Security Standards + +The BuildKit strategy contains fields with regards to security settings. It therefore depends on the respective cluster setup and administrative configuration. These settings are: + +* Defining the `unconfined` profile for both AppArmor and seccomp as required by the underlying `rootlesskit`. +* The `allowPrivilegeEscalation` settings is set to `true` to be able to use binaries that have the `setuid` bit set in order to run with "root" level privileges. In case of BuildKit, this is required by `rootlesskit` in order to set the user namespace mapping file `/proc//uid_map`. +* Use of non-root user with UID 1000/GID 1000 as the `runAsUser`. + +These settings have no effect in case Pod Security Standards are not used. + +_Please note:_ At this point in time, there is no way to run `rootlesskit` to start the BuildKit daemon without the `allowPrivilegeEscalation` flag set to `true`. Clusters with the `Restricted` security standard in place will not be able to use this build strategy. + +=== Installing BuildKit Strategy + +To install the cluster scope strategy, use: + +[,sh] +---- +kubectl apply -f samples/buildstrategy/buildkit/buildstrategy_buildkit_cr.yaml +---- + +''' + +== ko + +The `ko` ClusterBuilderStrategy is using https://github.com/google/ko[ko]'s publish command to build an image from a Golang main package. + +=== Installing ko Strategy + +To install the cluster scope strategy, use: + +[,sh] +---- +kubectl apply -f samples/buildstrategy/ko/buildstrategy_ko_cr.yaml +---- + +=== Parameters + +The build strategy provides the following parameters that you can set in a Build or BuildRun to control its behavior: + +|=== +| Parameter | Description | Default + +| `go-flags` +| Value for the GOFLAGS environment variable. +| Empty + +| `go-version` +| Version of Go, must match a tag from https://hub.docker.com/_/golang?tab=tags[the golang image] +| `1.17` + +| `ko-version` +| Version of ko, must be either `latest` for the newest release, or a https://github.com/google/ko/releases[ko release name] +| `latest` + +| `package-directory` +| The directory inside the context directory containing the main package. +| `.` + +| `target-platform` +| Target platform to be built. For example: `linux/arm64`. Multiple platforms can be provided separated by comma, for example: `linux/arm64,linux/amd64`. The value `all` will build all platforms supported by the base image. The value `current` will build the platform on which the build runs. +| `current` +|=== + +== Source to Image + +This BuildStrategy is composed by https://github.com/openshift/source-to-image[`source-to-image`] and https://github.com/GoogleContainerTools/kaniko[`kaniko`] in order to generate a `Dockerfile` and prepare the application to be built later on with a builder. + +`s2i` requires a specially crafted image, which can be informed as `builderImage` parameter on the `Build` resource. + +=== Installing Source to Image Strategy + +To install the cluster scope strategy use: + +[,sh] +---- +kubectl apply -f samples/buildstrategy/source-to-image/buildstrategy_source-to-image_cr.yaml +---- + +=== Build Steps + +. `s2i` in order to generate a `Dockerfile` and prepare source-code for image build; +. `kaniko` to create and push the container image to what is defined as `output.image`; + +== Strategy parameters + +Strategy parameters allow users to parameterize their strategy definition, by allowing users to control the _parameters_ values via the `Build` or `BuildRun` resources. + +Users defining _parameters_ under their strategies require to understand the following: + +* *Definition*: A list of parameters should be defined under `spec.parameters`. Each list item should consist of a _name_, a _description_, a _type_ (either `"array"` or `"string"`) and optionally a _default_ value (for type=string), or _defaults_ values (for type=array). If no default(s) are provided, then the user must define a value in the Build or BuildRun. +* *Usage*: In order to use a parameter in the strategy steps, use the following syntax for type=string: `$(params.your-parameter-name)`. String parameters can be used in all places in the `buildSteps`. Some example scenarios are: + ** `image`: to use a custom tag, for example `golang:$(params.go-version)` as it is done in the link:../samples/buildstrategy/ko/buildstrategy_ko_cr.yaml[ko sample build strategy]) + ** `args`: to pass data into your builder command + ** `env`: to force a user to provide a value for an environment variable. + ++ +Arrays are referenced using `$(params.your-array-parameter-name[*])`, and can only be used in as the value for `args` or `command` because the defined as arrays by Kubernetes. For every item in the array, an arg will be set. For example, if you specify this in your build strategy step: ++ +[,yaml] +---- +spec: + parameters: + - name: tool-args + description: Parameters for the tool + type: array + buildSteps: + - name: a-step + command: + - some-tool + args: + - $(params.tool-args[*]) +---- ++ +If the build user sets the value of tool-args to ["--some-arg", "some-value"], then the Pod will contain these args: ++ +[,yaml] +---- +spec: + containers: + - name: a-step + args: + ... + - --some-arg + - some-value +---- +* *Parameterize*: Any `Build` or `BuildRun` referencing your strategy, can set a value for _your-parameter-name_ parameter if needed. + +NOTE: Users can provide parameter values as simple strings or as references to keys in https://kubernetes.io/docs/concepts/configuration/configmap/[ConfigMaps] and https://kubernetes.io/docs/concepts/configuration/secret/[Secrets]. If they use a ConfigMap or Secret, then the value can only be used if the parameter is used in the `command`, `args`, or `env` section of the `buildSteps`. For example, the above mentioned scenario to set a step's `image` to `golang:$(params.go-version)` does not allow the usage of ConfigMaps or Secrets. + +The following example is from the link:../samples/buildstrategy/buildkit/buildstrategy_buildkit_cr.yaml[BuildKit sample build strategy]. It defines and uses several parameters: + +[,yaml] +---- +--- +apiVersion: shipwright.io/v1alpha1 +kind: ClusterBuildStrategy +metadata: + name: buildkit + ... +spec: + parameters: + - name: build-args + description: "The values for the ARGs in the Dockerfile. Values must be in the format KEY=VALUE." + type: array + defaults: [] + - name: cache + description: "Configure BuildKit's cache usage. Allowed values are 'disabled' and 'registry'. The default is 'registry'." + type: string + default: registry + - name: insecure-registry + type: string + description: "enables the push to an insecure registry" + default: "false" + - name: secrets + description: "The secrets to pass to the build. Values must be in the format ID=FILE_CONTENT." + type: array + defaults: [] + buildSteps: + ... + - name: build-and-push + image: moby/buildkit:nightly-rootless + imagePullPolicy: Always + workingDir: $(params.shp-source-root) + ... + command: + - /bin/ash + args: + - -c + - | + set -euo pipefail + + # Prepare the file arguments + DOCKERFILE_PATH='$(params.shp-source-context)/$(build.dockerfile)' + DOCKERFILE_DIR="$(dirname "${DOCKERFILE_PATH}")" + DOCKERFILE_NAME="$(basename "${DOCKERFILE_PATH}")" + + # We only have ash here and therefore no bash arrays to help add dynamic arguments (the build-args) to the build command. + + echo "#!/bin/ash" > /tmp/run.sh + echo "set -euo pipefail" >> /tmp/run.sh + echo "buildctl-daemonless.sh \\" >> /tmp/run.sh + echo "build \\" >> /tmp/run.sh + echo "--progress=plain \\" >> /tmp/run.sh + echo "--frontend=dockerfile.v0 \\" >> /tmp/run.sh + echo "--opt=filename=\"${DOCKERFILE_NAME}\" \\" >> /tmp/run.sh + echo "--local=context='$(params.shp-source-context)' \\" >> /tmp/run.sh + echo "--local=dockerfile=\"${DOCKERFILE_DIR}\" \\" >> /tmp/run.sh + echo "--output=type=image,name='$(params.shp-output-image)',push=true,registry.insecure=$(params.insecure-registry) \\" >> /tmp/run.sh + if [ "$(params.cache)" == "registry" ]; then + echo "--export-cache=type=inline \\" >> /tmp/run.sh + echo "--import-cache=type=registry,ref='$(params.shp-output-image)' \\" >> /tmp/run.sh + elif [ "$(params.cache)" == "disabled" ]; then + echo "--no-cache \\" >> /tmp/run.sh + else + echo -e "An invalid value for the parameter 'cache' has been provided: '$(params.cache)'. Allowed values are 'disabled' and 'registry'." + echo -n "InvalidParameterValue" > '$(results.shp-error-reason.path)' + echo -n "An invalid value for the parameter 'cache' has been provided: '$(params.cache)'. Allowed values are 'disabled' and 'registry'." > '$(results.shp-error-message.path)' + exit 1 + fi + + stage="" + for a in "$@" + do + if [ "${a}" == "--build-args" ]; then + stage=build-args + elif [ "${a}" == "--secrets" ]; then + stage=secrets + elif [ "${stage}" == "build-args" ]; then + echo "--opt=\"build-arg:${a}\" \\" >> /tmp/run.sh + elif [ "${stage}" == "secrets" ]; then + # Split ID=FILE_CONTENT into variables id and data + + # using head because the data could be multiline + id="$(echo "${a}" | head -1 | sed 's/=.*//')" + + # This is hacky, we remove the suffix ${id}= from all lines of the data. + # If the data would be multiple lines and a line would start with ${id}= + # then we would remove it. We could force users to give us the secret + # base64 encoded. But ultimately, the best solution might be if the user + # mounts the secret and just gives us the path here. + data="$(echo "${a}" | sed "s/^${id}=//")" + + # Write the secret data into a temporary file, once we have volume support + # in the build strategy, we should use a memory based emptyDir for this. + echo -n "${data}" > "/tmp/secret_${id}" + + # Add the secret argument + echo "--secret id=${id},src="/tmp/secret_${id}" \\" >> /tmp/run.sh + fi + done + + echo "--metadata-file /tmp/image-metadata.json" >> /tmp/run.sh + + chmod +x /tmp/run.sh + /tmp/run.sh + + # Store the image digest + sed -E 's/.*containerimage.digest":"([^"]*).*/\1/' < /tmp/image-metadata.json > '$(results.shp-image-digest.path)' + # That's the separator between the shell script and its args + - -- + - --build-args + - $(params.build-args[*]) + - --secrets + - $(params.secrets[*]) +---- + +See more information on how to use these parameters in a `Build` or `BuildRun` in the related link:./build.md#defining-paramvalues[documentation]. + +== System parameters + +Contrary to the strategy `spec.parameters`, you can use system parameters and their values defined at runtime when defining the steps of a build strategy to access system information as well as information provided by the user in their Build or BuildRun. The following parameters are available: + +|=== +| Parameter | Description + +| `$(params.shp-source-root)` +| The absolute path to the directory that contains the user's sources. + +| `$(params.shp-source-context)` +| The absolute path to the context directory of the user's sources. If the user specified no value for `spec.source.contextDir` in their `Build`, then this value will equal the value for `$(params.shp-source-root)`. Note that this directory is not guaranteed to exist at the time the container for your step is started, you can therefore not use this parameter as a step's working directory. + +| `$(params.shp-output-image)` +| The URL of the image that the user wants to push as specified in the Build's `spec.output.image`, or the override from the BuildRun's `spec.output.image`. +|=== + +== System parameters vs Strategy Parameters Comparison + +|=== +| Parameter Type | User Configurable | Definition + +| System Parameter +| No +| At run-time, by the `BuildRun` controller. + +| Strategy Parameter +| Yes +| At build-time, during the `BuildStrategy` creation. +|=== + +== Securely referencing string parameters + +In build strategy steps, string parameters are referenced using `$(params.PARAM_NAME)`. This applies to system parameters, and those parameters defined in the build strategy. You can reference those parameters at many locations in the build steps, such as environment variables values, arguments, image, and more. In the Pod, all `$(params.PARAM_NAME)` tokens will be replaced by simple string replaces. This is safe in most locations but requires your attention when you define an inline script using an argument. For example: + +[,yaml] +---- +spec: + parameters: + - name: sample-parameter + description: A sample parameter + type: string + buildSteps: + - name: sample-step + command: + - /bin/bash + args: + - -c + - | + set -euo pipefail + + some-tool --sample-argument "$(params.sample-parameter)" +---- + +This opens the door to script injection, for example if the user sets the `sample-parameter` to `argument-value" && malicious-command && echo "`, the resulting pod argument will look like this: + +[,yaml] +---- + - | + set -euo pipefail + + some-tool --sample-argument "argument-value" && malicious-command && echo "" +---- + +To securely pass a parameter value into a script-style argument, you can chose between these two approaches: + +. Using environment variables. This is used in some of our sample strategies, for example link:../samples/buildstrategy/ko/buildstrategy_ko_cr.yaml[ko], or link:../samples/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_cr.yaml[buildpacks]. Basically, instead of directly using the parameter inside the script, you pass it via environment variable. Using quoting, shells ensure that no command injection is possible: ++ +[,yaml] +---- +spec: + parameters: + - name: sample-parameter + description: A sample parameter + type: string + buildSteps: + - name: sample-step + env: + - name: PARAM_SAMPLE_PARAMETER + value: $(params.sample-parameter) + command: + - /bin/bash + args: + - -c + - | + set -euo pipefail + + some-tool --sample-argument "${PARAM_SAMPLE_PARAMETER}" +---- + +. Using arguments. This is used in some of our sample build strategies, for example link:../samples/buildstrategy/buildah/buildstrategy_buildah_cr.yaml[buildah]. Here, you use arguments to your own inline script. Appropriate shell quoting guards against command injection. ++ +[,yaml] +---- +spec: + parameters: + - name: sample-parameter + description: A sample parameter + type: string + buildSteps: + - name: sample-step + command: + - /bin/bash + args: + - -c + - | + set -euo pipefail + + SAMPLE_PARAMETER="$1" + + some-tool --sample-argument "${SAMPLE_PARAMETER}" + - -- + - $(params.sample-parameter) +---- + +== System results + +You can optionally store the size and digest of the image your build strategy created to a set of files. + +|=== +| Result file | Description + +| `$(results.shp-image-digest.path)` +| File to store the digest of the image. + +| `$(results.shp-image-size.path)` +| File to store the compressed size of the image. +|=== + +You can look at sample build strategies, such as link:../samples/buildstrategy/kaniko/buildstrategy_kaniko_cr.yaml[Kaniko], or link:../samples/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_cr.yaml[Buildpacks], to see how they fill some or all of the results files. + +This information will be available in the `.status.output` field of the BuildRun. + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: BuildRun +# [...] +status: + # [...] + output: + digest: sha256:07626e3c7fdd28d5328a8d6df8d29cd3da760c7f5e2070b534f9b880ed093a53 + size: 1989004 + # [...] +---- + +Additionally, you can store error details for debugging purposes when a BuildRun fails using your strategy. + +|=== +| Result file | Description + +| `$(results.shp-error-reason.path)` +| File to store the error reason. + +| `$(results.shp-error-message.path)` +| File to store the error message. +|=== + +Reason is intended to be a one-word CamelCase classification of the error source, with the first letter capitalized. +Error details are only propagated if the build container terminates with a non-zero exit code. +This information will be available in the `.status.failureDetails` field of the BuildRun. + +[,yaml] +---- +apiVersion: shipwright.io/v1alpha1 +kind: BuildRun +# [...] +status: + # [...] + failureDetails: + location: + container: step-source-default + pod: baran-build-buildrun-gzmv5-b7wbf-pod-bbpqr + message: The source repository does not exist, or you have insufficient permission + to access it. + reason: GitRemotePrivate +---- + +== Steps Resource Definition + +All strategies steps can include a definition of resources(_limits and requests_) for CPU, memory and disk. For strategies with more than one step, each step(_container_) could require more resources than others. Strategy admins are free to define the values that they consider the best fit for each step. Also, identical strategies with the same steps that are only different in their name and step resources can be installed on the cluster to allow users to create a build with smaller and larger resource requirements. + +=== Strategies with different resources + +If the strategy admins would require to have multiple flavours of the same strategy, where one strategy has more resources that the other. Then, multiple strategies for the same type should be defined on the cluster. In the following example, we use Kaniko as the type: + +[,yaml] +---- +--- +apiVersion: shipwright.io/v1alpha1 +kind: ClusterBuildStrategy +metadata: + name: kaniko-small +spec: + buildSteps: + - name: build-and-push + image: gcr.io/kaniko-project/executor:v1.8.1 + workingDir: $(params.shp-source-root) + securityContext: + runAsUser: 0 + capabilities: + add: + - CHOWN + - DAC_OVERRIDE + - FOWNER + - SETGID + - SETUID + - SETFCAP + - KILL + env: + - name: DOCKER_CONFIG + value: /tekton/home/.docker + - name: AWS_ACCESS_KEY_ID + value: NOT_SET + - name: AWS_SECRET_KEY + value: NOT_SET + command: + - /kaniko/executor + args: + - --skip-tls-verify=true + - --dockerfile=$(build.dockerfile) + - --context=$(params.shp-source-context) + - --destination=$(params.shp-output-image) + - --snapshotMode=redo + - --push-retry=3 + resources: + limits: + cpu: 250m + memory: 65Mi + requests: + cpu: 250m + memory: 65Mi +--- +apiVersion: shipwright.io/v1alpha1 +kind: ClusterBuildStrategy +metadata: + name: kaniko-medium +spec: + buildSteps: + - name: build-and-push + image: gcr.io/kaniko-project/executor:v1.8.1 + workingDir: $(params.shp-source-root) + securityContext: + runAsUser: 0 + capabilities: + add: + - CHOWN + - DAC_OVERRIDE + - FOWNER + - SETGID + - SETUID + - SETFCAP + - KILL + env: + - name: DOCKER_CONFIG + value: /tekton/home/.docker + - name: AWS_ACCESS_KEY_ID + value: NOT_SET + - name: AWS_SECRET_KEY + value: NOT_SET + command: + - /kaniko/executor + args: + - --skip-tls-verify=true + - --dockerfile=$(build.dockerfile) + - --context=$(params.shp-source-context) + - --destination=$(params.shp-output-image) + - --snapshotMode=redo + - --push-retry=3 + resources: + limits: + cpu: 500m + memory: 1Gi + requests: + cpu: 500m + memory: 1Gi +---- + +The above provides more control and flexibility for the strategy admins. For `end-users`, all they need to do, is to reference the proper strategy. For example: + +[,yaml] +---- +--- +apiVersion: shipwright.io/v1alpha1 +kind: Build +metadata: + name: kaniko-medium +spec: + source: + url: https://github.com/shipwright-io/sample-go + contextDir: docker-build + strategy: + name: kaniko + kind: ClusterBuildStrategy + dockerfile: Dockerfile +---- + +=== How does Tekton Pipelines handle resources + +The *Build* controller relies on the Tekton https://github.com/tektoncd/pipeline[pipeline controller] to schedule the `pods` that execute the above strategy steps. In a nutshell, the *Build* controller creates on run-time a Tekton *TaskRun*, and the *TaskRun* generates a new pod in the particular namespace. In order to build an image, the pod executes all the strategy steps one-by-one. + +Tekton manage each step resources *request* in a very particular way, see the https://github.com/tektoncd/pipeline/blob/main/docs/tasks.md#defining-steps[docs]. From this document, it mentions the following: + +____ +The CPU, memory, and ephemeral storage resource requests will be set to zero, or, if specified, the minimums set through LimitRanges in that Namespace, if the container image does not have the largest resource request out of all container images in the Task. This ensures that the Pod that executes the Task only requests enough resources to run a single container image in the Task rather than hoard resources for all container images in the Task at once. +____ + +=== Examples of Tekton resources management + +For a more concrete example, let´s take a look on the following scenarios: + +''' + +*Scenario 1.* Namespace without `LimitRange`, both steps with the same resource values. + +If we will apply the following resources: + +* link:../samples/build/build_buildah_cr.yaml[buildahBuild] +* link:../samples/buildrun/buildrun_buildah_cr.yaml[buildahBuildRun] +* link:../samples/buildstrategy/buildah/buildstrategy_buildah_cr.yaml[buildahClusterBuildStrategy] + +We will see some differences between the `TaskRun` definition and the `pod` definition. + +For the `TaskRun`, as expected we can see the resources on each `step`, as we previously define on our link:../samples/buildstrategy/buildah/buildstrategy_buildah_cr.yaml[strategy]. + +[,sh] +---- +$ kubectl -n test-build get tr buildah-golang-buildrun-9gmcx-pod-lhzbc -o json | jq '.spec.taskSpec.steps[] | select(.name == "step-buildah-bud" ) | .resources' +{ + "limits": { + "cpu": "500m", + "memory": "1Gi" + }, + "requests": { + "cpu": "250m", + "memory": "65Mi" + } +} + +$ kubectl -n test-build get tr buildah-golang-buildrun-9gmcx-pod-lhzbc -o json | jq '.spec.taskSpec.steps[] | select(.name == "step-buildah-push" ) | .resources' +{ + "limits": { + "cpu": "500m", + "memory": "1Gi" + }, + "requests": { + "cpu": "250m", + "memory": "65Mi" + } +} +---- + +The pod definition is different, while Tekton will only use the *highest* values of one container, and set the rest(lowest) to zero: + +[,sh] +---- +$ kubectl -n test-build get pods buildah-golang-buildrun-9gmcx-pod-lhzbc -o json | jq '.spec.containers[] | select(.name == "step-step-buildah-bud" ) | .resources' +{ + "limits": { + "cpu": "500m", + "memory": "1Gi" + }, + "requests": { + "cpu": "250m", + "ephemeral-storage": "0", + "memory": "65Mi" + } +} + +$ kubectl -n test-build get pods buildah-golang-buildrun-9gmcx-pod-lhzbc -o json | jq '.spec.containers[] | select(.name == "step-step-buildah-push" ) | .resources' +{ + "limits": { + "cpu": "500m", + "memory": "1Gi" + }, + "requests": { + "cpu": "0", <------------------- See how the request is set to ZERO. + "ephemeral-storage": "0", <------------------- See how the request is set to ZERO. + "memory": "0" <------------------- See how the request is set to ZERO. + } +} +---- + +In this scenario, only one container can have the `spec.resources.requests` definition. Even when both steps have the same values, only one container will get them, the others will be set to zero. + +''' + +*Scenario 2.* Namespace without `LimitRange`, steps with different resources: + +If we will apply the following resources: + +* link:../samples/build/build_buildah_cr.yaml[buildahBuild] +* link:../samples/buildrun/buildrun_buildah_cr.yaml[buildahBuildRun] +* We will use a modified buildah strategy, with the following steps resources: ++ +[,yaml] +---- + - name: buildah-bud + image: quay.io/containers/buildah:v1.20.1 + workingDir: $(params.shp-source-root) + securityContext: + privileged: true + command: + - /usr/bin/buildah + args: + - bud + - --tag=$(params.shp-output-image) + - --file=$(build.dockerfile) + - $(build.source.contextDir) + resources: + limits: + cpu: 500m + memory: 1Gi + requests: + cpu: 250m + memory: 65Mi + volumeMounts: + - name: buildah-images + mountPath: /var/lib/containers/storage + - name: buildah-push + image: quay.io/containers/buildah:v1.20.1 + securityContext: + privileged: true + command: + - /usr/bin/buildah + args: + - push + - --tls-verify=false + - docker://$(params.shp-output-image) + resources: + limits: + cpu: 500m + memory: 1Gi + requests: + cpu: 250m + memory: 100Mi <------ See how we provide more memory to step-buildah-push, compared to the 65Mi of the other step +---- + +For the `TaskRun`, as expected we can see the resources on each `step`. + +[,sh] +---- +$ kubectl -n test-build get tr buildah-golang-buildrun-skgrp -o json | jq '.spec.taskSpec.steps[] | select(.name == "step-buildah-bud" ) | .resources' +{ + "limits": { + "cpu": "500m", + "memory": "1Gi" + }, + "requests": { + "cpu": "250m", + "memory": "65Mi" + } +} + +$ kubectl -n test-build get tr buildah-golang-buildrun-skgrp -o json | jq '.spec.taskSpec.steps[] | select(.name == "step-buildah-push" ) | .resources' +{ + "limits": { + "cpu": "500m", + "memory": "1Gi" + }, + "requests": { + "cpu": "250m", + "memory": "100Mi" + } +} +---- + +The pod definition is different, while Tekton will only use the *highest* values of one container, and set the rest(lowest) to zero: + +[,sh] +---- +$ kubectl -n test-build get pods buildah-golang-buildrun-95xq8-pod-mww8d -o json | jq '.spec.containers[] | select(.name == "step-step-buildah-bud" ) | .resources' +{ + "limits": { + "cpu": "500m", + "memory": "1Gi" + }, + "requests": { + "cpu": "250m", <------------------- See how the CPU is preserved + "ephemeral-storage": "0", + "memory": "0" <------------------- See how the memory is set to ZERO + } +} +$ kubectl -n test-build get pods buildah-golang-buildrun-95xq8-pod-mww8d -o json | jq '.spec.containers[] | select(.name == "step-step-buildah-push" ) | .resources' +{ + "limits": { + "cpu": "500m", + "memory": "1Gi" + }, + "requests": { + "cpu": "0", <------------------- See how the CPU is set to zero. + "ephemeral-storage": "0", + "memory": "100Mi" <------------------- See how the memory is preserved on this container + } +} +---- + +In the above scenario, we can see how the maximum numbers for resource requests are distributed between containers. The container `step-buildah-push` gets the `100mi` for the memory requests, while it was the one defining the highest number. At the same time, the container `step-buildah-bud` is assigned a `0` for its memory request. + +''' + +*Scenario 3.* Namespace *with* a `LimitRange`. + +When a `LimitRange` exists on the namespace, `Tekton Pipeline` controller will do the same approach as stated in the above two scenarios. The difference is that for the containers that have lower values, instead of zero, they will get the `minimum values of the LimitRange`. + +== Annotations + +Annotations can be defined for a BuildStrategy/ClusterBuildStrategy as for any other Kubernetes object. Annotations are propagated to the TaskRun and from there, Tekton propagates them to the Pod. Use cases for this are for example: + +* The Kubernetes https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#support-traffic-shaping[Network Traffic Shaping] feature looks for the `kubernetes.io/ingress-bandwidth` and `kubernetes.io/egress-bandwidth` annotations to limit the network bandwidth the `Pod` is allowed to use. +* The https://kubernetes.io/docs/tutorials/clusters/apparmor/[AppArmor profile of a container] is defined using the `container.apparmor.security.beta.kubernetes.io/` annotation. + +The following annotations are not propagated: + +* `kubectl.kubernetes.io/last-applied-configuration` +* `clusterbuildstrategy.shipwright.io/*` +* `buildstrategy.shipwright.io/*` +* `build.shipwright.io/*` +* `buildrun.shipwright.io/*` + +A Kubernetes administrator can further restrict the usage of annotations by using policy engines like https://www.openpolicyagent.org/[Open Policy Agent]. + +== Volumes and VolumeMounts + +Build steps can declare a `volumeMount`, which allows data in the provided path to be shared across build steps. +When a `volumeMount` is declared, Shipwright will create an `emptyDir` volume with the corresponding name. +Build steps whose volume mounts share the same name will share the same underlying `emtpyDir` volume. + +In a future release, build strategy authors will be able to use other volume types for the volume mounts. +When this feature is introduced, the volume and volume type will need to be explicitly declared. diff --git a/docs/configuration.adoc b/docs/configuration.adoc new file mode 100644 index 0000000000..8927ac6eb6 --- /dev/null +++ b/docs/configuration.adoc @@ -0,0 +1,86 @@ +//// +Copyright The Shipwright Contributors + +SPDX-License-Identifier: Apache-2.0 +//// += Configuration + +== Controller Settings + +The controller is installed into Kubernetes with reasonable defaults. However, there are some settings that can be overridden using environment variables in link:../deploy/500-controller.yaml[`controller.yaml`]. + +The following environment variables are available: + +|=== +| Environment Variable | Description + +| `CTX_TIMEOUT` +| Override the default context timeout used for all Custom Resource Definition reconciliation operations. + +| `REMOTE_ARTIFACTS_CONTAINER_IMAGE` +| Specify the container image used for the `.spec.sources` remote artifacts download, by default it uses `busybox:latest`. + +| `GIT_CONTAINER_TEMPLATE` +| JSON representation of a https://pkg.go.dev/k8s.io/api/core/v1#Container[Container] template that is used for steps that clone a Git repository. Default is `{"image":"ghcr.io/shipwright-io/build/git:latest", "command":["/ko-app/git"], "securityContext":{"runAsUser":1000,"runAsGroup":1000}}`. The following properties are ignored as they are set by the controller: `args`, `name`. + +| `GIT_CONTAINER_IMAGE` +| Custom container image for Git clone steps. If `GIT_CONTAINER_TEMPLATE` is also specifying an image, then the value for `GIT_CONTAINER_IMAGE` has precedence. + +| `MUTATE_IMAGE_CONTAINER_TEMPLATE` +| JSON representation of a https://pkg.go.dev/k8s.io/api/core/v1#Container[Container] template that is used for steps that mutates an image if a `Build` has annotations or labels defined in the output. Default is `{"image": "ghcr.io/shipwright-io/build/mutate-image:latest", "command": ["/ko-app/mutate-image"], "env": [{"name": "HOME","value": "/tekton/home"}], "securityContext": {"runAsUser": 0, "capabilities": {"add": ["DAC_OVERRIDE"]}}}`. The following properties are ignored as they are set by the controller: `args`, `name`. + +| `MUTATE_IMAGE_CONTAINER_IMAGE` +| Custom container image that is used for steps that mutates an image if a `Build` has annotations or labels defined in the output. If `MUTATE_IMAGE_CONTAINER_TEMPLATE` is also specifying an image, then the value for `MUTATE_IMAGE_CONTAINER_IMAGE` has precedence. + +| `BUILD_CONTROLLER_LEADER_ELECTION_NAMESPACE` +| Set the namespace to be used to store the `shipwright-build-controller` lock, by default it is in the same namespace as the controller itself. + +| `BUILD_CONTROLLER_LEASE_DURATION` +| Override the `LeaseDuration`, which is the duration that non-leader candidates will wait to force acquire leadership. + +| `BUILD_CONTROLLER_RENEW_DEADLINE` +| Override the `RenewDeadline`, which is the duration that the acting leader will retry refreshing leadership before giving up. + +| `BUILD_CONTROLLER_RETRY_PERIOD` +| Override the `RetryPeriod`, which is the duration the LeaderElector clients should wait between tries of actions. + +| `BUILD_MAX_CONCURRENT_RECONCILES` +| The number of concurrent reconciles by the build controller. A value of 0 or lower will use the default from the https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/controller#Options[controller-runtime controller Options]. Default is 0. + +| `BUILDRUN_MAX_CONCURRENT_RECONCILES` +| The number of concurrent reconciles by the buildrun controller. A value of 0 or lower will use the default from the https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/controller#Options[controller-runtime controller Options]. Default is 0. + +| `BUILDSTRATEGY_MAX_CONCURRENT_RECONCILES` +| The number of concurrent reconciles by the buildstrategy controller. A value of 0 or lower will use the default from the https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/controller#Options[controller-runtime controller Options]. Default is 0. + +| `CLUSTERBUILDSTRATEGY_MAX_CONCURRENT_RECONCILES` +| The number of concurrent reconciles by the clusterbuildstrategy controller. A value of 0 or lower will use the default from the https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/controller#Options[controller-runtime controller Options]. Default is 0. + +| `KUBE_API_BURST` +| Burst to use for the Kubernetes API client. See https://pkg.go.dev/k8s.io/client-go/rest#Config.Burst[Config.Burst]. A value of 0 or lower will use the default from client-go, which currently is 10. Default is 0. + +| `KUBE_API_QPS` +| QPS to use for the Kubernetes API client. See https://pkg.go.dev/k8s.io/client-go/rest#Config.QPS[Config.QPS]. A value of 0 or lower will use the default from client-go, which currently is 5. Default is 0. + +| `TERMINATION_LOG_PATH` +| Path of the termination log. This is where controller application will write the reason of its termination. Default value is `/dev/termination-log`. + +| `GIT_ENABLE_REWRITE_RULE` +| Enable Git wrapper to setup a URL `insteadOf` Git config rewrite rule for the respective source URL hostname. Default is `false`. +|=== + +== Role-based Access Control + +The release deployment YAML file includes two cluster-wide roles for using Shipwright Build objects. +The following roles are installed: + +* `shpwright-build-aggregate-view`: this role grants read access (get, list, watch) to most Shipwright Build objects. +This includes `BuildStrategy`, `ClusterBuildStrategy`, `Build`, and `BuildRun` objects. +This role is aggregated to the https://kubernetes.io/docs/reference/access-authn-authz/rbac/#default-roles-and-role-bindings[Kubernetes "view" role]. +* `shipwright-build-aggregate-edit`: this role grants write access (create, update, patch, delete) to Shipwright objects that are namespace-scoped. +This includes `BuildStrategy`, `Builds`, and `BuildRuns`. +Read access is granted to all `ClusterBuildStrategy` objects. +This role is aggregated to the https://kubernetes.io/docs/reference/access-authn-authz/rbac/#default-roles-and-role-bindings[Kubernetes "edit" and "admin" roles]. + +Only cluster administrators are granted write access to `ClusterBuildStrategy` objects. +This can be changed by creating a separate https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole[Kubernetes `ClusterRole`] with these permissions and binding the role to appropriate users. diff --git a/docs/metrics.adoc b/docs/metrics.adoc new file mode 100644 index 0000000000..98d78af940 --- /dev/null +++ b/docs/metrics.adoc @@ -0,0 +1,161 @@ +//// +Copyright The Shipwright Contributors + +SPDX-License-Identifier: Apache-2.0 +//// += Build Controller Metrics + +The Build component exposes several metrics to help you monitor the health and behavior of your build resources. + +Following build metrics are exposed on port `8383`. + +|=== +| Name | Type | Description | Labels | Status + +| `build_builds_registered_total` +| Counter +| Number of total registered Builds. +| buildstrategy=++++++^1^ + +namespace=++++++^1^ + +build=++++++^1^++++++++++++++++++ +| experimental + +| `build_buildruns_completed_total` +| Counter +| Number of total completed BuildRuns. +| buildstrategy=++++++^1^ + +namespace=++++++^1^ + +build=++++++^1^ + +buildrun=++++++^1^++++++++++++++++++++++++ +| experimental + +| `build_buildrun_establish_duration_seconds` +| Histogram +| BuildRun establish duration in seconds. +| buildstrategy=++++++^1^ + +namespace=++++++^1^ + +build=++++++^1^ + +buildrun=++++++^1^++++++++++++++++++++++++ +| experimental + +| `build_buildrun_completion_duration_seconds` +| Histogram +| BuildRun completion duration in seconds. +| buildstrategy=++++++^1^ + +namespace=++++++^1^ + +build=++++++^1^ + +buildrun=++++++^1^++++++++++++++++++++++++ +| experimental + +| `build_buildrun_rampup_duration_seconds` +| Histogram +| BuildRun ramp-up duration in seconds +| buildstrategy=++++++^1^ + +namespace=++++++^1^ + +build=++++++^1^ + +buildrun=++++++^1^++++++++++++++++++++++++ +| experimental + +| `build_buildrun_taskrun_rampup_duration_seconds` +| Histogram +| BuildRun taskrun ramp-up duration in seconds. +| buildstrategy=++++++^1^ + +namespace=++++++^1^ + +build=++++++^1^ + +buildrun=++++++^1^++++++++++++++++++++++++ +| experimental + +| `build_buildrun_taskrun_pod_rampup_duration_seconds` +| Histogram +| BuildRun taskrun pod ramp-up duration in seconds. +| buildstrategy=++++++^1^ + +namespace=++++++^1^ + +build=++++++^1^ + +buildrun=++++++^1^++++++++++++++++++++++++ +| experimental +|=== + +^1^ Labels for metric are disabled by default. See <> to enable them. + +== Configuration of histogram buckets + +Environment variables can be set to use custom buckets for the histogram metrics: + +|=== +| Metric | Environment variable | Default + +| `build_buildrun_establish_duration_seconds` +| `PROMETHEUS_BR_EST_DUR_BUCKETS` +| `0,1,2,3,5,7,10,15,20,30` + +| `build_buildrun_completion_duration_seconds` +| `PROMETHEUS_BR_COMP_DUR_BUCKETS` +| `50,100,150,200,250,300,350,400,450,500` + +| `build_buildrun_rampup_duration_seconds` +| `PROMETHEUS_BR_RAMPUP_DUR_BUCKETS` +| `0,1,2,3,4,5,6,7,8,9,10` + +| `build_buildrun_taskrun_rampup_duration_seconds` +| `PROMETHEUS_BR_RAMPUP_DUR_BUCKETS` +| `0,1,2,3,4,5,6,7,8,9,10` + +| `build_buildrun_taskrun_pod_rampup_duration_seconds` +| `PROMETHEUS_BR_RAMPUP_DUR_BUCKETS` +| `0,1,2,3,4,5,6,7,8,9,10` +|=== + +The values have to be a comma-separated list of numbers. You need to set the environment variable for the build controller for your customization to become active. When running locally, set the variable right before starting the controller: + +[,bash] +---- +export PROMETHEUS_BR_COMP_DUR_BUCKETS=30,60,90,120,180,240,300,360,420,480 +make local +---- + +When you deploy the build controller in a Kubernetes cluster, you need to extend the `spec.containers[0].spec.env` section of the sample deployment file, link:../deploy/500-controller.yaml[controller.yaml]. Add an additional entry: + +[,yaml] +---- +[...] + env: + - name: PROMETHEUS_BR_COMP_DUR_BUCKETS + value: "30,60,90,120,180,240,300,360,420,480" +[...] +---- + +== Configuration of metric labels + +As the amount of buckets and labels has a direct impact on the number of Prometheus time series, you can selectively enable labels that you are interested in using the `PROMETHEUS_ENABLED_LABELS` environment variable. The supported labels are: + +* buildstrategy +* namespace +* build +* buildrun + +Use a comma-separated value to enable multiple labels. For example: + +[,bash] +---- +export PROMETHEUS_ENABLED_LABELS=namespace +make local +---- + +or + +[,bash] +---- +export PROMETHEUS_ENABLED_LABELS=buildstrategy,namespace,build +make local +---- + +When you deploy the build controller in a Kubernetes cluster, you need to extend the `spec.containers[0].spec.env` section of the sample deployment file, link:../deploy/controller.yaml[controller.yaml]. Add an additional entry: + +[,yaml] +---- +[...] + env: + - name: PROMETHEUS_ENABLED_LABELS + value: namespace +[...] +---- diff --git a/docs/profiling.adoc b/docs/profiling.adoc new file mode 100644 index 0000000000..4b652f00c1 --- /dev/null +++ b/docs/profiling.adoc @@ -0,0 +1,46 @@ +//// +Copyright The Shipwright Contributors + +SPDX-License-Identifier: Apache-2.0 +//// += Build Controller Profiling + +The build controller supports a `pprof` profiling mode, which is omitted from the binary by default. To use the profiling, use the controller image that was built with `pprof` enabled. + +== Enable `pprof` in the build controller + +In the Kubernetes cluster, edit the `shipwright-build-controller` deployment to use the container tag with the `debug` suffix. + +[,sh] +---- +kubectl --namespace set image \ + deployment/shipwright-build-controller \ + shipwright-build-controller="$(kubectl --namespace get deployment shipwright-build-controller --output jsonpath='{.spec.template.spec.containers[].image}')-debug" +---- + +== Connect `go pprof` to build controller + +Depending on the respective setup, there could be multiple build controller pods for high availability reasons. In this case, you have to look-up the current leader first. The following command can be used to verify the currently active leader: + +[,sh] +---- +kubectl --namespace get configmap shipwright-build-controller-lock --output json \ + | jq --raw-output '.metadata.annotations["control-plane.alpha.kubernetes.io/leader"]' \ + | jq --raw-output .holderIdentity +---- + +The `pprof` endpoint is not exposed in the cluster and can only be used from inside the container. Therefore, set-up port-forwarding to make the `pprof` port available locally. + +[,sh] +---- +kubectl --namespace port-forward 8383:8383 +---- + +Now, you can setup a local webserver to browse through the profiling data. + +[,sh] +---- +go tool pprof -http localhost:8080 http://localhost:8383/debug/pprof/heap +---- + +_Please note:_ For it to work, you have to have `graphviz` installed on your system, for example using `brew install graphviz`, `apt-get install graphviz`, `yum install graphviz`, or similar.