Merge pull request #16 from Michkov/task-bundles

Task bundles and release
konflux-ci · Jan 28, 2022 · 5abb137 · 5abb137
2 parents 3fe3c8b + f7f7cc3
commit 5abb137
Show file tree

Hide file tree

Showing 34 changed files with 413 additions and 402 deletions.
diff --git a/README.md b/README.md
@@ -2,34 +2,36 @@
 
 This repository contains components that are installed or managed by the managed CI and Build Team.
 
-This includes default Pipelines and Tasks. You need to have bootstrapped a working appstudio configuration from (see `https://github.com/redhat-appstudio/infra-deployments`) for the dev of pipelines or new tasks. 
+This includes default Pipelines and Tasks. You need to have bootstrapped a working appstudio configuration from (see `https://github.com/redhat-appstudio/infra-deployments`) for the dev of pipelines or new tasks.
 
 Pipelines are delivered into App Studio via `quay.io/redhat-appstudio/build-templates-bundle:v0.1.3` (the tag will be updated every change)
 
-Tasks are delivered into App Studio via Cluster tasks installedfrom `https://github.com/redhat-appstudio/infra-deployments/blob/main/components/build/clustertasks` 
-App Studio specific cluster tasks will be found in this repository. Currently a set of utilties are bundled with App Studio in `quay.io/redhat-appstudio/appstudio-utils:v0.1.3` as a convenience but tasks may be run from different per-task containers in future.
+Tasks are delivered into App Studio via `quay.io/redhat-appstudio/appstudio-tasks`. Where each task is bundled and pushed into tag in format `${TASK_NAME}-${VERSION}` where version is the same as pipelines bundle version.
 
-## Devmode for Pipelines 
+Currently a set of utilties are bundled with App Studio in `quay.io/redhat-appstudio/appstudio-utils:v0.1.3` as a convenience but tasks may be run from different per-task containers in future.
 
-The pipelines can be found in the `pipelines` directories. 
+## Devmode
 
-Once your configuration is set you can modify pipelines installed via this repository in two ways.
+Script `dev.sh` creates bundles for pipelines, each task and create appstudio-utils image. Images are pushed into your quay.io repository. You will need to set `MY_QUAY_USER` to use this feature and be logged into quay.io on your workstation.
+Once you run the `dev.sh` all pipelines will come from your bundle instead of from the default installed by gitops into the cluster.
 
-### Override mode. 
-Every time you run `dev-mode.sh`, it will take the current directory and package into a bundle into your own quay.io repository. You will need to set `MY_QUAY_USER` to use this feature and be logged into quay.io on your workstation.
-Once you run the `dev-mode.sh` all pipelines will come from your bundle instead of from the default installed by gitops into the cluster.  
-### Gitops Mode
-Replace the file `https://github.com/redhat-appstudio/infra-deployments/blob/main/components/build/build-templates/bundle-config.yaml` in your own fork (dev mode). This will sync to the cluster and all builds-definitions will come from the bundle you configure. 
+### Pipelines
 
-Please test in _gitops mode_ when doing a new release into staging as it will be the best way to ensure that the deployment will function correctly when deployed via gitops. 
+The pipelines can be found in the `pipelines` directories.
 
-Releasing new bundles are currently manual. (TODO, CI will automatically publish updates `infra-deployments`) via a SHA/pull request. 
+#### Gitops Mode
 
-## Devmode for Tasks 
+Replace the file `https://github.com/redhat-appstudio/infra-deployments/blob/main/components/build/build-templates/bundle-config.yaml` in your own fork (dev mode). This will sync to the cluster and all builds-definitions will come from the bundle you configure.
 
-The tasks can be found in the `tasks` directories. Replacing tasks in App Studio is more complex as we will currently  deliver tasks as ClusterTasks. See `https://github.com/redhat-appstudio/infra-deployments/blob/main/components/build/` to install new app studio cluster tasks via the gitops delivery mode. 
-For quick local innerloop style task development, you may install new Tasks in your local namespace manually and create your pipelines as well as the base task image to test new function. This is a manual process which requires you create Tasks and use in your namespace. This may be faster but requires you to later change the task type to ClusterTask and release in the infra deployment directory.
+Please test in _gitops mode_ when doing a new release into staging as it will be the best way to ensure that the deployment will function correctly when deployed via gitops.
 
-There is a container which is used to support multiple set of tasks called `quay.io/redhat-appstudio/appstudio-utils:v0.1.3` , which is a single container which is used by multiple tasks. Tasks may also be in their own container as well however many simple tasks are utiltities and will be packaged for app studio in a single container. Tasks can rely on other tasks in the system which are co-packed in a container allowing combined tasks (build-only vs build-deploy) which use the same core implementations.
+### Tasks
 
+The tasks can be found in the `tasks` directories. Tasks are bundled and used by bundled pipelines. Tasks are not stored in the Cluster.
+For quick local innerloop style task development, you may install new Tasks in your local namespace manually and create your pipelines as well as the base task image to test new function. Tasks can be installed into local namespace using `oc apply -k tasks/appstudio-utils/util-tasks`.
 
+There is a container which is used to support multiple set of tasks called `quay.io/redhat-appstudio/appstudio-utils:v0.1.3` , which is a single container which is used by multiple tasks. Tasks may also be in their own container as well however many simple tasks are utilities and will be packaged for app studio in a single container. Tasks can rely on other tasks in the system which are co-packed in a container allowing combined tasks (build-only vs build-deploy) which use the same core implementations.
+
+## Release
+
+Prerequisite for new release is creation of tag or release in GitHub repository. After that pull changes and execute `./release.sh $TAG` where `TAG` must match with git tag of current branch.
diff --git a/dev.sh b/dev.sh
@@ -1,2 +1,18 @@
-./pipelines/dev-build.sh 
-(cd ./tasks/appstudio-utils; ./dev-build.sh)
+#!/bin/bash
+
+export BUILD_TAG=$1
+
+if [ -z "$MY_QUAY_USER" ]; then
+    echo "MY_QUAY_USER environment variable must be set"
+    exit 1
+fi
+if [ "$MY_QUAY_USER" = "redhat-appstudio" ]; then
+    echo "Cannot use devmode as redhat-appstudio user "
+    exit 1
+fi
+if [ -z "$BUILD_TAG" ]; then
+    export BUILD_TAG=$(date +"%Y-%m-%d-%H%M%S")
+fi
+
+./tasks/appstudio-utils/dev-build.sh
+./pipelines/dev-build.sh
diff --git a/pipelines/README.md b/pipelines/README.md
@@ -1,11 +1,3 @@
 # build-definitions
 
 This directory the OCI bundle with default pipelines installed for build-definitions.
-
-Update `pipelines/release-build.sh` to set the release tag to a specific version.  
-
-Currently set manually (v0.1, v0.1.1), but when integrated into a CI, will shift to use git commit SHA. 
-
-This has to be updated on the infra-deployment cluster when the version is updated.
-
-
diff --git a/pipelines/build-templates-bundle/devfile-build.yaml b/pipelines/build-templates-bundle/devfile-build.yaml
@@ -1,11 +1,11 @@
 apiVersion: tekton.dev/v1beta1
 kind: Pipeline
 metadata:
-  name: devfile-build  
+  name: devfile-build
   labels:
     "pipelines.openshift.io/used-by" : "build-cloud"
     "pipelines.openshift.io/runtime" : "generic"
-    "pipelines.openshift.io/strategy" : "docker" 
+    "pipelines.openshift.io/strategy" : "docker"
 spec:
   params:
     - description: 'Source Repository URL'
@@ -19,7 +19,7 @@ spec:
       name: output-image
       type: string
     - description: The path to your source code
-      name: path-context 
+      name: path-context
       type: string
       default: .
     - description: Path to the Dockerfile
@@ -28,19 +28,19 @@ spec:
       default: Dockerfile
   tasks:
     - name: appstudio-init
-      params: 
-        - name: image-url 
+      params:
+        - name: image-url
           value: "$(params.output-image)"
       taskRef:
-        kind: ClusterTask
-        name: appstudio-init
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:init-${TAG}
+        name: init
       workspaces:
         - name: source
-          workspace: workspace  
+          workspace: workspace
         - name: registry-auth
-          workspace: registry-auth  
-        - name: git-auth 
-          workspace: git-auth   
+          workspace: registry-auth
+        - name: git-auth
+          workspace: git-auth
     - name: clone-repository
       runAfter:
         -  appstudio-init
@@ -59,28 +59,28 @@ spec:
       runAfter:
         - clone-repository
       taskRef:
-        kind: ClusterTask
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:analyze-devfile-${TAG}
         name: analyze-devfile
       workspaces:
         - name: source
           workspace: workspace
     - name: appstudio-configure-build
       runAfter:
-        - analyze-devfile 
+        - analyze-devfile
       when:
-      - input: $(tasks.appstudio-init.results.exists) 
+      - input: $(tasks.appstudio-init.results.exists)
         operator: in
-        values: ["false"] 
+        values: ["false"]
       taskRef:
-        kind: ClusterTask
-        name: appstudio-configure-build
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:configure-build-${TAG}
+        name: configure-build
       workspaces:
         - name: source
-          workspace: workspace  
+          workspace: workspace
         - name: registry-auth
-          workspace: registry-auth  
-        - name: git-auth 
-          workspace: git-auth      
+          workspace: registry-auth
+        - name: git-auth
+          workspace: git-auth
     - name: build-container
       params:
         - name: IMAGE
@@ -103,51 +103,51 @@ spec:
           value: "$(tasks.appstudio-configure-build.results.buildah-auth-param)"
         - name: PUSH_EXTRA_ARGS
           value: "$(tasks.appstudio-configure-build.results.buildah-auth-param)"
-      runAfter: 
+      runAfter:
         - appstudio-configure-build
       taskRef:
         kind: ClusterTask
         name: buildah
       workspaces:
         - name: source
-          workspace: workspace 
+          workspace: workspace
     - name: show-summary
       runAfter:
-        - build-container 
+        - build-container
       taskRef:
-        kind: ClusterTask
-        name: appstudio-summary
-      params: 
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:summary-${TAG}
+        name: summary
+      params:
       - name: pipeline-run-name
         value: "$(context.pipelineRun.name)"
       - name: git-url
         value: "$(params.git-url)"
       - name: image-url
-        value: $(params.output-image) 
+        value: $(params.output-image)
       - name: deploy-yaml
-        value: $(tasks.analyze-devfile.results.deploy) 
-    - name: skip-rebuild-summary 
+        value: $(tasks.analyze-devfile.results.deploy)
+    - name: skip-rebuild-summary
       when:
-      - input: $(tasks.appstudio-init.results.exists) 
+      - input: $(tasks.appstudio-init.results.exists)
         operator: in
-        values: ["true"]  
+        values: ["true"]
       runAfter:
         - analyze-devfile
       taskRef:
-        kind: ClusterTask
-        name: appstudio-summary
-      params: 
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:summary-${TAG}
+        name: summary
+      params:
       - name: pipeline-run-name
         value: "$(context.pipelineRun.name)"
       - name: git-url
         value: "$(params.git-url)"
       - name: image-url
-        value: $(params.output-image) 
+        value: $(params.output-image)
       - name: deploy-yaml
         value: $(tasks.analyze-devfile.results.deploy)
   workspaces:
     - name: workspace
     - name: registry-auth
       optional: true
-    - name: git-auth 
-      optional: true
+    - name: git-auth
+      optional: true
diff --git a/pipelines/build-templates-bundle/docker-build.yaml b/pipelines/build-templates-bundle/docker-build.yaml
@@ -32,8 +32,8 @@ spec:
         - name: image-url 
           value: "$(params.output-image)"
       taskRef:
-        kind: ClusterTask
-        name: appstudio-init
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:init-${TAG}
+        name: init
       workspaces:
         - name: source
           workspace: workspace  
@@ -67,8 +67,8 @@ spec:
         operator: in
         values: ["false"] 
       taskRef:
-        kind: ClusterTask
-        name: appstudio-configure-build
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:configure-build-${TAG}
+        name: configure-build
       workspaces:
         - name: source
           workspace: workspace  
@@ -110,8 +110,8 @@ spec:
       runAfter:
         - build-container 
       taskRef:
-        kind: ClusterTask
-        name: appstudio-summary
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:summary-${TAG}
+        name: summary
       params: 
       - name: pipeline-run-name
         value: "$(context.pipelineRun.name)"
@@ -127,8 +127,8 @@ spec:
       runAfter:
         - appstudio-init
       taskRef:
-        kind: ClusterTask
-        name: appstudio-summary
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:summary-${TAG}
+        name: summary
       params:
       - name: pipeline-run-name
         value: "$(context.pipelineRun.name)"
@@ -141,4 +141,4 @@ spec:
     - name: registry-auth
       optional: true
     - name: git-auth 
-      optional: true
+      optional: true
diff --git a/pipelines/build-templates-bundle/java-builder.yaml b/pipelines/build-templates-bundle/java-builder.yaml
@@ -38,8 +38,8 @@ spec:
         - name: image-url 
           value: "$(params.output-image)"
       taskRef:
-        kind: ClusterTask
-        name: appstudio-init
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:init-${TAG}
+        name: init
       workspaces:
         - name: source
           workspace: workspace  
@@ -84,8 +84,8 @@ spec:
       runAfter:
         - git-clone 
       taskRef:
-        kind: ClusterTask
-        name: appstudio-configure-build
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:configure-build-${TAG}
+        name: configure-build
       workspaces:
         - name: source
           workspace: workspace  
@@ -120,8 +120,8 @@ spec:
       runAfter:
         - s2i-java 
       taskRef:
-        kind: ClusterTask
-        name: appstudio-summary
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:summary-${TAG}
+        name: summary
       params:
       - name: pipeline-run-name
         value: "$(context.pipelineRun.name)" 
@@ -137,8 +137,8 @@ spec:
       runAfter:
         - appstudio-init
       taskRef:
-        kind: ClusterTask
-        name: appstudio-summary
+        bundle: ${CONTAINER_REPO}/appstudio-tasks:summary-${TAG}
+        name: summary
       params:
       - name: pipeline-run-name
         value: "$(context.pipelineRun.name)" 
@@ -151,4 +151,4 @@ spec:
     - name: registry-auth
       optional: true
     - name: git-auth 
-      optional: true
+      optional: true