Keep remote pipelines up to date documentation #173

fabriziosta · 2024-11-18T12:55:37Z

No description provided.

arewm

Some questions asked to start discussions. This topic also seems like it might fit better within the "workflow" section similar to the git submodule approach.

docs/modules/ROOT/pages/advanced-how-tos/keep-remote-pipelines-up-to-date.adoc

arewm · 2024-11-18T14:00:26Z

docs/modules/ROOT/pages/advanced-how-tos/keep-remote-pipelines-up-to-date.adoc

+├── .tekton
+│   ├── <repository-name>-pipelines-pull-request.yaml # Used by Konflux to listen to pull requests
+│   └── <repository-name>-pipelines-push.yaml # Used by Konflux to listen to pushes
+├── pipelines
+│   ├── remote-pipeline1.yaml
+│   └── remote-pipeline2.yaml
+├── tasks
+│   ├── remote-task1.yaml
+│   └── remote-task2.yaml


What is the difference between all of these files? What should go into each of these?

The several remote pipelines that could be run from different components. I.E.

multi aarch pipeline

hermetic pipeline

pipeline with disabled checks or a very few tasks in it

pipeline with pre-fetch enabled

custom pipeline with untrusted Tasks
and so on.

github-actions · 2024-11-21T16:33:27Z

🚀 Preview is available at https://pr-173--konflux-docs.netlify.app

arewm

I like how this page is shaping up!

Instead of using Konflux directly, we use {ProductName}.

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

arewm · 2024-11-21T17:08:24Z

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

+
+. Pipelines are stored in a Git repository and fetched during execution, eliminating the need to maintain local or static pipeline definitions.
+. Teams can leverage Git’s versioning to access specific pipeline branches
+. In case of issues, it's very simple to rollback to a previous pipeline version, and apply it globally among the Konflux Components.


It is possible to roll back to previous versions without this remote pipeline strategy as well. I guess that the suggestion is to rollback the reference pipeline commit based on the use of a floating branch tag? If you are suggesting that it is easy to roll back, I think we should document that in this page.

I guess that the suggestion is to rollback the reference pipeline commit based on the use of a floating branch tag?
Yup, that's exactly what I meant. Let me specify it better

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

arewm · 2024-11-21T17:18:14Z

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

+
+== The solution
+
+To workaround the above issue, the suggested approach is to onboard the repository where the shared pipelines are stored as a Konflux Component, so that Konflux and MintMaker will be able to discover and update digests for us.


Should we give this location of the shared pipelines a name to make it easier to refer to? For example a common pipeline component or reference component (I don't think that my naming suggestions are that good right now).

Could you rephrase your idea? I am not sure I get it

arewm · 2024-11-21T17:18:47Z

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

+.*Prerequisites*
+
+* Having a repository where to store shared Pipelines and Tasks.


Is this not a prerequisite for the entire workflow? Or is it possible to achieve this workflow without this?

Yeah it's a prerequisite for the entire workflow, should I move it under another section or paragraph?

arewm · 2024-11-21T17:24:10Z

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

@@ -0,0 +1,107 @@
+= Maintaining remote Tekton pipelines up to date


General comment on structure of this page. If we create a structure similar to the git submodule approach, I think that this page would have sections like:

(brief description of the problem we are trying to solve)

Implementing this guideline

Basic definitions

Create a reference component (see comment about naming the component)

Configure renovate

Configure additional components with the remote resolver

Benefits of this guideline

Potential drawbacks

arewm · 2024-11-21T17:25:49Z

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

+├── pipelines
+│   ├── remote-pipeline1.yaml
+│   └── remote-pipeline2.yaml


I feel like the general recommendation might be to have a reference component build for each of these pipelines (i.e. if you have a single arch and multi-arch pipeline, one component that exercises each in a way that is representative for your other components). This would enable the CI for this repo to catch when task/pipeline updates will present issues downstream.

Are you suggesting to rename the pipelines with some more descriptive namings?

arewm · 2024-11-21T17:26:24Z

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

+│   ├── <repository-name>-pipelines-pull-request.yaml # Used by Konflux to listen to pull requests
+│   └── <repository-name>-pipelines-push.yaml # Used by Konflux to listen to pushes


These PipelineRun files should reference the common pipelines (and by extension common tasks) below, right?

No they don't need to reference the common pipelines. I mean, they probably could reference them as the repo itself it's a Component. But for the sake of the documentation I would like to keep them separated, as it will be more clear to the users the difference between shared pipelines and non-shared ones, wdyt?

…s-up-to-date.adoc Co-authored-by: Andrew McNamara <[email protected]>

swadeley · 2024-12-05T03:25:09Z

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

+
+.*Prerequisites*
+
+* Having a repository where to store shared Pipelines and Tasks.


Suggested change

* Having a repository where to store shared Pipelines and Tasks.

* Having a repository to store shared Pipelines and Tasks.

swadeley · 2024-12-05T03:28:46Z

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

+* MintMaker will read the default configuration from "https://docs.renovatebot.com/renovate-schema.json" and will extend it using the subsequent custom settings.
+====
+
+. xref:how-tos/creating.adoc[Create a Konflux Application and a Konflux Component] where to reference the repository and branch name.


Suggested change

. xref:how-tos/creating.adoc[Create a Konflux Application and a Konflux Component] where to reference the repository and branch name.

. xref:how-tos/creating.adoc[Create a Konflux Application and a Konflux Component] to reference the repository and branch name.

swadeley · 2024-12-05T03:33:08Z

docs/modules/ROOT/pages/how-tos/_nav.adoc

@@ -35,3 +35,4 @@
 ** xref:how-tos/deleting.adoc[Deleting applications and components]
 ** xref:how-tos/workflows/index.adoc[Guidelines for {ProductName} integrated repositories]
 *** xref:how-tos/workflows/git-submodules.adoc[Building upstream projects with git submodules]
+*** xref:how-tos/workflows/keep-remote-pipelines-up-to-date.adoc[Maintaining remote pipelines up to date]


Suggested change

*** xref:how-tos/workflows/keep-remote-pipelines-up-to-date.adoc[Maintaining remote pipelines up to date]

*** xref:how-tos/workflows/keep-remote-pipelines-up-to-date.adoc[Maintaining remote pipelines]

I think "Maintaining" implies "keeping up to date"

swadeley · 2024-12-05T03:33:47Z

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

@@ -0,0 +1,107 @@
+= Maintaining remote Tekton pipelines up to date


Suggested change

= Maintaining remote Tekton pipelines up to date

= Maintaining remote Tekton pipelines

reason as above

swadeley · 2024-12-05T03:34:53Z

docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipelines-up-to-date.adoc

+
+[IMPORTANT]
+====
+* This article assumed knowledge of link:https://tekton.dev/docs/pipelines/resolution/[Tekton remote pipelines]. If this is not the case, please read more about this feature before proceeding.


Suggested change

* This article assumed knowledge of link:https://tekton.dev/docs/pipelines/resolution/[Tekton remote pipelines]. If this is not the case, please read more about this feature before proceeding.

* This article assumes knowledge of link:https://tekton.dev/docs/pipelines/resolution/[Tekton remote pipelines]. If this is not the case, please read more about this feature before proceeding.

fabriziosta force-pushed the add-remote-pipeline-doc branch from 176a5b3 to bc754cf Compare November 18, 2024 13:09

fabriziosta marked this pull request as ready for review November 18, 2024 13:09

arewm reviewed Nov 18, 2024

View reviewed changes

Keep remote pipelines up to date documentation

497673c

fabriziosta force-pushed the add-remote-pipeline-doc branch from bc754cf to 497673c Compare November 19, 2024 21:53

arewm reviewed Nov 21, 2024

View reviewed changes

fabriziosta and others added 4 commits November 22, 2024 20:11

Update docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipeline…

da822cf

…s-up-to-date.adoc Co-authored-by: Andrew McNamara <[email protected]>

Update docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipeline…

5a98a73

…s-up-to-date.adoc Co-authored-by: Andrew McNamara <[email protected]>

Update docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipeline…

33d4edb

…s-up-to-date.adoc Co-authored-by: Andrew McNamara <[email protected]>

Update docs/modules/ROOT/pages/how-tos/workflows/keep-remote-pipeline…

041411a

…s-up-to-date.adoc Co-authored-by: Andrew McNamara <[email protected]>

swadeley reviewed Dec 5, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Keep remote pipelines up to date documentation #173

Keep remote pipelines up to date documentation #173

fabriziosta commented Nov 18, 2024

arewm left a comment

arewm Nov 18, 2024

fabriziosta Nov 18, 2024

github-actions bot commented Nov 21, 2024

arewm left a comment

arewm Nov 21, 2024

fabriziosta Nov 25, 2024

arewm Nov 21, 2024

fabriziosta Nov 25, 2024

arewm Nov 21, 2024

fabriziosta Nov 25, 2024

arewm Nov 21, 2024

arewm Nov 21, 2024

fabriziosta Nov 25, 2024

arewm Nov 21, 2024

fabriziosta Nov 25, 2024

swadeley Dec 5, 2024

swadeley Dec 5, 2024

swadeley Dec 5, 2024

swadeley Dec 5, 2024

swadeley Dec 5, 2024


		== The solution

		To workaround the above issue, the suggested approach is to onboard the repository where the shared pipelines are stored as a Konflux Component, so that Konflux and MintMaker will be able to discover and update digests for us.

		.Prerequisites

		* Having a repository where to store shared Pipelines and Tasks.

		@@ -0,0 +1,107 @@
		= Maintaining remote Tekton pipelines up to date

		│ ├── <repository-name>-pipelines-pull-request.yaml # Used by Konflux to listen to pull requests
		│ └── <repository-name>-pipelines-push.yaml # Used by Konflux to listen to pushes

	* Having a repository where to store shared Pipelines and Tasks.
	* Having a repository to store shared Pipelines and Tasks.

	. xref:how-tos/creating.adoc[Create a Konflux Application and a Konflux Component] where to reference the repository and branch name.
	. xref:how-tos/creating.adoc[Create a Konflux Application and a Konflux Component] to reference the repository and branch name.

	*** xref:how-tos/workflows/keep-remote-pipelines-up-to-date.adoc[Maintaining remote pipelines up to date]
	*** xref:how-tos/workflows/keep-remote-pipelines-up-to-date.adoc[Maintaining remote pipelines]

	= Maintaining remote Tekton pipelines up to date
	= Maintaining remote Tekton pipelines

	* This article assumed knowledge of link:https://tekton.dev/docs/pipelines/resolution/[Tekton remote pipelines]. If this is not the case, please read more about this feature before proceeding.
	* This article assumes knowledge of link:https://tekton.dev/docs/pipelines/resolution/[Tekton remote pipelines]. If this is not the case, please read more about this feature before proceeding.

Keep remote pipelines up to date documentation #173

Are you sure you want to change the base?

Keep remote pipelines up to date documentation #173

Conversation

fabriziosta commented Nov 18, 2024

arewm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Nov 21, 2024

arewm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment