feature request: running pipelines locally

[BenTheElder]

Hello from kubernetes/test-infra! 👋 -- We think the pipeline is really interesting and we'd like to use it more, and we (I) have a feature request which we get a lot with our current "jobs" system:

how do I test this [job definition] on my machine

and

how do I run this test job [to test code for some repo using the jobs for ci]

We've considered solving this for "ProwJobs" and while looking at prior art I found https://github.com/GoogleCloudPlatform/cloud-build-local, which has an implementation of this for GCB which looks a bit similar ...

I'd like to start a discussion around how we could potentially accomplish this for the build pipeline.

For the first case just being able to run it locally in some form is sufficient, for the second case users would likely want to be able to mount not-yet-pushed code to the job and build / test that.

If you all are interested, I'd like to help with this.

I think it could be a "killer" feature to be able to iterate on running and testing your containerized pipelines locally before submitting them to whatever CI you will use in production. I know Prow users would love this.

cc @bobcatfish

[imjasonh]

I think we should recommend minikube for local development/iteration of pipelines, and have good documentation to cover this use case.

cloud-build-local effectively packages GCB's custom container orchestration into a locally runnable binary, but if I could go back in time I'd prefer to use K8s' container orchestration rather than building a minimal bespoke one ourselves. (This repo hopefully moves us toward resolving this long-standing annoyance 😉)

If minikube is too heavy a dependency to take, it might be possible to do minimal local orchestration with docker run calls, but then the prod implementation won't match the local development implementation, so you're not really buying much.

[BenTheElder]

... and have good documentation to cover this use case.

That would be great.

If minikube is too heavy a dependency to take, it might be possible to do minimal local orchestration with docker run calls, ...

That's what I was thinking. Running a cluster is a little bit heavy, this is coming from someone working on another "light" local cluster (sigs.k8s.io/kind).

... but then the prod implementation won't match the local development implementation, so you're not really buying much.

Your minikube deployment won't look quite like prod either. 🙃
On the other hand, getting your test / build tools / scripts running in a container and figuring out how to write a pipeline spec is pretty different vs running them without a pipeline or a container.

I understand not wanting to implement that though :-)

---

To the other point, what is the expectation around iterating on the code under build / test rather than the pipeline? This seems further heavy handed at the moment, it looks like we need to tell people to push their code first and then tweak the pipeline to download their branch / fork instead ...?

[bobcatfish]

To the other point, what is the expectation around iterating on the code under build / test rather than the pipeline? This seems further heavy handed at the moment, it looks like we need to tell people to push their code first and then tweak the pipeline to download their branch / fork instead ...?

That's a great point, I think this is
a) not something we've thought through yet and also
b) way too much to ask folks to push their code first

We should definitely push for being able to try this locally without pushing anything!

As far as our roadmap is concerned we probably won't have this (#235) prioritized for our current milestone but I think we could look at it for the next one! (Unless one of our contributors wants to take a look in the meantime!!)

[bobcatfish]

@ndeloof's https://github.com/ndeloof/kitten could be a super promising answer to part of this :D

[bobcatfish]

Quick update: I think this is looking more and more promising as we add features like workspaces and also as the CLI matures - the CLI could be an interesting way to explore a local execution mode (+ @sbwsg )

[carltonmason]

FYI, I've been happily running Tekton locally on my Mac with Rancher's k3d.

k3d is very lightweight compared to minikube.

I recently created a gist for how I set it up https://gist.github.com/carltonmason/72a494080887175245a8abb7d3dc14f7.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bobcatfish]

This is currently in our roadmap but also under debate in tektoncd/community#145

/lifecycle frozen

[jerop]

There's a solution from @vdemeester - https://github.com/vdemeester/buildkit-tekton

Maybe we can consider this done after adopting the project? tektoncd/community#674
@vdemeester can we discuss the adoption of buildkit in the next governance/community meeting?

[vdemeester]

@jerop I need to update the project a bit, but yes, this is definitely something we could discuss 👼🏼