Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the time-to-deploy 2x quicker: 2mins vs 4mins #522

Merged
merged 1 commit into from
Sep 1, 2024

Conversation

gerhard
Copy link
Member

@gerhard gerhard commented Sep 1, 2024

First run, empty cache (first time penalty)

image image image

Second run, primed cache

image image image

This replaces our Dagger on Fly.io setup with Dagger on Namespace.so

We still keep K8s to see if we can improve on Namespace.so, the new Speed King 🏎️

This is the first part of the required change. The second part will follow after this gets merged. We need to:

  • Cleanup secrets & variables from GitHub Actions
  • De-provision the Dagger Engine on Fly.io
  • Remove the deploy config from fly.io dir
  • Update the INFRASTRUCTURE diagram

Before this change, throughout August 2024:

image

image

This replaces our Dagger on Fly.io setup with Dagger on Namespace.so

Before this change, throughout August 2024:
- deploys took `4m 36s` 95% of the time
- `1` in `14` deploys failed

Let's see how the above changes after a few weeks of running.

This is the first part of the required change. The second part will
follow after this gets merged. We need to:
- Cleanup secrets & variables from GitHub Actions
- De-provision the Dagger Engine on Fly.io
- Remove the deploy config from fly.io dir
- Update the INFRASTRUCTURE diagram

We still keep K8s to see if we can improve on Namespace.so, the new
Speed King 🏎️

Signed-off-by: Gerhard Lazu <[email protected]>
@gerhard gerhard force-pushed the 2x-faster-time-to-deploy branch from a90b8bf to 21f1ed1 Compare September 1, 2024 19:15
@gerhard gerhard merged commit c9252c6 into thechangelog:master Sep 1, 2024
4 checks passed
@gerhard gerhard deleted the 2x-faster-time-to-deploy branch September 1, 2024 19:30
@gerhard gerhard mentioned this pull request Sep 2, 2024
4 tasks
@gerhard
Copy link
Member Author

gerhard commented Sep 2, 2024

On the second run, I was expecting Build, test & publish... to use the cached Docker volume /cache/dagger-0-6-4: https://github.com/thechangelog/changelog.com/actions/runs/10664539072/job/29556003383

image

This is how the same run behaved on K8s, where the cache was definitely used: https://github.com/thechangelog/changelog.com/actions/runs/10664539072/job/29556002949
image

I was expecting that particular step to take around 40s on Namespace, but it took 2m 56s. I have confirmed that none of the operations were cached.

Why did the cached volume not work as expected on Namespace.so? This message might explain it, even if it's somewhat hidden:

image

If cache misses are indeed "very rare", I would expect the next run to have a cache hit. Let's see...

@gerhard
Copy link
Member Author

gerhard commented Sep 2, 2024

That was it!

Second run worked much better: 20s for the Build, test & publish... step: https://github.com/thechangelog/changelog.com/actions/runs/10664539072/job/29556746670
image

This now looks right too:
image

It will be worth keeping an eye on these cache misses over the next few weeks (notice the time difference between the first 2 uncached runs vs the third cached run):
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant