-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bump cache qcow size; skip compression on manifest.json #3942
Conversation
dustymabe
commented
Nov 13, 2024
- cmdlib: bump cache size to 30G
- cmd-build: skip-compression on manifest json
Not sure if there will be fallout from this or not. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did miss something in the commit message?
It only has With
Fun. Yes - there is a thought that is missing there. It should be:
If this looks good I'll push up that change and merge it. |
Ouch. This is getting kinda awkwardly large. Hmm, are we storing artifacts twice in the cache qcow2 due to the "musical chair" bit we have there? Feels like instead we should just change osbuild's Anyway, LGTM! |
Theoretically we aren't storing artifacts twice in the cache qcow2 because it reflinks things between the store and the outdir. We do just have a lot of pipelines/stages that copy a largeish image from one stage to another and then makes a small change to it.. when we make that small change I think we lose the reflink and it takes up more space now??
Yeah. Maybe we should just patch it? Though I think soon with idmapped virtioFS in kernel 6.12 we can solve that problem for ourselves. |
I noticed this when running some build tests: ``` + cosa compress Targeting build: 41.20241112.20.0 Compressing: builds/41.20241112.20.0/x86_64 INFO - Running command: ['xz', '-c9', '-T6', 'builds/41.20241112.20.0/x86_64/fedora-coreos-41.20241112.20.0-ostree.x86_64-manifest.json'] Compressed: fedora-coreos-41.20241112.20.0-ostree.x86_64-manifest.json.xz ``` Which I don't think we really want. Also let's make the spacing the same as the previous entry in the JSON.
With us now building more in OSBuild in a single run [1] we need a larger cache2.qcow2 to hold more pipeline outputs concurrently. [1] coreos/fedora-coreos-pipeline#1055
c5df67c
to
6f74c71
Compare