Low disk space on download server (again)

[petefoth]

While investigating problems with zip files not being rynced to the download server (see #503 ), a manual rsync failed with
rsync: [receiver] write failed on "/mnt/archive/zips/berlna/lineage-20.0-20231102-microG-berlna.zip": No space left on device (28)

Checking free space on the server showed we have nearly no space left

df -h
    Filesystem      Size  Used Avail Use% Mounted on
    <snip>
    /dev/vdb1       590G  559G  920M 100% /mnt/archive

The lack of space is party due to the fact we now publish loads of .img files in addition to the main ROM zip files and recovery images - see #405 & #467

Zipping up these images (see #483) caused the 'Update recovery` function of the Updater to fail #487, so was reverted - See PR #490

Unless we can find a way to zip up the shipped image files without breaking anything else we will probably have to keep only two recent builds instead of three

We may be able to free up enough space to avoid this by trimming some of the 17.1 builds that are still hanging around for the following devices:
kiwi, harpia, I001D, crackling, a7xelte, mdarcy_tab, merlin, s2, surnia, m8, I01WD, j7elte, m8d, osprey, RMX1801, mdarcy, s5neolte, quill_tab, Amber, quill, porg_tab, a3xelte, a5xelte, lux,wt88047,

[st3rox]

590GB total disk space? Is it not possible/practical to add more storage? I'm sure we could community-fund a new hard drive

On that note, the "Donate" button on the website goes straight to the microg project. I would like to donate directly to this project if possible

[petefoth]

590GB total disk space? Is it not possible/practical to add more storage? I'm sure we could community-fund a new hard drive

That's a possibility, though it poses some logistical challenges :) And it may not be necessary: once the current build run is complete, there are a number of 17.1 builds that can be removed (even leaving the most recent 17.1 build for each device), I'll see how much space that frees up.

On that note, the "Donate" button on the website goes straight to the microg project. I would like to donate directly to this project if possible

Thanks. I will consult with the other maintainers and, if we decide we do need more disk space, we'll put in place a way of funding it. Apart from that, the project has very few expenses, and none of the project maintainers are doing it for the money ;)

[st3rox]

Thanks. I will consult with the other maintainers and, if we decide we do need more disk space, we'll put in place a way of funding it. Apart from that, the project has very few expenses, and none of the project maintainers are doing it for the money ;)

Idk what the physical system specs are like but donations could also go to upgrades, maybe even getting back up to 2 builds per month ;)

I personally have 4 devices that rely on this project and will gladly donate to keep it going smoothly

[petefoth]

Idk what the physical system specs are like but donations could also go to upgrades, maybe even getting back up to 2 builds per month ;)
I personally have 4 devices that rely on this project and will gladly donate to keep it going smoothly

Thanks for the offer. I will discuss the disk space issue with the other maintainers, once I have tidied up and know how much spare we have. We will explore the possibility of increasing capacity, although upgrading our servers is not entirely straightforward.

PS Maybe not wise to rely too much on a project with very few resources and which offers no guarantees :)

[st3rox]

PS Maybe not wise to rely too much on a project with very few resources and which offers no guarantees :)

Hahaha yeah i know

My PC can build if I should ever need to, I have run it a few times to verify it works

I also could switch to vanilla lineageOS or something else out there, but I prefer microg to work out of the box , and I've been around since Cyanogenmod 7 so I'd like to stick with LineageOS

[petefoth]

After the December build run completed the disk is less than 60% full.
But, some devices now only have a single build stored instead of two. That should be rectified in the January build run, so the used space will increase. I will wait and see how much free space we have then before deciding whether or not we need to do anthong more about this issue.

Filesystem      Size  Used Avail Use% Mounted on
....
/dev/vdb1       590G  329G  231G  59% /mnt/archive

[petefoth]

After 30 hours and 12 builds in the January build run, we have used 4GB more disk space i.e. roughly 3 builds per GB. If that rate is typical, we should end up using just over 400GB out of 590GB :)

[petefoth]

January build run complete

df -h
Filesystem      Size  Used Avail Use% Mounted on
...
/dev/vdb1       590G  331G  229G  60% /mnt/archive

But for many devices we have only a single 200.build. Check again after February build run, when we should have two.

[petefoth]

Check again after February build run, when we should have two.

The free disk space numbers after the February build run, with two builds stored per device

Filesystem      Size  Used Avail Use% Mounted on
/dev/vdb1       590G  491G   70G  88% /mnt/archive

The figures include the new archive folder containing the final builds for devices which are no longer supported by LOS (see Build retention strategy #573 ), which takes up a total of 24G. Even if we removed the archive, we would still not have space to keep three builds per device rather than two. This is mainly due to the fact that we now keep several img files for each device build

[petefoth]

At the end of the April build run, we are using 85% (473G) of the available disk space (590G). This consists of

• 392 builds for 201 current devices: max 2 builds per device, 70 x 21.0 and 322 x 20.0 builds
• 87 builds in archive for discontinued devices: 1 build per device, 84 x 18.1 and 3 x 20.0 builds

[petefoth]

As of today (Mon 29 Jul 24) we are down to only 11GB free space: /dev/vdb1 590G 550G 11G 99%

We may have to start throwing stuff away