-
Notifications
You must be signed in to change notification settings - Fork 361
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New script needed to pull down tarballs. #33
Comments
If any @fink/fink-developers have the bandwidth to take this up please do not hesitate to assign this to yourself. ☺ |
Is there a real reason to delete tarballs apart from disk storage requirements? I'd advocate not deleting them on such a short timeframe because of the following scenario:
If the discovery that foo-version2 is broken occurs sufficiently late, then foo-version1 can't be built any more. |
We could just as easily specify that it keep them for 3 months or so; the reason indeed is to keep disk space requirements down. If we can make a system that can delete tarballs after a specified time period at all, we can make that period whatever we think is best and can even make it configurable to fit the needs of different mirror hosts. |
Just updated the script to the one we are using (the one from my experimental tree) |
hope it doesn't break snitch, since I see you changed the default checkout to something that will need a password. and snitch has been using the one from git and selfupdates it. |
As for the delete, it should just check all dists and just keep what is in the current dists by default and delete the rest on an interval, that would be the safest. Just adding that note here for when I start the rewrite of this script. |
Okay my initial rewrite based on @gecko2 work is in ea7d557 it's a WIP and does not restrict any dists yet and has not been fully tested, I've just get it to start a run and start downloading things. It does not to the dists pull, you would write a script that would update the fink and dists dir before running it. That way both fink and dists could be cvs, svn, rsync, git it won't matter. It just requires that the base of the dists dir (which doesn't need to be called dists) contain the 10.x dirs. |
Also as for deleting, based on how the info files get loaded, it's hard to know what is currently in use and what isn't, I think we might need a different script for cleanup, it would have to load all info files and gather all files that should exist, OR you could run finkdist to a new dir, then move it into place remove the old which would clear any old ones, this would assume no retention of old tar balls though and likely isn't the best idea, just spit balling here ;) |
Already cleaned the download dir earlier by collecting all currently used checksums and removing every not referenced files. That got me around the half of the archive we have the first time i've done that. |
An outline of how a cleanup script might work: It should take a time to live as an option because different mirror may want to keep things for different amounts of time.
|
A new script to replace distfiles/generate-distfiles-and-finkinfo-mirror.pl is needed.
It should meet the following requirements; given a collection of info files on disk as fetched by one of the selfupdate-server scripts, it should:
It explicitly should not care about how the info files end up on disk though it should use the timestamp files as described to rate limit itself; it should also set those self same timestamps to signal to other mirrors about general heath.
This is being worked on in distfiles/finkdist.
The text was updated successfully, but these errors were encountered: