feat: improve db update check #2020
Conversation
Signed-off-by: Keith Zantow <[email protected]>
kzantow
force-pushed
the
feat/if-modified-since-header
branch
from
24439aa to
0942add
Compare
| @@ -148,13 +152,14 @@ func (c *Curator) Update() (bool, error) { | |||
| defer downloadProgress.SetCompleted() | |||
| defer importProgress.SetCompleted() | |||
|
|
|||
| updateAvailable, metadata, updateEntry, err := c.IsUpdateAvailable() | |||
| metadata := c.GetMetadata() | |||
There was a problem hiding this comment.
Should we always download the listing on grype db check and grype db update?
There was a problem hiding this comment.
I don't think so, if we reliably know they are already up-to-date, why waste CPU cycles/network?
There was a problem hiding this comment.
I am pessimistically assuming that something will go wrong, and having a command that means, "no really, I mean it, download that database whether you need to or not" might be helpful.
The goal of this change is to alleviate traffic. Making rare commands less useful isn't going to alleviate traffic.
But I do see your point.
There was a problem hiding this comment.
I'm not sure how this is making commands less useful, they all continue to function exactly as they do: if we downloaded a listing file and got the latest database, we have the latest database, so downloading a listing again and downloading the database again is rather pointless. I think I understand your point, though. I'll see about making this particular function isolated as much as possible.
There was a problem hiding this comment.
I think as a user, I would prefer grype db update to be efficient and use this function to eliminate unnecessary downloads if it's already up-to-date. Maybe we add a --force argument or similar to make it download everything anew?
There have been multiple reports about issues while attempting to check for database updates. We think it will be good to address these on multiple fronts. One of the things we can do instead of downloading a listing file of ~150Kb every time Grype is run is to use the standard
If-Modified-Sincebehavior and track a useable time we obtained the database to include in this header.This PR adds an
Updatedfield to the metadata, which is set at db download time and used when further requests are made for the database to prevent downloading the file when it won't be used anyway.