Update file dependencies for up-to-date tasks. #431
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR updates file dependencies in the doit database even if the task is already up to date. The change improves performance for large files under certain circumstances.
Consider the following task which simply copies
large_file.txt
tooutput.txt
.The first time doit runs, it saves the timestamp, size, and md5 hash. On the second run, doit smartly skips calculating the md5 hash of
large_file.txt
because the timestamps match. So far so good.Now suppose the timestamp changes but the content does not. This might happen if we delete an intermediate file which is then regenerated. On the second run, doit will evaluate the md5 on
large_file.txt
and skip the task because it's up to date--as expected. But it won't update the timestamp in the database. So every time we run doit, it'll evaluate the md5 hash oflarge_file.txt
.This PR ensures the file dependencies are updated in the database even if the task is already up to date. Here's a concrete example using
touch
to update the timestamp. I've modified thecheck_modified
function to report some debugging information (see end of description for details).Updated check_modified to report debug information.