For REMIND AMTs: execute make test-full-slurm
#92
Conversation
…about its results to mattermost bot message
| system(paste0(gitPath, " reset --hard origin/develop && ", gitPath, " pull")) | ||
| }) | ||
| # 2. rename old test-full.log | ||
| system("mv test-full.log test-full-previous.log") |
There was a problem hiding this comment.
You don't want to keep the logs long-term? I mean, you could also adjust make test-full-slurm to something like
sbatch --qos=priority --wrap="make test-full" --job-name=test-full --mail-type=END --output=test-full-2024-03-01.log --comment="test-full-2024-03-01.log"
in REMIND's makefile (not exactly sure how this would be implemented that the date is always adjusted). Or, at the end of the script, when everything is done, you copy the test-full.log to test-full-2024-03-01.txt and avoid like that that it is overridden.
There was a problem hiding this comment.
I wanted to avoid another file piling up in some folder that nobody needs. But if you really think we need the output of tests older than two weeks, I can change that.
There was a problem hiding this comment.
Can you just move it into tests subdirectory where it bothers no-one but is available if needed?
| # start make test-full | ||
| # 1. pull changes to magpie develop | ||
| withr::with_dir("magpie", { | ||
| system(paste0(gitPath, " reset --hard origin/develop && ", gitPath, " pull")) |
There was a problem hiding this comment.
| system(paste0(gitPath, " reset --hard origin/develop && ", gitPath, " pull")) | |
| system(paste0(gitPath, " reset --hard origin/master && ", gitPath, " pull")) |
Do we really want to test against MAgPIE develop? In case there is some temporaty error in MAgPIE, why should we have REMIND fails? Not sure about that.
There was a problem hiding this comment.
I agree, it is a bit of a risk. However, the advantage is that it allows us to detect changes in the coupling interface immediately (= when the name of variables change that are required for the coupling) and not after it has already been established.
There was a problem hiding this comment.
I agree, using develop is actually the better choice.
| } else if (any(grep("FAIL 1", logStatus))) { | ||
| testthatResult <- "Some tests FAIL in `make test-full`. Check test-full.log" | ||
| } else { | ||
| # "make test-full reports all tests PASS" |
There was a problem hiding this comment.
I would prefer something like tail(grep("[ FAIL", logStatus, value = TRUE, fixed = TRUE)) to be appended to the log status, so you see directly how many fails exists etc.
And: Why grep("Fail 1", logStatus)? Does that work if you have more than one fail?
There was a problem hiding this comment.
You are right. I tested this with the output of make test-coupled-slurm (test-coupled.log), but its output seems to be different in terms of structure from the output of make test-full-slurm. I`ll adapt it.
Together with the REMIND AMTs execute
make test-full-slurmand evaluate the result and append it to the mattermost bot message.