-
Notifications
You must be signed in to change notification settings - Fork 306
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add command line options support for ant in the compile command #471
base: master
Are you sure you want to change the base?
Conversation
@rjust WDYT? |
Signed-off-by: André Silva <[email protected]>
d46d5f1
to
501637d
Compare
I am little hesitant to allow build system options to be set in the compile command. The compile command is supposed to abstract away the actual build system being used. While Defects4J is currently using Ant, it is possible that this will change to mvn, gradle, or plain javac invocations. What is the use case for changing the default behavior of defects4j compile? |
Hi @rjust, We used this to study fault localization tools under different Java versions. For this, we needed to enforce specific target versions, which can be done by passing that option to the compile command. See RQ1 https://arxiv.org/pdf/2111.12513 and https://github.com/ASSERT-KTH/flacoco_experiment/blob/9f5a0bac1fe564b2ff622423773dee3b79117e54/flacoco/scripts/runAllBugs.py#L114-L329 |
Unfortunately, forcing a specific version of Defects4J to use a different version of Java than it is designed for will lead to spurious failures. We have observed many such failures, so this is not a hypothetical concern. My recommendation is: We are currently preparing Defects4J version 4, which will support (only) Java 17. |
Yes, we observed the same actually. In case is useful to you, we found 191 bugs from D4J v2.0 to be compilable and correctly executable (i.e. same sets of executed and failing tests as the original version) under Java 7,8, 11, and 17. List is here: https://github.com/ASSERT-KTH/flacoco_experiment/blob/main/flacoco/results_faultlocalization/all_bugs.csv
That's cool, thanks :) |
Thanks for the list of bugs that work under all 4 Java versions without change. Have you tried Java 21, which is the current version of Java? Our approach has been to adjust tests in order to adapt to different execution environments. On the plus side, this means far more than 191 usable bugs. However, this means that a given test may be slightly different between versions of Defects4J. For example, handling of locales changed between Java 8 and Java 11, which forced us to revise tests that depended on locales. |
I have not. We did these experiments more than 2 years ago, before Java 21 was released. At the time those were the LTS versions available.
Are you also planning to collect new bugs that would natively be Java 17/21 compatible? |
Yes. That is an ongoing process. Help is always appreciated! |
Cool! On this topic, you might be interested in our recent tool GitBug-Actions (paper, repo). The core principle is to use the existing CI scripts to reproduce bugs in isolated environments inside docker containers, ensuring reproducibility. We have recently collected a Java benchmark using it, GitBug-Java (paper, repo). |
That's nice (and useful!) work. Thanks for pointing it out to me. I will check it out. |
This PR introduces a new option in the
compile
command of defects4j.This option enables manually setting additional ant options when compiling through defects4j.