Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conformance-test report not automatically regenerated after jspecify change #184

Open
cpovirk opened this issue Jul 9, 2024 · 0 comments
Assignees

Comments

@cpovirk
Copy link
Collaborator

cpovirk commented Jul 9, 2024

The action seems to have run for #508, but I don't see a resulting PR. And a clean run...

git clone [email protected]:jspecify/jspecify-reference-checker.git && git clone [email protected]:jspecify/jspecify.git && cd jspecify-reference-checker/ && JAVA_HOME=$HOME/jdk-17.0.2 ./gradlew build conformanceTests demoTest --include-build ../jspecify

...fails:

tests.ConformanceTest > conformanceTests FAILED
    diff (-expected +actual):
        @@ -1,4 +1,4 @@
        -# 90 pass; 24 fail; 114 total; 78.9% score
        +# 93 pass; 24 fail; 117 total; 79.5% score
         PASS: Basic.java:28:test:expression-type:Object?:nullable
         PASS: Basic.java:28:test:sink-type:Object!:return
         PASS: Basic.java:28:test:cannot-convert:Object? to Object!
        @@ -44,6 +44,7 @@
         PASS: irrelevantannotations/notnullmarked/Other.java:NonNull local variable array:test:irrelevant-annotation:NonNull
         PASS: irrelevantannotations/notnullmarked/Other.java:Nullable exception parameter:test:irrelevant-annotation:Nullable
         PASS: irrelevantannotations/notnullmarked/Other.java:NonNull exception parameter:test:irrelevant-annotation:NonNull
        +PASS: irrelevantannotations/notnullmarked/Other.java:Intrinsically NonNull exception parameter cannot be assigned null:test:cannot-convert:null? to RuntimeException!
         PASS: irrelevantannotations/notnullmarked/Other.java:Nullable try-with-resources:test:irrelevant-annotation:Nullable
         FAIL: irrelevantannotations/notnullmarked/Other.java:NonNull try-with-resources:test:irrelevant-annotation:NonNull
         FAIL: irrelevantannotations/notnullmarked/Other.java:Nullable exception type:test:irrelevant-annotation:Nullable
        @@ -75,6 +76,7 @@
         PASS: irrelevantannotations/nullmarked/Other.java:NonNull local variable array:test:irrelevant-annotation:NonNull
         PASS: irrelevantannotations/nullmarked/Other.java:Nullable exception parameter:test:irrelevant-annotation:Nullable
         PASS: irrelevantannotations/nullmarked/Other.java:NonNull exception parameter:test:irrelevant-annotation:NonNull
        +PASS: irrelevantannotations/nullmarked/Other.java:Intrinsically NonNull exception parameter cannot be assigned null:test:cannot-convert:null? to RuntimeException!
         PASS: irrelevantannotations/nullmarked/Other.java:Nullable try-with-resources:test:irrelevant-annotation:Nullable
         FAIL: irrelevantannotations/nullmarked/Other.java:NonNull try-with-resources:test:irrelevant-annotation:NonNull
         FAIL: irrelevantannotations/nullmarked/Other.java:Nullable exception type:test:irrelevant-annotation:Nullable
        @@ -107,6 +109,7 @@
         PASS: irrelevantannotations/nullunmarked/Other.java:NonNull local variable array:test:irrelevant-annotation:NonNull
         PASS: irrelevantannotations/nullunmarked/Other.java:Nullable exception parameter:test:irrelevant-annotation:Nullable
         PASS: irrelevantannotations/nullunmarked/Other.java:NonNull exception parameter:test:irrelevant-annotation:NonNull
        +PASS: irrelevantannotations/nullunmarked/Other.java:Intrinsically NonNull exception parameter cannot be assigned null:test:cannot-convert:null? to RuntimeException!
         PASS: irrelevantannotations/nullunmarked/Other.java:Nullable try-with-resources:test:irrelevant-annotation:Nullable
         FAIL: irrelevantannotations/nullunmarked/Other.java:NonNull try-with-resources:test:irrelevant-annotation:NonNull
         FAIL: irrelevantannotations/nullunmarked/Other.java:Nullable exception type:test:irrelevant-annotation:Nullable

I noticed this when I saw that set of failures for #183.

cpovirk added a commit that referenced this issue Jul 9, 2024
This hasn't happened automatically, as discussed in
#184.
cpovirk added a commit that referenced this issue Jul 9, 2024
This hasn't happened automatically, as discussed in
#184.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants