Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add hashed labels to expression rule in JavaParser #4310

Merged
merged 4 commits into from
Nov 17, 2024

Conversation

samatanna
Copy link
Contributor

No description provided.

java/java/JavaParser.g4 Outdated Show resolved Hide resolved
Copy link
Contributor

@kaby76 kaby76 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems fine. The code builds and runs fine.

11/07-09:35:54 ~/issues/g4-4310/java/java
$ bash ../../_scripts/test.sh
Adding java/java
Number of grammars before sorting and making unique: 1
Number of grammars after sorting and making unique: 1
Grammars to do are: java/java
grammars = java/java
targets = Antlr4ng CSharp Cpp Dart Go Java JavaScript Python3 TypeScript
order = grammars
filter = all
Tests now java/java,Antlr4ng java/java,CSharp java/java,Cpp java/java,Dart java/java,Go java/java,Java java/java,JavaScript java/java,Python3 java/java,TypeScript

java/java,Antlr4ng:
 Build 00:00:12
 Test 00:00:12
 Succeeded.

java/java,CSharp:
 Build 00:00:09
 Test 00:00:10
 Succeeded.

java/java,Cpp:
 Build 00:00:22
 Test 00:00:09
 Succeeded.

java/java,Dart:
 Build 00:00:10
 Test 00:00:09
 Succeeded.

java/java,Go:
 Build 00:00:06
 Test 00:00:10
 Succeeded.

java/java,Java:
 Build 00:00:06
 Test 00:00:10
 Succeeded.

java/java,JavaScript:
 Build 00:00:04
 Test 00:00:10
 Succeeded.

java/java,Python3:
 Build 00:00:04
 Test 00:00:16
 Succeeded.

java/java,TypeScript:
 Build 00:00:11
 Test 00:00:12
 Succeeded.
11/07-09:39:24 ~/issues/g4-4310/java/java

Note, the grammar still has a lot of ambiguity, which existed before this PR. As we address that and SLL fallback, we'll try to preserve the labeled alt names as best as possible.

11/07-09:39:24 ~/issues/g4-4310/java/java
$ cd Generated-CSharp/
11/07-09:42:06 ~/issues/g4-4310/java/java/Generated-CSharp
$ ((trperf ../examples/AllInOne11.java -c ardFiTkmfet -h 2>/dev/null | head -1) ; (trperf ../examples/*.java -c ardFiTkmfet 2>/dev/null | grep -v '^0' | sort -k1 -n -r )) | column -t
Ambiguities  Rule                            Decision  File                                       Invocations  Time      Total-k  Max-k  Fallback  Errors  Transitions
100          annotation                      88        ../examples/AllInOne8.java                 102          0.106597  582      4      100       0       205
41           expression                      171       ../examples/AllInOne7.java                 699          0.56063   1048     9      233       0       356
26           formalParameters                75        ../examples/AllInOne17.java                47           0.026107  133      4      26        0       28
25           formalParameters                75        ../examples/AllInOne7.java                 47           0.030179  178      16     25        0       36
24           formalParameters                75        ../examples/AllInOne11.java                29           0.027947  68       4      24        0       32
19           switchRuleOutcome               190       ../examples/AllInOne17.java                65           0.361916  448      36     19        0       173
16           expression                      171       ../examples/AllInOne17.java                385          0.205093  593      15     81        0       177
16           annotation                      88        ../examples/AllInOne17.java                17           0.036413  98       4      16        0       36
15           formalParameters                75        ../examples/AllInOne8.java                 32           0.084016  293      25     15        0       93
13           annotation                      88        ../examples/AllInOne11.java                13           0.109442  71       4      13        0       31
9            expression                      171       ../examples/AllInOne8.java                 187          0.082198  299      10     74        0       111
8            pattern                         172       ../examples/AllInOne17.java                34           0.024551  173      9      8         0       51
7            expression                      171       ../examples/ExpressionOrder.java           23           0.059109  92       25     8         0       63
6            typeParameter                   20        ../examples/AllInOne8.java                 12           0.018952  60       8      6         0       31
6            statement                       138       ../examples/AllInOne17.java                138          0.514529  862      78     6         0       191
5            guardedPattern                  186       ../examples/AllInOne17.java                21           0.213308  145      25     5         0       74
5            expression                      162       ../examples/AllInOne8.java                 11           0.032845  56       8      5         0       32
5            blockStatement                  120       ../examples/AllInOne8.java                 38           0.227595  460      39     5         0       150
4            annotation                      88        ../examples/AllInOne7.java                 10           0.005303  31       6      4         0       11
3            lastFormalParameter             81        ../examples/AllInOne8.java                 7            0.012701  44       9      3         0       22
3            interfaceBodyDeclaration        47        ../examples/AllInOne8.java                 13           0.036205  36       6      3         0       18
3            formalParameter                 80        ../examples/AllInOne8.java                 25           0.008459  76       6      3         0       16
3            formalParameter                 80        ../examples/AllInOne7.java                 30           0.00841   109      11     3         0       31
2            switchLabeledRule               182       ../examples/AllInOne17.java                45           0.022918  125      9      2         0       27
2            memberDeclaration               40        ../examples/AllInOne17.java                95           0.040962  307      23     2         0       54
2            lambdaParameters                177       ../examples/AllInOne8.java                 9            0.003063  20       4      2         0       7
2            interfaceCommonBodyDeclaration  55        ../examples/AllInOne8.java                 12           0.010351  39       9      2         0       18
2            interfaceBodyDeclaration        47        ../examples/AllInOne11.java                16           0.009367  26       3      2         0       11
2            expression                      171       ../examples/AllInOne11.java                113          0.095831  149      4      31        0       47
2            expression                      164       ../examples/ExpressionOrder.java           15           0.023408  74       30     2         0       15
2            expression                      164       ../examples/AllInOne8.java                 113          0.827767  1099     359    4         0       266
2            expression                      164       ../examples/AllInOne17.java                304          0.330951  688      15     10        0       109
2            annotation                      88        ../examples/ManyStringsConcat.java         2            0.006191  11       4      2         0       5
1            switchRuleOutcome               190       ../examples/SwitchExpression.java          2            0.007765  39       19     1         0       11
1            statement                       138       ../examples/AllInOne7.java                 221          0.081063  489      61     1         0       64
1            statement                       138       ../examples/AllInOne11.java                32           0.085333  84       19     1         0       37
1            memberDeclaration               40        ../examples/RecordsTesting.java            5            0.004541  58       16     1         0       12
1            interfaceMemberDeclaration      49        ../examples/RecordsTesting.java            1            0.006022  42       21     1         0       21
1            interfaceMemberDeclaration      49        ../examples/AllInOne17.java                3            0.014994  25       9      1         0       14
1            formalParameters                75        ../examples/Escapes.java                   1            0.001547  2        1      1         0       1
1            expression                      164       ../examples/AllInOne11.java                82           0.376549  643      469    1         0       218
1            blockStatement                  120       ../examples/LocalVariableDeclaration.java  3            0.016292  43       18     1         0       16
1            blockStatement                  120       ../examples/GenericConstructor.java        1            0.015253  34       17     1         0       13
1            blockStatement                  120       ../examples/AllInOne17.java                159          0.057488  425      18     1         0       74
1            blockStatement                  120       ../examples/AllInOne11.java                60           0.084872  205      17     1         0       65
1            arrayCreatorRest                202       ../examples/AllInOne7.java                 6            0.060499  13       4      1         0       9
1            annotation                      89        ../examples/ManyStringsConcat.java         1            1.512423  7206     3603   1         0       1805
1            annotation                      89        ../examples/AllInOne11.java                7            0.026148  21       6      1         0       11
11/07-09:42:15 ~/issues/g4-4310/java/java/Generated-CSharp

@@ -39,7 +39,7 @@ options {
}

compilationUnit
: packageDeclaration? (importDeclaration | ';')* (typeDeclaration | ';')*
: packageDeclaration? (importDeclaration | ';')* (typeDeclaration | ';')* EOF
Copy link
Contributor

@kaby76 kaby76 Nov 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Malformed start rules, where EOF is not on all alts, and more generally, an ATN that does not exit with EOF, will need to be detected. kaby76/g4-scripts#5

@kaby76
Copy link
Contributor

kaby76 commented Nov 10, 2024

Note: The parse trees, which are contained in the examples/*.tree files, just needs to be remastered. This is because <EOF> now shows--correctly--in the tree/

@codeFather2
Copy link
Contributor

@teverett, can we merge this PR into master please?

@teverett
Copy link
Member

@samatanna thanks!

@teverett teverett merged commit 97bd288 into antlr:master Nov 17, 2024
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants