Why this rule cant match start symbols?

[vega0]

https://www.youtube.com/watch?v=Lq-gkbLfOyo

Why lexer can't match start two symbol which one is optional (optional symbol) and it's last.

[vega0]

how to split lexer rules?

[jimidle]

Once again, I implore you to do some training/reading before attempting this.

[kaby76]

Why lexer can't match start two symbol which one is optional (optional symbol) and it's last.

Input string "@/hello/world" is tokenized as:

[@0,0:0='@',<5>,1:0]
[@1,1:1='/',<6>,1:1]
[@2,2:6='hello',<31>,1:2]
[@3,7:7='/',<6>,1:7]
[@4,8:12='world',<31>,1:8]
[@5,13:12='<EOF>',<-1>,1:13]

Input string "@@hello/world" is tokenized as:

[@0,0:1='@@',<31>,1:0]
[@1,2:2='/',<6>,1:2]
[@2,3:7='hello',<31>,1:3]
[@3,8:8='/',<6>,1:8]
[@4,9:13='world',<31>,1:9]
[@5,14:13='<EOF>',<-1>,1:14]

FRAGMENTPUTI is defined as FRAGMENTPUTI: ('\\ ' | ~(' ' | '/' | '.'))+;, line 99 of YapComplete.g4). String literal '@' is defined implicitly by Antlr as T__4: '@';. String literal lexer rules are defined before all lexer rules you wrote in YapComplete.g4. You now have a .g4 file implicitly defined as:

T__4: '@';
...
FRAGMENTPUTI: ('\\ ' | ~(' ' | '/' | '.'))+;

So, why is "@@" tokenized as token FRAGMENTPUTI token type 31 instead of "@" as string literal '@', token type 5?

It is because the lexer matches the LONGEST string first regardless of the order of the lexer rules.

"@@" is two characters in length. It matches FRAGMENTPUTI in the input. The input "@" also matches T__4. But, length("@@") = 2 > 1 = length("@"). The lexer always returns the token type that matches the longest string.

[vega0]

WHY ITS JUST DONT WORK AS RULE THAT USED IN PARSER WORD?

[vega0]

FOR EXAMPLE I USE RULE (' ' | '\t')* AS PREDICTION FOR SPACES IN virajeniya

https://i.imgur.com/8iTYQiE.png But its expecting some line 1:14 extraneous input ' ' expecting {'+', '-', '(', A_ДЕСЯТЕРИЧНЫЙ_НОМЕР, A_ДЕСЯТЕРИЧНОЕ_ЧИСЛО, W_БИНАРНОЕ_ЧИСЛО, W_ТРИТИЧНОЕ_ЧИСЛО, W_ЧЕТВЕРИЧНОЕ_ЧИСЛО, W_ПЯТЕРИЧНОЕ_ЧИСЛО, W_ШЕСТЕРИЧНОЕ_ЧИСЛО, W_СЕМИРИЧНОЕ_ЧИСЛО, W_ВОСМЕРИЧНОЕ_ЧИСЛО, W_ДЕВЯТИРИЧНОЕ_ЧИСЛО, W_ШЕСТНАДЦАТИРИЧНОЕ_ЧИСЛО, '.', PROBELNIYODIN, UTF8CHARSET, KVANTIFIKATOR_IDENTIFIKATORA_POLN}

another rules that not linked with parser schema.

[vega0]

EXPECTING RULES THAT NOT IN SCHEMA INVOLVED.

[kaby76]

The tokens for the input "5+345-sdffs.g+ 123" are:

[@0,0:0='5',<10>,1:0]
[@1,1:1='+',<4>,1:1]
[@2,2:4='345',<11>,1:2]
[@3,5:5='-',<5>,1:5]
[@4,6:10='sdffs',<32>,1:6]
[@5,11:11='.',<23>,1:11]
[@6,12:12='g',<32>,1:12]
[@7,13:13='+',<4>,1:13]
[@8,14:14=' ',<22>,1:14]
[@9,15:17='123',<11>,1:15]
[@10,18:17='<EOF>',<-1>,1:18]

(Note, from the generated .tokens file, token type 22 is PROBEL. Token type 27 is PROBELNIYODIN.)

In imported file Oglavleniye.g4 (Virajeniya.g4 imports Peremenniye.g4 imports Stroki imports Oglavleniye.g4, among several other import paths--which I said might cause problems):

grammar Oglavleniye;

probelniy: PROBEL | TABYLACIYA | PROBELNIY;
utf8charset: UTF8CHARSET;

PROBEL: ' ';
TOCHKA: '.';
COMMA: ',';
NL: '\n';
TABYLACIYA: '\t';
PROBELNIYODIN: PROBEL | TABYLACIYA;

Here you have two rules, PROBEL and PROBELNIYODIN. PROBELNIYODIN uses PROBEL, but PROBEL occurs in the grammar first. So, when the lexer scans a space " ", the lexer finds that PROBEL and PROBELNIYODIN both match with string length of 1 character. But, Antlr always returns the token for the first rule that matches.

In your now modified grammar. uncommited into git, Virajeniya.g4 (with empty lines removed):

grammar Virajeniya;
import Chisla2, Peremenniye;
virajeniye: virajeniye PROBELNIYODIN* operaciya=('*' | '/' | '^') PROBELNIYODIN* virajeniye // умножение / деление
		  | virajeniye PROBELNIYODIN* operaciya=('+' | '-') 	   PROBELNIYODIN* virajeniye // сложение / вычитание
		  | '(' PROBELNIYODIN* virajeniye PROBELNIYODIN* ')'
		  | ssilkaNaObiekt
		  | aПолныйВидЧисла
		 ;

Here, you use PROBELNIYODIN. But, that can never match because the parser does not influence the lexer. That rule cannot "tell" the lexer how to do its job. It cannot "tell" the lexer to prefer PROBELNIYODIN over PROBEL. The tokenization has nothing to do with the parser. This is how traditional lexers and parsers work.

If you really do not want PROBEL to be a token that is returned by the lexer, make it "fragment PROBEL".