Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a way to fail with "lexer error" #23

Open
osa1 opened this issue Oct 21, 2021 · 0 comments
Open

Provide a way to fail with "lexer error" #23

osa1 opened this issue Oct 21, 2021 · 0 comments
Labels
design question Further information is requested

Comments

@osa1
Copy link
Owner

osa1 commented Oct 21, 2021

Currently there's no way to fail with lexgen's "lexer error" when an error is detected. We need to specify a user error type, and fail using a value of that type.

It could be useful to provide a fail method in the lexers to fail with the standard "lexer error" error, without having to define an error type.

Use case: Rust string literals do not allow standalone carriage return, i.e. \r needs to be followed by \n.

One way to implement it is this:

lexer! {
    rule Init {
        ...

        '"' => |lexer| lexer.swtich(LexerRule::String),
    }

    rule String {
        ...

        '\r' => |lexer| lexer.switch(LexerRule::StringCR),

        _,
    }

    rule StringCR {
        '\n' => |lexer| lexer.switch(LexerRule::String),
    }
}

If we had a fail method on lexer structs it could be slightly more concise:

lexer! {
    rule Init {
        ...

        '"' => |lexer| lexer.swtich(LexerRule::String),
    }

    rule String {
        ...

        "\r\n",

        '\r' => |lexer| lexer.fail(),

        _,
    }
}

We don't need another rule to make sure \r is always followed by \n. We can already do this today, but we have to define an error type and return an error: (not tested)

lexer! {
    type Error = UserError;

    rule Init {
        ...

        '"' => |lexer| lexer.swtich(LexerRule::String),
    }

    rule String {
        ...

        "\r\n",

        '\r' =? |lexer| lexer.return_(Err(UserError { ... })),

        _,
    }
}
@osa1 osa1 added question Further information is requested design labels Oct 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
design question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant