How to only match full token?

[donny-dont]

I've been working on a library that parses Web IDL and I've hit an issue I'm not sure how to resolve.

Here's a partial of the grammar.

DistinguishableType ::
    PrimitiveType Null
    StringType Null
    identifier Null
    sequence < TypeWithExtendedAttributes > Null
    object Null
    symbol Null
    BufferRelatedType Null
    FrozenArray < TypeWithExtendedAttributes > Null
    ObservableArray < TypeWithExtendedAttributes > Null
    RecordType Null

BufferRelatedType ::
    ArrayBuffer
    DataView
    Int8Array
    Int16Array
    Int32Array
    Uint8Array
    Uint16Array
    Uint32Array
    Uint8ClampedArray
    Float32Array
    Float64Array

I encountered the issue when I was parsing a type definition typedef (ArrayBufferView or ArrayBuffer) BufferSource; which fails to parse. I dug in a bit more and it looks like because of BufferRelatedType it matches the ArrayBuffer of ArrayBufferView and then fails the rest of the parse.

I based my grammar around the examples I saw so its using a token function like this.

  Parser bufferRelatedType() =>
      ref(token, 'ArrayBuffer') | ....

Is there any way to make it so the parser has to be surrounded by whitespace?

[renggli]

This is a common problem when parsing keywords. One way to solve it is to add a negative lookahead make sure the keyword is not followed by more letter/digits:

Parser bufferRelatedType() =>
      ref(token, 'ArrayBuffer').seq(word().not()).pick(0) | ...

Potentially you want to integrate that code into your token-builder, but often it is not needed for all tokens (or at all, depending on the language to parse).

Here is an example where this is done for the true token, and to avoid running into a problem when trying to parse a variable like trueBinding.