Skip to content

Commit

Permalink
escape troublesome characters in regexp expressions
Browse files Browse the repository at this point in the history
Found this issue while parsing the ABNF JSON string representation.
These two rules caused parsing issues because of the characters
generated within the regular expressions that they result in:

escape    = %x5C ; \
unescaped = %x20-21 / %x23-5B / %x5D-10FFFF ; [ -!] / [#-[] / []-\u{10FFF}]
  • Loading branch information
AaronLasseigne authored and gkellogg committed Feb 23, 2022
1 parent b37fc88 commit a67642f
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 4 deletions.
12 changes: 9 additions & 3 deletions lib/ebnf/rule.rb
Original file line number Diff line number Diff line change
Expand Up @@ -367,11 +367,11 @@ def to_peg
def to_regexp
case expr.first
when :hex
Regexp.new(translate_codepoints(expr[1]))
Regexp.new(Regexp.escape(translate_codepoints(expr[1])))
when :istr
/#{expr.last}/ui
when :range
Regexp.new("[#{translate_codepoints(expr[1])}]")
Regexp.new("[#{escape_regexp_character_range(translate_codepoints(expr[1]))}]")
else
raise "Can't turn #{expr.inspect} into a regexp"
end
Expand Down Expand Up @@ -770,5 +770,11 @@ def make_sym_id(variation = nil)
@id_seq += 1
["_#{@sym}_#{@id_seq}#{variation}".to_sym, ("#{@id}.#{@id_seq}#{variation}" if @id)]
end

# Escape "[", "]", and "\" in ranges so they don't result in a warning or error
# about empty character classes.
def escape_regexp_character_range(character_range)
character_range.gsub(/([\[\]\\])/) {|char| "\\#{char}"}
end
end
end
end
6 changes: 5 additions & 1 deletion spec/rule_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -481,9 +481,13 @@
describe "#to_regexp" do
{
hex: [:hex, "#x20", / /],
hex: [:hex, "#x5c", /\\/],
range: [:range, "a-b", /[a-b]/],
range2: [:range, "a-zA-Z", /[a-zA-Z]/],
range3: [:range, "abc-", /[abc-]/],
range4: [:range, "#x23-#x5b", /[#-\[]/],
range5: [:range, "#x5d-#x5e", /[\]-^]/],
range6: [:range, "#x5c-#x5e", /[\\-^]/],
}.each do |title, (op, exp, regexp)|
it title do
expect(EBNF::Rule.new(title, nil, [op, exp]).to_regexp).to eql regexp
Expand Down Expand Up @@ -1055,4 +1059,4 @@
end
end
end
end
end

0 comments on commit a67642f

Please sign in to comment.