I defined the following grammar:
from lark import Lark grammar = r''' start: ( bold_text | text) bold_text.2: "**" text "**" text.1: /.+/ ''' parser = Lark(grammar, start = "start", debug=True, ambiguity="explicit") text = r"""**test**""" parsed = parser.parse(text) print(parsed.pretty()) For "**test**", this parses "text" and not "bold_text" as expected. Lark does not even show an ambiguity. What am I doing wrong? Any help would be greatly appreciated. Thank you.
Lark is using greedy regular expressions as the infrastructure of its lexer, as is common for parsers.
The odd behavior you're seeing happens because when bold_text matches the text rule, its regexp also catches the two stars at the end, because they are matched by .
A way to fix your grammar would be to do this instead:
start: bold_text | text bold_text.2: "**" text "**" text.1: /[^*]+/ That does add some difficulties, since there are situations when you might want to allow * as part of the text. But there are tricks to work around it.
You can even use this solution with LALR, which will run much faster.
An "easier" solution would be to enable the complete-Earley feature of Lark, which isn't affected by the greediness of regexps (but it also runs much slower):
grammar = r''' start: ( bold_text | text) bold_text.2: "**" text "**" text.1: /.+/ ''' parser = Lark(grammar, start = "start", debug=True, ambiguity="explicit", lexer="dynamic_complete") And the result is -
_ambig start bold_text text test start text **test** But using complete-Earley for very large texts will prove to performance-intensive, especially if you use catch-all regexps like .+.
Having said all that, Lark is better suited for parsing structured languages like Python than for free-form text like markup. You might want to consider writing a specialized parser (aka "hand-written parser") for this task.
from lark import Lark, Transformer, v_args grammar = r""" start: (bold_text | text_ ) text_: text bold_text: "**" text "**" text.1: (/./)+ """ parser = Lark(grammar, start="start", debug=True) text = r"""**test**""" @v_args(inline=True) class important(Transformer): def text(self,*chars): return "".join(chars) parsed = important().transform(parser.parse(text)) print(parsed.pretty()) Returns:
start bold_text test Maybe this is not the best, but it should work.
