Skip to main content
Software Engineering

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

2
  • excellent answer Jonathan. Thank you very much. I do have a query about you answer though; You said that regex is directly related to using a FSM. If that is so, wouldn't regex be considered a higher level FSM, and thus be more efficient? Or would just directly using a FSM be a better choice? Commented Sep 2, 2016 at 17:23
  • Regexs are higher level than pure FSM, to accomodate those extra features (look-ahead, look-behind, ...) that make specifying tokens easier / richer. You almost surely don't want to use an FSM directly (too low level and nitsy). For a toy language, I wouldn't sweat the lexer choice too much. Sketch out what you want the language to look like (e.g. with examples of the langauge being used), choose a convenient parser engine, then use whatever lexer works easiest with your chosen parser engine. Commented Sep 2, 2016 at 17:32