Transform a JSON grammar into a CodeMirror syntax-highlight parser

A simple and light-weight (~ 55kB minified, ~ 18kB zipped) CodeMirror add-on

to generate syntax-highlight parsers (codemirror modes) from a grammar specification in JSON format.

Note: The invariant codebase for all the *-grammar add-ons resides at editor-grammar repository (used as a git submodule)

• Live Playground Example
• Todo
• Features
• How To use
• API Reference
• Grammar Reference
• Other Examples

Code Indentation is Codemirror default, see Modularity and Future Directions

• handle arbitrary, user-defined, code (de-)indentation in the grammar specification (e.g via indent action tokens)
• handle arbitrary, user-defined, code matching (e.g brackets, tags, etc..) in the grammar specification (e.g via match action tokens) [DONE]

• handle arbitrary, user-defined, local/global/scoped relations in the grammar specification (e.g via scope action tokens) [DONE]
• and so on..
• enable grammar add-on to pre-compile a grammar specification directly into mode source code, so it can be used without the add-on as standalone mode [TODO, maybe]

• A Grammar can extend other Grammars (so arbitrary variations and dialects can be handled more easily)
• Grammar includes: Style Model , Lex Model and Syntax Model (optional), plus a couple of settings (see examples)
• Grammar specification can be minimal, defaults will be used (see example grammars)
• Grammar.Syntax Model can enable highlight in a more context-specific way, plus detect possible syntax errors and display appropriate error messages (see below)
• Grammar.Syntax Model can contain recursive references (see /test/grammar-js-recursion.html)
• Grammar.Syntax Model can be (fully) specificed using PEG-like notation or BNF-like notation (NEW feature)
• Grammar.Syntax Model implements positive / negative lookahead tokens (analogous to PEG and-/not- predicates) (NEW feature)
• Grammar.Syntax Model can include external (sub-)grammars so that new multiplexed / mixed grammars are created easily and intuitively (see test examples) (NEW feature)
• Grammar can define action tokens to perform complex context-specific parsing functionality, including associated tag matching and duplicate identifiers (see for example xml.grammar example) (NEW feature)
• Generated highlight modes can support toggle comments and keyword autocompletion functionality if defined in the grammar
• Context-sensitive autocompletion extracted directly from the grammar specification (NEW feature)
• Dynamic (Context-sensitive) autocompletion from typed user actions like code/token/symbols (NEW feature)
• Generated highlight modes can support lint-like syntax-annotation functionality generated from the grammar
• Generated highlight modes can support custom, user-defined, code folding functionality from the grammar fold model (NEW feature)
• Generated highlight modes can support custom, user-defined, code token matching functionality from the grammar match model (NEW feature)
• Generated parsers are optimized for speed and size
• Can generate a syntax-highlight parser from a grammar interactively and on-the-fly ( see example, http://foo123.github.io/examples/codemirror-grammar )
• see also Modularity and Future Directions

See working examples under /test folder.

An example for XML:

// 1. a partial xml grammar in simple JSON format var xml_grammar = { // prefix ID for regular expressions, represented as strings, used in the grammar "RegExpID" : "RE::", "Extra" : { "fold" : "xml" //"electricChars" : "<" }, // Style model "Style" : { "comment" : "comment" ,"declaration" : "tag" ,"doctype" : "meta" ,"meta" : "meta" ,"cdata" : "atom" ,"tag" : "tag" ,"attribute" : "attribute" ,"string" : "string" ,"atom" : "atom" ,"number" : "number" ,"error" : "error" }, // Lexical model "Lex" : { "comment:comment" : ["<!--", "-->"] ,"declaration:block" : ["<?xml", "?>"] ,"doctype:block" : ["RE::/<!doctype\\b/i", ">"] ,"meta:block" : ["RE::/<\\?[_a-zA-Z][\\w\\._\\-]*/", "?>"] ,"cdata:block" : ["<![CDATA[", "]]>"] ,"open_tag" : "RE::/<((?:[_a-zA-Z][_a-zA-Z0-9\\-]*:)?[_a-zA-Z][_a-zA-Z0-9\\-]*)\\b/" ,"close_tag" : "RE::/<\\/((?:[_a-zA-Z][_a-zA-Z0-9\\-]*:)?[_a-zA-Z][_a-zA-Z0-9\\-]*)>/" ,"attribute" : "RE::/[_a-zA-Z][_a-zA-Z0-9\\-]*/" ,"string:line-block" : [["\""], ["'"]] ,"number" : ["RE::/[0-9]\\d*/", "RE::/#[0-9a-fA-F]+/"] ,"atom" : ["RE::/&#x[a-fA-F\\d]+;/", "RE::/&#[\\d]+;/", "RE::/&[a-zA-Z][a-zA-Z0-9]*;/"] ,"text" : "RE::/[^<&]+/" // actions ,"@tag:action" : {"context":true} ,"tag@:action" : {"context":false} ,"@unique_id:action" : {"unique":["xml", "$1"],"msg":"Duplicate id value \"$0\"","mode":"hash"} ,"@unique_att:action" : {"unique":["att", "$0"],"msg":"Duplicate attribute \"$0\"","mode":"hash","in-context":true} ,"@tag_opened:action" : {"push":"<$1>","ci":true} ,"@tag_closed:action" : {"pop":"<$1>","ci":true,"msg":"Tags \"$0\" and \"$1\" do not match"} ,"@tag_autoclosed:action" : {"pop":null} ,"@autocomplete:action" : {"define":["autocomplete","$1"],"msg":false,"autocomplete":true,"mode":"hash"} ,"@out_of_place:error" : "\"$2$3\" can only be at the beginning of XML document" }, // Syntax model (optional) "Syntax" : { "tag_att" : "'id'.attribute @unique_att '=' string @unique_id | attribute @unique_att '=' (string | number)" ,"start_tag" : "open_tag.tag @tag @autocomplete @tag_opened tag_att* ('>'.tag | '/>'.tag @tag_autoclosed) tag@" ,"end_tag" : "close_tag.tag @autocomplete @tag_closed" ,"xml" : "(^^1 declaration? doctype?) (declaration.error @out_of_place | doctype.error @out_of_place | comment | meta | cdata | start_tag | end_tag | atom | text)*" }, // what to parse and in what order "Parser" : [ ["xml"] ] }; // 2. parse the grammar into a Codemirror syntax-highlight mode var xml_mode = CodeMirrorGrammar.getMode( xml_grammar ); // 3. use it with Codemirror CodeMirror.defineMode("xml", xml_mode); // enable user-defined code folding in the specification (new feature) xml_mode.supportCodeFolding = true; CodeMirror.registerHelper("fold", xml_mode.foldType, xml_mode.folder); // enable user-defined code matching in the specification (new feature) xml_mode.supportCodeMatching = true; xml_mode.matcher.options = {maxHighlightLineLength:1000}; // default CodeMirror.defineOption("matching", false, function( cm, val, old ) { if ( old && old != CodeMirror.Init ) { cm.off( "cursorActivity", xml_mode.matcher ); xml_mode.matcher.clear( cm ); } if ( val ) { cm.on( "cursorActivity", xml_mode.matcher ); xml_mode.matcher( cm ); } }); // enable syntax lint-like validation in the grammar xml_mode.supportGrammarAnnotations = true; CodeMirror.registerHelper("lint", "xml", xml_mode.validator); // enable user-defined autocompletion (if defined) xml_mode.supportAutoCompletion = true; CodeMirror.commands['my_autocompletion'] = function( cm ) { CodeMirror.showHint(cm, xml_mode.autocompleter, {prefixMatch:true, caseInsensitiveMatch:false}); }; // this also works (takes priority if set) xml_mode.autocompleter.options = {prefixMatch:true, caseInsensitiveMatch:false}; // or for context-sensitive autocompletion, extracted from the grammar xml_mode.autocompleter.options = {prefixMatch:true, caseInsensitiveMatch:false, inContext:true}; // or for dynamic (context-sensitive) autocompletion, extracted from user actions xml_mode.autocompleter.options = {prefixMatch:true, caseInsensitiveMatch:false, inContext:true|false, dynamic:true}; var editor = CodeMirror.fromTextArea(document.getElementById("code"), { mode: "xml", lineNumbers: true, indentUnit: 4, indentWithTabs: false, lint: true, // enable lint validation matching: true, // enable token matching, e.g braces, tags etc.. extraKeys: {"Ctrl-Space": 'my_autocompletion', "Ctrl-L": "toggleComment"}, foldGutter: true, gutters: ["CodeMirror-lint-markers", "CodeMirror-linenumbers", "CodeMirror-foldgutter"] });

Result:

see also:

• Abacus Computer Algebra and Symbolic Computation System for Combinatorics and Algebraic Number Theory for JavaScript and Python
• SciLite Scientific Computing Environment similar to Octave/Matlab in pure JavaScript
• TensorView view array data as multidimensional tensors of various shapes efficiently
• FILTER.js video and image processing and computer vision Library in pure JavaScript (browser and nodejs)
• HAAR.js image feature detection based on Haar Cascades in JavaScript (Viola-Jones-Lienhart et al Algorithm)
• HAARPHP image feature detection based on Haar Cascades in PHP (Viola-Jones-Lienhart et al Algorithm)
• Fuzzion a library of fuzzy / approximate string metrics for PHP, JavaScript, Python
• Matchy a library of string matching algorithms for PHP, JavaScript, Python
• Regex Analyzer/Composer Regular Expression Analyzer and Composer for PHP, JavaScript, Python
• Xpresion a simple and flexible eXpression parser engine (with custom functions and variables support), based on GrammarTemplate, for PHP, JavaScript, Python
• GrammarTemplate grammar-based templating for PHP, JavaScript, Python
• codemirror-grammar transform a formal grammar in JSON format into a syntax-highlight parser for CodeMirror editor
• ace-grammar transform a formal grammar in JSON format into a syntax-highlight parser for ACE editor
• prism-grammar transform a formal grammar in JSON format into a syntax-highlighter for Prism code highlighter
• highlightjs-grammar transform a formal grammar in JSON format into a syntax-highlight mode for Highlight.js code highlighter
• syntaxhighlighter-grammar transform a formal grammar in JSON format to a highlight brush for SyntaxHighlighter code highlighter
• MOD3 3D Modifier Library in JavaScript
• Geometrize Computational Geometry and Rendering Library for JavaScript
• Plot.js simple and small library which can plot graphs of functions and various simple charts and can render to Canvas, SVG and plain HTML
• CanvasLite an html canvas implementation in pure JavaScript
• Rasterizer stroke and fill lines, rectangles, curves and paths, without canvas
• Gradient create linear, radial, conic and elliptic gradients and image patterns without canvas
• css-color simple class to parse and manipulate colors in various formats
• PatternMatchingAlgorithms library of Pattern Matching Algorithms in JavaScript using Matchy
• SortingAlgorithms library of Sorting Algorithms in JavaScript