Home
flexmark-javaJava library for parsing and rendering Markdown text according to the CommonMark specification (and some extensions).
Provides classes for parsing input to an abstract syntax tree of nodes (AST), visiting and manipulating nodes, and rendering to HTML. It is a rework of commonmark-java to generate AST which allows recreating the original source, full source position references for all nodes and easier JetBrains Open API PsiTree generation.
It has a similar API to commonmark-java and many added extensions and enhancements to the core with the following features:
- Small (minimal dependencies)
- Fast, about 25 times faster than pegdown and 10 times faster than intellij-markdown only about 35%-40% slower than commonmark-java after adding easy source tracking
- Flexible (manipulate the AST after parsing, customize HTML rendering)
- Extensible (tables, strikethrough, autolinks and more, see below)
- Complete source position tracking in the AST with all source elements represented in the AST
- Extensible common options setting mechanism to allow easy parser, renderer and extension behaviour modification.
- Practically all behavior of the core parser can be modified, including the default InlineParser.
- Java 8 or above
- The core has no dependencies; for extensions, see below
-
The project is not yet on Maven
-
Java compatibility raised to 1.8 so that lambdas could be used
-
Android compatibility neglected for now
-
No attempt is made to keep API backward compatibility to the original project.
This is a work in progress with many API changes.
| Feature | flexmark-java | commmonmark-java | pegdown |
|---|---|---|---|
| Relative parse time (less is better) | ✔️ 1x | ✔️ 0.6x to 0.7x | ❌ 25x average, 20,000x to ∞ for pathological input (2) |
| All source elements in the AST | ✔️ | ❌ | ✔️ |
| AST elements with source position | ✔️ | ❌ | ✔️ with some errors and idiosyncrasies |
| AST can be easily manipulated | ✔️ AST post processing is an extension mechanism | ✔️ AST post processing is an extension mechanism | ❌ not an option. No node's parent information, children as List<>. |
| AST elements have detailed source position for all parts | ✔️ | ❌ | ❌ only node start/end |
| Can disable core parsing features | ✔️ | ❌ | ❌ |
| Core parser implemented via the extension API | ✔️ | ❌ instanceOf tests for specific block parser and node classes |
❌ core exposes few extension points |
| Easy to understand and modify parser implementation | ✔️ | ✔️ | ❌ one massive PEG parser with complex interactions (2) |
| Parsing of block elements is independent from each other | ✔️ (1) | ✔️ (1) | ❌ everything in one PEG grammar |
| Uniform configuration across: parser, renderer and all extensions | ✔️ | ❌ none beyond extension list | ❌ int bit flags for core, none for extensions |
| Parsing performance optimized for use with extensions | ✔️ | ❌ parsing performance for core, extensions do what they can | ❌ performance is not a feature |
| Feature rich with many configuration options and extensions out of the box | ✔️ | ❌ limited extensions, no options | ✔️ |
| Dependency definitions for processors to guarantee the right order of processing | ✔️ | ❌ order specified by extension list ordering, error prone | ❌ not applicable, core defines where extension processing is added |
pathological input of 10,000 [ parses in 11ms, 10,000 nested [ ] parse in 450ms
pathological input of 17 [ parses in 650ms, 18 [ in 1300ms
The motivation for creating this project was to replace pegdown as the parser for Markdown Navigator plugin for JetBrains line of IDEs. pegdown has many performance issues that cannot be resolved because of its implementation. Additionally, pegdown parsing of markdown elements interact with each other and it can only parse the full file with no ability to mark a spot beyond which no roll back should occur. For some sources this causes pegdown to go into exponential parse times and in a few cases into infinite parsing loops which could only be resolved by changing the grammar to be no longer markdown compatible.
Since I needed to implement a lot of extensions to make this parser a superset of pegdown I wanted to improve the ability of extensions to modify the behaviour of the parser to allow implementation of any markdown dialect through the extension mechanism. I also wanted to remove boiler plate code and make tests in extensions use the commonmark spec.txt format but with the addition of having the AST as part of the test so that the AST could be validated for every construct and every extension.
The end goal is to have a parser that can be easily extended to be compatible with: