> A Markdown parser written in Go. Easy to extend, standard compliant, well structured.
> A Markdown parser written in Go. Easy to extend, standards-compliant, well-structured.
goldmark is compliant with CommonMark 0.29.
Motivation
----------------------
I need a Markdown parser for Go that meets following conditions:
I needed a Markdown parser for Go that satisfies the following requirements:
- Easy to extend.
- Markdown is poor in document expressions compared with other light markup languages like reStructuredText.
- Markdown is poor in document expressions compared to other light markup languages such as reStructuredText.
- We have extensions to the Markdown syntax, e.g. PHP Markdown Extra, GitHub Flavored Markdown.
- Standardcompliant.
- Standards-compliant.
- Markdown has many dialects.
- GitHub Flavored Markdown is widely used and it is based on CommonMark aside from whether CommonMark is good specification or not.
- CommonMark is too complicated and hard to implement.
- Wellstructured.
- AST based, and preserves source position of nodes.
- GitHub-Flavored Markdown is widely used and is based upon CommonMark, effectively mooting the question of whether or not CommonMark is an ideal specification.
- CommonMark is complicated and hard to implement.
- Well-structured.
- AST-based; preserves source position of nodes.
- Written in pure Go.
[golang-commonmark](https://gitlab.com/golang-commonmark/markdown) may be a good choice, but it seems to be a copy of [markdown-it](https://github.com/markdown-it).
[blackfriday.v2](https://github.com/russross/blackfriday/tree/v2) is a fast and widely used implementation, but it is not CommonMark compliant and cannot be extended from outside of the package since its AST uses structs instead of interfaces.
[blackfriday.v2](https://github.com/russross/blackfriday/tree/v2) is a fast and widely-used implementation, but is not CommonMark-compliant and cannot be extended from outside of the package, since its AST uses structs instead of interfaces.
Furthermore, its behavior differs from other implementations in some cases, especially regarding lists: ([Deep nested lists don't output correctly #329](https://github.com/russross/blackfriday/issues/329), [List block cannot have a second line #244](https://github.com/russross/blackfriday/issues/244), etc).
Furthermore, its behavior differs from other implementations in some cases, especially regarding lists: [Deep nested lists don't output correctly #329](https://github.com/russross/blackfriday/issues/329), [List block cannot have a second line #244](https://github.com/russross/blackfriday/issues/244), etc.
This behavior sometimes causes problems. If you migrate your Markdown text to blackfriday-based wikis from GitHub, many lists will immediately be broken.
This behavior sometimes causes problems. If you migrate your Markdown text from GitHub to blackfriday-based wikis, many lists will immediately be broken.
As mentioned above, CommonMark is too complicated and hard to implement, so Markdown parsers based on CommonMark barely exist.
As mentioned above, CommonMark is complicated and hard to implement, so Markdown parsers based on CommonMark are few and far between.
Features
----------------------
- **Standard compliant.** goldmark gets full compliance with the latest CommonMark spec.
- **Standards-compliant.** goldmark is fully compliant with the latest [CommonMark](https://commonmark.org/) specification.
- **Extensible.** Do you want to add a `@username` mention syntax to Markdown?
You can easily do it in goldmark. You can add your AST nodes,
parsers for block level elements, parsers for inline level elements,
transformers for paragraphs, transformers for whole AST structure, and
You can easily do so in goldmark. You can add your AST nodes,
parsers for block-level elements, parsers for inline-level elements,
transformers for paragraphs, transformers for the whole AST structure, and
renderers.
- **Performance.** goldmark performs pretty much equally to cmark,
- **Performance.** goldmark's performance is on par with that of cmark,
the CommonMark reference implementation written in C.
- **Robust.** goldmark is tested with [go-fuzz](https://github.com/dvyukov/go-fuzz), a fuzz testing tool.
- **Builtin extensions.** goldmark ships with common extensions like tables, strikethrough,
- **Built-in extensions.** goldmark ships with common extensions like tables, strikethrough,
task lists, and definition lists.
- **Depends only on standard libraries.**
@ -62,15 +62,15 @@ Usage
----------------------
Import packages:
```
```go
import (
"bytes"
"github.com/yuin/goldmark"
"bytes"
"github.com/yuin/goldmark"
)
```
Convert Markdown documents with the CommonMarkcompliant mode:
Convert Markdown documents with the CommonMark-compliant mode:
| `html.WithWriter` | `html.Writer` | `html.Writer` for writing contents to an `io.Writer`. |
| `html.WithHardWraps` | `-` | Render newlines as `<br>`.|
| `html.WithHardWraps` | `-` | Render newlines as `<br>`.|
| `html.WithXHTML` | `-` | Render as XHTML. |
| `html.WithUnsafe` | `-` | By default, goldmark does not render raw HTML and potentially dangerous links. With this option, goldmark renders these contents as written. |
| `html.WithUnsafe` | `-` | By default, goldmark does not render raw HTML or potentially dangerous links. With this option, goldmark renders such content as written. |
- [GitHub Flavored Markdown: Task list items](https://github.github.com/gfm/#task-list-items-extension-)
- [GitHub Flavored Markdown: Task list items](https://github.github.com/gfm/#task-list-items-extension-)
- `extension.GFM`
- This extension enables Table, Strikethrough, Linkify and TaskList.
- This extension does not filter tags defined in [6.11: Disallowed Raw HTML (extension)](https://github.github.com/gfm/#disallowed-raw-html-extension-).
If you need to filter HTML tags, see [Security](#security)
- This extension enables Table, Strikethrough, Linkify and TaskList.
- This extension does not filter tags defined in [6.11: Disallowed Raw HTML (extension)](https://github.github.com/gfm/#disallowed-raw-html-extension-).
If you need to filter HTML tags, see [Security](#security).
Example, using [xurls](https://github.com/mvdan/xurls):
```go
import "mvdan.cc/xurls/v2"
markdown := goldmark.New(
goldmark.WithRendererOptions(
html.WithXHTML(),
html.WithUnsafe(),
),
goldmark.WithExtensions(
extension.NewLinkify(
extension.WithLinkifyAllowedProtocols([][]byte{
[]byte("http:"),
[]byte("https:"),
}),
extension.WithLinkifyURLRegexp(
xurls.Strict(),
),
),
),
)
```
Security
--------------------
By default, goldmark does not render raw HTML and potentially dangerous URLs.
If you need to gain more control over untrusted contents, it is recommended to
By default, goldmark does not render raw HTML or potentially-dangerous URLs.
If you need to gain more control over untrusted contents, it is recommended that you
use an HTML sanitizer such as [bluemonday](https://github.com/microcosm-cc/bluemonday).
Benchmark
@ -238,11 +286,10 @@ You can run this benchmark in the `_benchmark` directory.
### against other golang libraries
blackfriday v2 seems to be fastest, but it is not CommonMark compliant, so the performance of
blackfriday v2 cannot simply be compared with that of the other CommonMark compliant libraries.
blackfriday v2 seems to be the fastest, but as it is not CommonMark compliant, its performance cannot be directly compared to that of the CommonMark-compliant libraries.
Though goldmark builds clean extensible AST structure and get full compliance with
CommonMark, it is reasonably fast and has lower memory consumption.
goldmark, meanwhile, builds a clean, extensible AST structure, achieves full compliance with
CommonMark, and consumes less memory, all while being reasonably fast.
```
goos: darwin
@ -268,21 +315,21 @@ iteration: 50
average: 0.0040964230 sec
```
As you can see, goldmark performs pretty much equally to cmark.
As you can see, goldmark's performance is on par with cmark's.
Extensions
--------------------
- [goldmark-meta](https://github.com/yuin/goldmark-meta): A YAML metadata
extension for the goldmark Markdown parser.
- [goldmark-highlighting](https://github.com/yuin/goldmark-highlighting): A Syntax highlighting extension
- [goldmark-highlighting](https://github.com/yuin/goldmark-highlighting): A syntax-highlighting extension
for the goldmark markdown parser.
- [goldmark-mathjax](https://github.com/litao91/goldmark-mathjax): Mathjax support for goldmark markdown parser
- [goldmark-mathjax](https://github.com/litao91/goldmark-mathjax): Mathjax support for the goldmark markdown parser
goldmark internal(for extension developers)
----------------------------------------------
### Overview
goldmark's Markdown processing is outlined as a bellow diagram.
goldmark's Markdown processing is outlined in the diagram below.
```
<Markdownin[]byte,parser.Context>
@ -313,10 +360,11 @@ goldmark's Markdown processing is outlined as a bellow diagram.
### Parsing
Markdown documents are read through `text.Reader` interface.
AST nodes do not have concrete text. AST nodes have segment information of the documents. It is represented by `text.Segment` .
AST nodes do not have concrete text. AST nodes have segment information of the documents, represented by `text.Segment` .
`text.Segment` has 3 attributes: `Start`, `End`, `Padding` .
6543
4 years ago
committed by
GitHub