HMD Language Specification¶

This document specifies HyperMarkDown 0.1, a syntactic extension of CommonMark 0.31.2.

Status of this document¶

This is the normative text for the language. It defines what a HyperMarkDown document is, what its constructs mean, and what an implementation must do with them. It does not describe any particular program, and no program's behaviour overrides it.

The language is versioned apart from the tools that implement it. A release of an implementation does not imply a new language version, and a language version does not wait for one. The version above changes when the constructs below change, and nothing else moves it; what each version changed is recorded in the language's changelog.

The baseline is named by version rather than as "markdown" because "markdown" names a family, not a standard, and a superset of an unnamed dialect specifies nothing. Adopting a later CommonMark is an edit to this document, not something that happens to it.

Prose that teaches the constructs rather than defining them is hmd-tutorial. The numbered inventory of every construct, including the ones this version does not have, is hmd-feature-list. Where any of those disagree with this document, this document is right.

Conformance¶

The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, and MAY are to be interpreted as described in RFC 2119.

A conforming implementation MUST accept every document this specification defines as well-formed, MUST assign it the meaning given here, and MUST report each condition listed under Diagnostics with the identifier and severity given there. An implementation MAY provide constructs beyond this specification, but MUST NOT assign a different meaning to any construct defined here.

CommonMark compatibility. Every document CommonMark 0.31.2 accepts is syntactically valid HyperMarkDown. CommonMark constructs retain their CommonMark meaning except where their bytes form a HyperMarkDown construct defined by this specification; HyperMarkDown also assigns semantics to some sequences CommonMark treats as literal text. Where this document and CommonMark 0.31.2 disagree about inherited syntax, CommonMark is right; the inherited syntax is specified at https://spec.commonmark.org/0.31.2/ and is not restated here.

The consequence worth stating plainly: renaming a .md file to .hmd preserves its syntactic validity, but can change how it is interpreted. For example, [[foo]] is literal text in CommonMark and a wikilink in HyperMarkDown, while a leading --- block that matches the grammar below is frontmatter in HyperMarkDown.

Arbitration. Where two implementations disagree about behaviour this document leaves under-determined, the language-neutral conformance corpus at examples/conformance/cases/ decides, and a case that starts passing or failing unexpectedly is a defect in the implementation rather than in the corpus.

Documents¶

A card is one file with the .hmd extension. It MUST be valid UTF-8.

A card consists of an OPTIONAL frontmatter block followed by CommonMark content. Nothing else is structurally required: a card with no frontmatter and no HyperMarkDown construct is a well-formed card.

Cards are addressed within a single rooted subtree of the filesystem, the namespace root, defined under The namespace. A file is a member of the namespace if and only if it lies inside that root and has the .hmd extension. Files with any other extension MUST be ignored for the purposes of this specification: they are neither link targets nor inputs to checking, which is what allows an ordinary README.md to sit inside a namespace without becoming part of it.

Lexical model¶

HyperMarkDown constructs are extracted by scanning, not by fully parsing the document as CommonMark. An implementation extracts only the constructs this specification owns and leaves every other byte to whatever renders the document.

Before extracting any construct, an implementation MUST mask the following regions, so their contents never yield a link, an embed, or an anchor: fenced code blocks (both ``` and ~~~), inline code spans, and HTML comments ().
Indented code blocks MUST NOT be masked. A four-space indent is also how a callout body and a footnote continuation are written, and masking it would silently drop real links out of ordinary prose. An author who needs indented code has fences.
Raw inline HTML is NOT masked. A <code> element written by hand is scanned like any other text, so it is not a way to show a construct without invoking it. The masked regions above are the only ones that are.
Masking MUST preserve byte offsets, so that every extracted construct carries an exact source span.
Every construct MUST record a span of (start_offset, end_offset, line, column), with line and column 1-indexed.
Scanning MUST NOT depend on filesystem iteration order, on the contents of any other file, or on any state outside the card's own bytes. Everything cross-file belongs to resolution.

Known divergence. A line-oriented masking scanner will differ from a full CommonMark parser on pathological input — a wikilink inside an HTML block, or a fence opened inside a blockquote. This is permitted in 0.1: the construct set is small and the divergences are diagnosable. It is not a licence to diverge on ordinary input.

Grammar¶

The constructs below are the format's own. Fenced blocks and code spans are literal by the rule above, which is the only mechanism for writing a construct rather than invoking one.

embed        := "!" wikilink
wikilink     := "[[" target [ "|" display ] "]]"
target       := page_ref [ fragment ]
page_ref     := absolute | relative | unqualified
absolute     := "/" segment { "/" segment }
path_component := segment | ".."
relative     := "./" [ path_component { "/" path_component } ]
              | "../" [ path_component { "/" path_component } ]
unqualified  := bare_name | unqualified_path
bare_name    := segment
unqualified_path := segment "/" segment { "/" segment }
fragment     := "#" heading_text
              | "#^" block_id
segment      := 1*( any character except "/" or reserved )
heading_text := heading_first { heading_char }
heading_first := any character except "[" / "]" / "|" / "^" / newline
heading_char := any character except "[" / "]" / "|" / newline
display      := 1*( any character except "[" / "]" / newline )
block_id     := ( ALPHA / DIGIT ) *63( ALPHA / DIGIT / "_" / "-" )
reserved     := "[" | "]" | "|" | "#" | "^" | newline

A bare name is exactly one segment, with no /, ./, ../, leading /, or namespace qualifier. A page reference with two or more segments and no leading path marker is an unqualified path. Bare names and unqualified paths are collectively unqualified references and use the same search phases below; only a bare name can match a named import.
The reserved namespace:path form is a namespace-qualified reference, not a bare name. Namespace qualification is not part of the 0.1 grammar or resolution algorithm.
The reserved character set inside a target is exactly [, ], |, #, ^, and newline. These MUST NOT appear literally in a segment.
/ is the segment separator and MUST NOT appear inside a segment.
A heading_text MUST contain at least one non-whitespace character. # and ^ MAY appear after its first character; [, ], |, and newline MUST NOT appear because they delimit or terminate the surrounding wikilink.
./ and ../ with no following component are valid relative references to the current and parent directory. They resolve only when that directory has a folder note, and the ordinary namespace containment rule still applies.
No escape mechanism exists in this version. A target that needs a reserved character is a malformed link, reported as HMD010. This is a known gap rather than a design position.
A page_ref MAY carry a trailing .hmd; an implementation MUST strip it before matching. Writing the extension is redundant but harmless.
A display MAY be meaningless but MUST NOT be empty of characters. The form with an empty display is malformed and reported as HMD010.

Block anchors¶

A block anchor is a trailing ^block_id at the end of a block's last line, preceded by at least one space and followed only by the line ending.

The anchor and its leading whitespace are NOT part of the block's content, and MUST NOT be rendered.
An identifier's scope is the card it appears in. Two cards MAY use the same identifier without colliding; two anchors in one card MUST NOT, and a duplicate is reported as HMD006.

Frontmatter block¶

A card MAY open with a frontmatter block: a --- fence that MUST begin at byte 0 of the file and MUST close with a --- line.

The block MUST parse as YAML, and MUST be a mapping at the top level. Otherwise it is reported as HMD009.
Exactly four top-level keys are reserved for the toolchain: tags, use, import, and nav. They are specified under Frontmatter.
Every other key is user-owned and MUST NOT be interpreted. An implementation MUST still expose the whole mapping to its consumers; the language ships mechanisms, not vocabulary.

Heading anchors¶

Fragments that name a heading must agree with what the renderer emits, so anchor assignment is delegated rather than invented.

A heading's slug MUST be produced by Python-Markdown's toc slugify with - as the separator, equivalent to markdown.extensions.toc.slugify(text, "-").
Collisions within one card MUST be deduplicated exactly as the toc extension does, by appending _1, _2, … in document order. A fragment naming that text therefore addresses the first such heading, and the _1 form the second.
A heading_text fragment MUST be matched by slugifying it and comparing against the card's assigned slugs. Capitalisation and spacing are therefore not significant: the written heading and its slug are equally valid fragments.
Two headings in one card that share a slug MUST be reported as HMD011, a warning. Resolution still succeeds by the deduplication rule above.

Reusing the renderer's own rule is what keeps a link and its rendered destination from disagreeing. An implementation that assigns slugs differently does not conform.

The namespace¶

The namespace is a single rooted subtree. In version 0.1, nothing outside it is addressable by an HMD page reference. Ordinary Markdown links remain able to address files and URLs outside that subtree.

The project root is the nearest ancestor directory containing a .hmd/ directory, falling back to the nearest ancestor containing .git.
The namespace root is the wiki setting of .hmd/config.toml, resolved relative to the project root, defaulting to doc/wiki.
An implementation MUST offer a way to name the namespace root directly, overriding both.
A resolved target MUST lie inside the namespace root. One that does not is reported as HMD003.
Path normalization MUST happen before that containment check, and symbolic links MUST NOT be followed out of the root.

A directory inside the namespace is a module: it is the unit that scopes a name and the unit that inherits configuration. A directory MAY carry a folder note, an index.hmd. A reference to the directory and a reference to its explicit index path resolve to that one card.

Name resolution¶

A page_ref is a local card reference. An absolute or relative path is followed directly. An unqualified reference is searched for and resolves to exactly one card or a diagnosable failure.

Unqualified resolution checks named imports, then runs a spine walk — pure path arithmetic from the card outward to the root — followed by imported search paths and, only if those find nothing, a single sweep of the tree. The spine and sweep answer different questions. The spine answers what does this reference mean here, which must be predictable enough to hold in your head. The sweep answers does this reference exist anywhere, which is a convenience and is allowed to fail loudly.

Binding an unqualified reference in one directory¶

Every search phase binds an unqualified reference identically, so a folder note behaves the same everywhere.

bind(dir, parts) -> Resolved(path) | NotFound

  file   := dir / join(parts) + ".hmd"
  folder := dir / join(parts)

  if file exists:                          return Resolved(file)
  if folder is a directory
     and folder/index.hmd exists:          return Resolved(folder/index.hmd)
  return NotFound

A target naming a directory MUST resolve to that directory's index.hmd. The directory-form reference and the explicit index reference therefore resolve to the same card.
If a card and a folder note claim the same name as siblings, the pair MUST be reported as HMD012. For deterministic diagnostics and rendering after that error, an implementation MUST select the file card. The selection does not make the collision valid.

The algorithm¶

resolve(target, source, root) -> Resolved(path) | Unresolved | Ambiguous(paths) | Escapes

  stem  := target.page_ref with any ".hmd" suffix removed
  parts := stem split on "/"
  bindings := named_imports(source)                      # name -> card
  paths    := wildcard_imports(source)                   # ordered dirs
  mode     := discovery_mode(source)

  1. ABSOLUTE — stem starts with "/"
       return bind(root, parts) or Unresolved

  2. RELATIVE — stem starts with "./" or "../"
       base := normalize(dirname(source) / stem)
       if base is outside root: return Escapes
       return bind(dirname(base), [basename(base)]) or Unresolved

  3. UNQUALIFIED — anything else

       # Phase 0 — named imports. The most explicit thing wins.
       if size(parts) == 1 and parts[0] in bindings:
           return Resolved(bindings[parts[0]])

       # Phase 1 — spine walk. Non-recursive, nearest first.
       spine := [dirname(source), parent(dirname(source)), ..., root]
       for dir in spine:
           r := bind(dir, parts)
           if r is Resolved: return r

       # Phase 2 — imported search paths, in declaration order.
       for dir in paths:
           r := bind(dir, parts)
           if r is Resolved: return r

       if mode == "spine": return Unresolved      # autodiscovery off

       # Phase 3 — autodiscovery. Runs at most once.
       if mode == "recursive":
           candidates := descendants(dirname(source), "*.hmd")
       else:                                      # mode == "both"
           candidates := descendants(root, "*.hmd")
       matches := { p in candidates
                    : addressable_parts(p) ends with parts }
       if size(matches) == 1: return Resolved(the match)
       if size(matches) >  1: return Ambiguous(sorted(matches))
       return Unresolved

addressable_parts(p) is p's root-relative parts without the suffix, plus — when p is a folder note — the parts of its directory, so that an unqualified reference means the same thing in the sweep as it does on the spine.

Rules the algorithm must satisfy¶

Resolution follows the phase order above. A named import is evaluated before the spine and MAY shadow a local card. The spine is evaluated before wildcard import origins, so a wildcard import MUST NOT override either a named import or a spine match. Because wildcard imports precede autodiscovery, adding one MAY change a reference that previously resolved through autodiscovery.
The spine MUST run from the card's own directory outward to the root, inclusive of both ends, and each directory MUST be probed non-recursively. The spine never searches sideways into a sibling module. For a bare name it probes the card's own directory and each ancestor directory in turn.
Imported search paths MUST be probed in declaration order, non-recursively. Where two of them hold the name, the earlier declaration wins; because that is easy to write by accident, the shadowed candidate MUST be reported as HMD016.
The sweep MUST run only when the earlier phases found nothing, and MUST run at most once. Every match it produces is off-spine and off-path by construction, so the phases can never disagree.
Complete unqualified-reference resolution is not confined to the spine. When the earlier phases find nothing and autodiscovery is enabled, the sweep MAY resolve a unique match in a sibling module or elsewhere in its configured scope.
The sweep MUST address folder notes by their directory name, exactly as binding does. Without this, one reference would mean two different things depending on where it was written.
In the sweep all matches rank equally. A shallower match MUST NOT beat a deeper one, and more than one match MUST be reported as HMD002 rather than resolved. The remedy is to qualify the link, not to memorise a tie-break.
Candidate lists MUST be sorted by root-relative POSIX path, so that a diagnostic is byte-identical across platforms and across runs.
An unresolved target is not a failure. It is a forward reference, reported as HMD001, a warning. Writing forward is how a knowledge base grows.

Fragments¶

Once a card is resolved, its fragment is matched against that card.

A heading_text fragment MUST match by the rule under Heading anchors. One that matches nothing is reported as HMD004.
A block_id fragment MUST match a block anchor in the resolved card. One that matches nothing is reported as HMD005.
A fragment MUST NOT influence which card is resolved. Resolution selects the card; the fragment then selects within it.

Frontmatter¶

Configuration is two-level, and the levels answer different questions. A card says whether a feature applies to it; project configuration says how that feature behaves. The ordering of resolution phases is uniform across the tree; a card may disable the final autodiscovery phase.

---
tags: [area/auth, status/accepted]
use: [no_autodiscovery]
import:
  - from /shared import tokens as shared-tokens
  - from ../billing import invoices
  - from /glossary import *
nav:
  order: 10
  visibility: public
---

`tags` — what a card is about¶

tags is a list of strings. A / separates hierarchy levels, so area/backend is a descendant of area. A malformed value is reported as HMD013.
Tags MUST NOT participate in resolution. A wikilink answers where a card lives; a tag answers what it is about. Collapsing the two axes breaks both, so a tag is not a link target and a target beginning with # is malformed (HMD010) — # is reserved for fragments.
Tag semantics beyond the shape — generated tag pages, hierarchy queries — are not specified in this version. Only the slot is fixed, so that authors can tag against a stable shape.

`use` — per-card feature toggles¶

use accepts a string or a list of strings. Each entry names a feature; prefixing no_ disables it. A feature not mentioned takes its project default.
The only feature this version defines is autodiscovery: whether the sweep phase of resolution runs at all. Disabling it restricts a card to its imports and its spine, so every dependency the card has is visible in its header or in the link itself.
use inherits. A use list in a directory's folder note applies to that directory's whole subtree; the nearest declaration wins, and a card overrides its directory. The folder note supplies module-level configuration; the directory is already a module by definition.
An unrecognised feature name MUST be reported as HMD013 rather than ignored. Silently dropping a misspelling hands the author the default while they believe they configured something else.

`import` — explicit name binding¶

import is a list of one-line statements in the grammar below, not nested YAML.

import_stmt := "from" WS ref WS "import" WS import_list
import_list := "*" | binding *( "," WS binding )
binding     := name [ WS "as" WS alias ]
ref         := absolute | relative          ; never unqualified
name        := segment
alias       := segment

The two forms do different jobs:

A named binding binds one name. The local name — the alias if given, otherwise the imported name — resolves to that one card and to nothing else.
A wildcard adds a search origin. That directory joins the list of directories an unqualified reference is probed against, after the spine. It does NOT eagerly bind the directory's contents, so a card added there later becomes reachable without editing the importing card.
A ref MUST be absolute or relative, never unqualified. An unqualified ref would need resolving by the very algorithm the import feeds.
An imported origin MUST be probed non-recursively, matching the spine, so the reachable set stays legible from a directory listing.
A named binding applies only to bare names. A binding names a card, not a module, so a multi-segment target falls through it. Search origins carry no such restriction.
Two named bindings of the same local name MUST be reported as HMD015. Last-wins is exactly the guessing this language forbids, and unlike the ordered-origin case there is no declaration order to appeal to: both bindings claim the identical name outright.
An import whose ref does not exist MUST be reported as HMD015 at the declaration, not deferred to first use. A wrong import is a defect in the header regardless of whether the body exercises it.
A malformed statement MUST be reported as HMD014.
import does NOT inherit. Unlike use, an import in a folder note binds names for that file alone. Inherited bindings would make an unqualified reference in a leaf card resolvable only by reading a file the author may never have opened — precisely the action at a distance that imports exist to eliminate.

`nav` — placement and publication¶

nav is a mapping describing where a card sits in a published site. It has two keys. It is a mapping rather than a scalar so that a further placement key can be added without inventing a second spelling for the first; a scalar nav MUST therefore be reported as HMD013 rather than accepted, because a card carrying it asked to be ordered and would otherwise silently sort last.

order is an integer. Cards that set it sort ahead of cards that do not, ascending; ties fall back to the default order, which is folder notes first and then cards by root-relative POSIX path.

visibility takes public or private and gates publication. Any other value MUST be reported as HMD013 rather than coerced, because guessing publishes a card nobody asked to publish.

Publication is opt-in. A card reaches a published site only if its effective visibility is public; the default is private. Both errors are possible, but they are not symmetric: a card that should have shipped and did not is visible to its author on the next build, while one that shipped and should not have is a leak nobody looks for.
An unpublished card MUST NOT be registered at all — no page, no address. Being absent from a sidebar is not sufficient: a page reachable by typing its address is published, whatever the sidebar says.
visibility inherits the way use does: the card's own value, then the nearest ancestor folder note that sets one, then the default. The corollary is worth stating plainly: public on a root folder note publishes that entire subtree, and a card opts out with its own private.
A published card that links to an unpublished one MUST render as an unresolved link and MUST be reported as HMD017, a warning. This is a work item, not a defect that should stop a build.
A published card that embeds an unpublished one MUST NOT expand it. Expansion copies the target's bytes into the page, which is the one outcome the gate exists to prevent; it degrades to the same unresolved link and the same HMD017. Expansion itself stays policy-free and takes the predicate from its caller, since rendering a card named directly is not the same situation as building a site.
A malformed nav value, an unknown key inside it, or a non-integer order MUST be reported as HMD013.

Project configuration¶

The .hmd/ directory at the project root doubles as the root marker. This version reads exactly two settings from .hmd/config.toml and MUST ignore the rest of the file, so that the full schema can be specified later without invalidating existing trees.

wiki = "doc/wiki"        # namespace root, relative to the project root

[discovery]
autodiscovery = true     # whether the sweep runs at all
mode = "both"            # both (default) | recursive — how the sweep searches

autodiscovery is the project default for the use feature of the same name. A card overrides it freely in either direction.
mode decides how the sweep searches once it runs. both sweeps the whole namespace; recursive sweeps only the source card's own subtree.
mode MUST NOT be settable per card. A card MAY disable autodiscovery, but it MUST NOT redefine the scope or ranking of an enabled autodiscovery phase.

Precedence, most specific first — frontmatter always beats configuration, because the card is the most local place an author can say what they mean:

1. the card's own `use`
2. `use` in the nearest ancestor folder note
3. [discovery] autodiscovery in .hmd/config.toml
4. the built-in default — autodiscovery is ON

Absent .hmd/config.toml entirely, the namespace root is doc/wiki, autodiscovery is on, and the mode is both. A tree with no configuration at all therefore behaves the way a new author would expect: bare names find cards.

Embed expansion¶

An embed is any wikilink with a ! in front. It is resolved by the algorithm above and then expanded in place.

An embed of a whole card MUST expand to the target's full body with its frontmatter block removed.
An embed of a heading fragment MUST expand to the matched heading and every subsequent line up to, but excluding, the next heading of the same or higher level.
An embed of a block fragment MUST expand to the anchored block with the trailing anchor stripped.
Expansion MUST be recursive: embeds inside expanded content are themselves expanded.
An implementation MUST maintain a stack of resolved (path, fragment) pairs. Re-entering a pair already on the stack is a cycle and MUST be reported as HMD007 rather than allowed to recur.
Maximum expansion depth is 16. Exceeding it MUST be reported as HMD008. The limit is deep enough for legitimate composition and shallow enough to bound work on adversarial input.
Expansion MUST NOT shift heading levels in embedded content. Levels arrive exactly as the source wrote them. Level shifting changes document outlines and is not specified in this version; leaving it out keeps expansion textual and reversible.

Diagnostics¶

Rule identifiers are stable. They MUST NOT be renumbered once released, because suppression comments and CI configuration reference them.

ID	Severity	Condition
HMD001	warning	Link target does not resolve
HMD002	error	An unqualified reference matches more than one card in the sweep
HMD003	error	A target resolves outside the namespace root
HMD004	error	A heading fragment is not present in the resolved card
HMD005	error	A block identifier is not present in the resolved card
HMD006	error	Duplicate block anchor identifier within one card
HMD007	error	Embed cycle
HMD008	error	Embed depth limit (16) exceeded
HMD009	error	Frontmatter is not valid YAML, or not a top-level mapping
HMD010	error	Malformed wikilink or embed syntax
HMD011	warning	Two headings in one card share a slug
HMD012	error	A card and a folder note claim the same name
HMD013	error	Unrecognised `use` feature, or a malformed `tags` or `nav` value
HMD014	error	Malformed `import` statement
HMD015	error	An import `ref` does not resolve, or two named imports bind one local name
HMD016	warning	An unqualified reference matched two imported search paths; the earlier won
HMD017	warning	A published card links to or embeds one that is not published

An unresolved reference is a warning: writing forward is permitted, and the report is a to-do list. Multiple autodiscovery matches are an error because autodiscovery has no ranking rule. Explicitly ordered wildcard import origins use declaration order, and a shadowed match is reported as HMD016.

Each diagnostic MUST carry the rule identifier, the severity, a root-relative path with POSIX separators, a line, a column, and a message. An ambiguity MUST additionally carry its sorted candidate list.
Diagnostics MUST be emitted sorted by path, then line, then column, then rule.
Strict mode MUST NOT alter diagnostic records or their reported severity. For aggregate counts and command success or failure only, it MUST count every warning as an error. A strict and ordinary run over the same tree therefore contain the same diagnostics but differ in their summary and outcome.

Nothing else about a card is checked. This language checks the graph and makes no claim about prose, headings, or inherited Markdown.

Rendering¶

The language does not specify a renderer, and which program displays a card is not part of it. Two constraints are nonetheless normative, because a renderer that breaks them changes what the document means:

A target that does not resolve MUST still render as a link, marked as unresolved, rather than being dropped or raising. A missing card is a work item; erasing the author's reference hides it.
Embed expansion MUST follow Embed expansion above, including the depth and cycle bounds, regardless of the output format.

An implementation MAY erase the graph on output — resolving every link to a path and expanding every embed — producing ordinary Markdown or HTML that a reader consumes without knowing the format exists. Erasure is lossy: the boundary between a card and the content it embedded is gone, and so is the provenance of each link. That is correct for a shipping format and wrong for anything that needs to show a reader where content came from.

Not in this version¶

The following are rejected: a second diagram notation alongside the one the rich layer already provides.

The following are deferred, specified nowhere yet and not to be inferred from their absence: inline properties, highlights, templates and parameterised transclusion, a query language over the document graph, backlinks and generated category pages, and heading-level shifting on embed.

The following is reserved: a cross-namespace address of the form namespace:path/to/card. The spelling is claimed so that it cannot be given another meaning; what it denotes is specified in outline only, in HMD-0004, and an implementation of this version MUST treat it as a malformed target.

The gap an author is most likely to meet is the absence of an escape mechanism, in both directions. A target cannot contain a reserved character, and the only way to show a construct without invoking it is to put it in a code span or a fenced block, since those are the regions the scanner masks. Raw inline HTML is not masked and is therefore not a workaround.