How does the transforms system work — how are custom transforms registered and executed? #121

Question

How does the transforms system work — how are custom transforms registered and executed?

Answer

Markdown Magic works by scanning markdown for HTML comment blocks (e.g. <!-- docs TRANSFORM_NAME opt=val --> … <!-- /docs -->), running a transform function for each block, and replacing the block's inner content with whatever that function returns. A "transform" is just a named function; "plugins" are simply transforms (or factories that return transforms) you register by name.

1. What a transform is

A transform is a function with the signature (api) => string | Promise<string>. The single api argument (the "plugin API") is built in packages/block-transformer/src/index.js and includes:

• content — current inner content of the comment block
• options — key/value options parsed from the opening comment (e.g. src="./x.js" lines=1-10)
• transform — the transform's name
• srcPath / outputPath — file paths
• settings — the merged run config plus settings.regex patterns
• currentContent / originalContent and getters getCurrentContent(), getOriginalContent(), getOriginalBlock(), getBlockDetails()

Whatever the function returns replaces the block's contents. Non-string returns are coerced: numbers become strings, arrays/objects are JSON.stringify'd. Returning falsy leaves the existing content unchanged.

2. Built-in transforms (the default registry)

The built-ins live in packages/core/src/transforms/ and are collected into a plain object in packages/core/src/transforms/index.js:

TOC, sectionToc, CODE, FILE, REMOTE, fileTree, install (+ wordCount)

This object is the default transform registry.

3. How custom transforms are registered

You register transforms by passing a transforms object on the config to markdownMagic() — a map of name -> function:

markdownMagic('README.md', {
  transforms: {
    // <!-- docs customTransform optionOne=hi -->
    customTransform({ content, options }) {
      return `Replaced with ${options.optionOne}`
    },
    // A plugin is just a factory that returns a transform function
    pluginExample: require('./plugin-example')({ addNewLine: true }),
  }
})

Registration happens in packages/core/src/index.js:

const useTransforms = Object.assign({}, defaultTransforms, transforms)

Key points:

• Your custom transforms are merged over the defaults, so a custom transform can override a built-in of the same name.
• The name is the object key — that key is what you write after START in the comment block. Lookup is case-insensitive with a lowercase fallback (transforms[name] || transforms[name.toLowerCase()], see getTransform).
• A "plugin" is conventionally a function that returns a transform function ((pluginOptions) => (api) => string), so you can configure it at registration time. There's no separate registration API — plugins are just entries in the same transforms object, and they run in registration order.
• If you have an md.config.js / markdown.config.js, its transforms are loaded automatically (CLI) via the config loader.

4. How transforms are executed

Execution flows core → processContents → blockTransformer:

1. Parse — packages/core/src/process-contents.js delegates to blockTransformer (comment-block-transformer), which calls parseBlocks (comment-block-parser) to find all comment blocks. Each block's first token is treated as the transform name (firstArgIsType: true); options are parsed into options.
2. Plan / validate — core builds a plan of blocks to run. Blocks whose transform name isn't in the registry are collected as missingTransforms. By default these are skipped with a warning; with failOnMissingTransforms: true the run errors (Transform "X" ... does not exist).
3. Sort — sortTransforms reorders so that TOC and sectionToc run last (they need the rest of the document finalized first). Position-based replacement offsets keep replacements correct even when blocks run out of document order.
4. Run sequentially — blockTransformer reduces over the blocks with an async accumulator so each transform sees the up-to-date document. For each block it:
   • applies any beforeMiddleware,
   • looks up the function via getTransform(name, transforms),
   • calls it with the plugin API object (awaiting, so transforms may be async),
   • applies afterMiddleware,
   • normalizes the return value, re-indents, and splices it back into the document using position offsets.
5. Result — returns { isChanged, updatedContents, transforms, missingTransforms, ... }. Core then writes the file back (or to outputDir), honoring flags like applyTransformsToSource, removeComments, and dryRun.

5. Inline vs block transforms

Transforms also work inline (no line break between open/close), e.g.
<!-- docs (INLINE_EXAMPLE) -->**⊂◉‿◉つ**<!-- /docs -->. The same registered function is used; the transformer detects single-line context and skips added indentation/newlines.

Where to look in the code

• packages/core/src/index.js — config handling, Object.assign({}, defaultTransforms, transforms) merge, planning/validation, file writing
• packages/core/src/transforms/index.js — the default built-in registry
• packages/core/src/transforms/* — individual built-in transform implementations
• packages/core/src/process-contents.js → packages/block-transformer/src/index.js — the execution engine (blockTransformer, getTransform, sortTransforms, middleware, plugin API construction)
• packages/block-parser/ (comment-block-parser) — parses the comment blocks and their options
• README "Adding Custom Transforms" / "Plugin Example" / "Inline transforms" sections — user-facing docs

This issue was generated in response to a dictated question about the repository (E2E test, plan 25). No code changes were made.