Preprocessors - mdBook Documentation
Description: Create book from markdown files. Like Gitbook but implemented in Rust

# Preprocessors

A preprocessor is simply a bit of code which gets run immediately after the book is loaded and before it gets rendered, allowing you to update and mutate the book. Possible use cases are:

- Creating custom helpers like `{{#include /path/to/file.md}}`
- Updating links so `[some chapter](some_chapter.md)` is automatically changed to `[some chapter](some_chapter.html)` for the HTML renderer
- Substituting in latex-style expressions (`$$ \frac{1}{3} $$`) with their mathjax equivalents

## Hooking Into MDBook

MDBook uses a fairly simple mechanism for discovering third party plugins. A new table is added to `book.toml` (e.g. `preprocessor.foo` for the `foo` preprocessor) and then `mdbook` will try to invoke the `mdbook-foo` program as part of the build process.

While preprocessors can be hard-coded to specify which backend it should be run for (e.g. it doesn't make sense for MathJax to be used for non-HTML renderers) with the `preprocessor.foo.renderer` key.

    [book]
    title = "My Book"
    authors = ["Michael-F-Bryan"]

    [preprocessor.foo]
    # The command can also be specified manually
    command = "python3 /path/to/foo.py"
    # Only run the `foo` preprocessor for the HTML and EPUB renderer
    renderer = ["html", "epub"]

In typical unix style, all inputs to the plugin will be written to `stdin` as JSON and `mdbook` will read from `stdout` if it is expecting output.

The easiest way to get started is by creating your own implementation of the `Preprocessor` trait (e.g. in `lib.rs`) and then creating a shell binary which translates inputs to the correct `Preprocessor` method. For convenience, there is an example no-op preprocessor in the `examples/` directory which can easily be adapted for other preprocessors.

Example no-op preprocessor

    // nop-preprocessors.rs

    use crate::nop_lib::Nop;
    use clap::{App, Arg, ArgMatches, SubCommand};
    use mdbook::book::Book;
    use mdbook::errors::Error;
    use mdbook::preprocess::{CmdPreprocessor, Preprocessor, PreprocessorContext};
    use std::io;
    use std::process;

    pub fn make_app() -> App<'static, 'static> {
        App::new("nop-preprocessor")
            .about("A mdbook preprocessor which does precisely nothing")
            .subcommand(
                SubCommand::with_name("supports")
                    .arg(Arg::with_name("renderer").required(true))
                    .about("Check whether a renderer is supported by this preprocessor"),
            )
    }

    fn main() {
        let matches = make_app().get_matches();

        // Users will want to construct their own preprocessor here
        let preprocessor = Nop::new();

        if let Some(sub_args) = matches.subcommand_matches("supports") {
            handle_supports(&preprocessor, sub_args);
        } else if let Err(e) = handle_preprocessing(&preprocessor) {
            eprintln!("{}", e);
            process::exit(1);
        }
    }

    fn handle_preprocessing(pre: &dyn Preprocessor) -> Result<(), Error> {
        let (ctx, book) = CmdPreprocessor::parse_input(io::stdin())?;

        if ctx.mdbook_version != mdbook::MDBOOK_VERSION {
            // We should probably use the `semver` crate to check compatibility
            // here...
            eprintln!(
                "Warning: The {} plugin was built against version {} of mdbook, \
                 but we're being called from version {}",
                pre.name(),
                mdbook::MDBOOK_VERSION,
                ctx.mdbook_version
            );
        }

        let processed_book = pre.run(&ctx, book)?;
        serde_json::to_writer(io::stdout(), &processed_book)?;

        Ok(())
    }

    fn handle_supports(pre: &dyn Preprocessor, sub_args: &ArgMatches) -> ! {
        let renderer = sub_args.value_of("renderer").expect("Required argument");
        let supported = pre.supports_renderer(&renderer);

        // Signal whether the renderer is supported by exiting with 1 or 0.
        if supported {
            process::exit(0);
        } else {
            process::exit(1);
        }
    }

    /// The actual implementation of the `Nop` preprocessor. This would usually go
    /// in your main `lib.rs` file.
    mod nop_lib {
        use super::*;

        /// A no-op preprocessor.
        pub struct Nop;

        impl Nop {
            pub fn new() -> Nop {
                Nop
            }
        }

        impl Preprocessor for Nop {
            fn name(&self) -> &str {
                "nop-preprocessor"
            }

            fn run(&self, ctx: &PreprocessorContext, book: Book) -> Result<Book, Error> {
                // In testing we want to tell the preprocessor to blow up by setting a
                // particular config value
                if let Some(nop_cfg) = ctx.config.get_preprocessor(self.name()) {
                    if nop_cfg.contains_key("blow-up") {
                        return Err("Boom!!1!".into());
                    }
                }

                // we *are* a no-op preprocessor after all
                Ok(book)
            }

            fn supports_renderer(&self, renderer: &str) -> bool {
                renderer != "not-supported"
            }
        }
    }

## Hints For Implementing A Preprocessor

By pulling in `mdbook` as a library, preprocessors can have access to the existing infrastructure for dealing with books.

For example, a custom preprocessor could use the `CmdPreprocessor::parse_input()` function to deserialize the JSON written to `stdin`. Then each chapter of the `Book` can be mutated in-place via `Book::for_each_mut()`, and then written to `stdout` with the `serde_json` crate.

Chapters can be accessed either directly (by recursively iterating over chapters) or via the `Book::for_each_mut()` convenience method.

The `chapter.content` is just a string which happens to be markdown. While it's entirely possible to use regular expressions or do a manual find & replace, you'll probably want to process the input into something more computer-friendly. The `pulldown-cmark` crate implements a production-quality event-based Markdown parser, with the `pulldown-cmark-to-cmark` allowing you to translate events back into markdown text.

The following code block shows how to remove all emphasis from markdown, without accidentally breaking the document.

    #![allow(unused_variables)]
    fn main() {
    fn remove_emphasis(
        num_removed_items: &mut usize,
        chapter: &mut Chapter,
    ) -> Result<String> {
        let mut buf = String::with_capacity(chapter.content.len());

        let events = Parser::new(&chapter.content).filter(|e| {
            let should_keep = match *e {
                Event::Start(Tag::Emphasis)
                | Event::Start(Tag::Strong)
                | Event::End(Tag::Emphasis)
                | Event::End(Tag::Strong) => false,
                _ => true,
            };
            if !should_keep {
                *num_removed_items += 1;
            }
            should_keep
        });

        cmark(events, &mut buf, None).map(|_| buf).map_err(|err| {
            Error::from(format!("Markdown serialization failed: {}", err))
        })
    }
    }

For everything else, have a look at the complete example.