Edit This Page Clicking 'Edit This Page' will open a new tab or window to edit this page on the Git forge. You'll need to be signed into the forge to edit. Close the edit page to return to this one.

2023-01-20 MWB Program flow

This note outlines the steps taken to build a Massive Wiki static website. The code outlined is mwb.py, version 2.1.0.

The mwb.py main function steps:

  • initialize: parse command line arguments

  • store configuration

  • get Jinja2 env

  • set up lunr filename and sitepath indices

  • render the wiki:

    • remove and recreate output directory

    • generate dict of filenames and wikipaths to support wikilinks (first loop through all files)

    • copy wiki to output; render .md files as HTML

      • second loop through files
        • parse Markdown files (frontmatter and markdown body)
        • collect git commit messages and times
        • append to all_pages list
    • build lunr search index (if specified)

    • set up search web page

    • build all_pages web page

cf. MWB-dataflowV3.png

Pete's notes, 2023-02-17

Paths for all non-hidden Markdown files

  • absolute file path of source file
    • /Users/peterkaminski/Documents/GitHub/developer-massive-wiki/Wiki Theory and Culture/Link As You Think.md
  • absolute file path or directory for output of .md, .html, .json
    • /Users/peterkaminski/Documents/GitHub/developer-massive-wiki/.massivewikibuilder/output/Wiki_Theory_and_Culture/Link_As_You_Think.md
    • /Users/peterkaminski/Documents/GitHub/developer-massive-wiki/.massivewikibuilder/output/Wiki_Theory_and_Culture/Link_As_You_Think.html
    • /Users/peterkaminski/Documents/GitHub/developer-massive-wiki/.massivewikibuilder/output/Wiki_Theory_and_Culture/Link_As_You_Think.json
  • absolute web path (with optional rootdir) (with non-safe r'([ _?\#]+)' characters replaced with _)
    • /rootdir/Wiki_Theory_and_Culture/Link_As_You_Think.html


All non-hidden files of any type:

  • are copied from wiki to output

All non-hidden Markdown files:

  • name of file is used as a key in the list of wikilink targets
  • are entered into the search index
  • file is parsed and
    • a .html version is saved to the location that will be next to the .md file
    • a .json extract of the YAML frontmatter is saved to the location that will be next to the .md file


If we accumulate all links seen during Markdown parsing, after all Markdown files are parsed, we can compare that to the list of wikilink targets, and determine:

  • unlinked files (could write an HTML report file)
  • links to non-existent files (each of which could be created as a stub HTML file)

These lists could also be output as log messages.

MassiveWikiBuilder V3-pseudocode

Pages that link to this page