Stats

Usage: lcd-stats

• Writes collected stats to a file or stdout after build, intended for consolidation e.g. with jq
• Presents deviations

• Runs a post build simple consolidation of major log events, presents them and triggers a build break at presence of log with severity higher than a configurable value

  Important

  You have to enable this plugin, when you want LP build breaks on CI based on lcd plugin log errors or higher severity.

Mechanics

• All hooks of plugins inheriting from lcdoc.mkdocs.MDPlugin are wrapped into a decorator, which
  • sets a named stats dict into them, so they can just set counters and metrics into those dicts.
  • customizes their logging
• When the hooks are page hooks we set the stats dicts per page.

Config

Code

C = config_options.Choice
log_maj = lambda d, C=C: C(['warning', 'error', 'fatal', 'none'], default=d)
config_scheme = (
    # if not starting with "/": relative to project root.
    # for stdout: set file="-"
    ('dump_stats', config_options.Type(str, default='build/lcd-stats.json')),
    # round floats to this precision:
    ('round_digits', config_options.Type(int, default=2)),
    # omit zero values:
    ('filter_0', config_options.Type(bool, default=True)),
    # helpfull to see changes at serve
    ('print_diff', config_options.Type(bool, default=True)),
    # write the logs as json (same dir than fn)
    ('dump_logs', config_options.Type(str, default='build/lcd-logs.json')),
    # print all logs from this level again at end of build:
    ('repeat_major_log_events', log_maj('warning')),
    # fail mkdocs build on errors, you don't want broken docs published:
    ('fail_build_on_log_events', log_maj('error')),
)

https://github.com/AXGKl/docutools/blob/master/src/lcdoc/mkdocs/stats/__init__.py#L100

Stats Output

On config setting dump_stats, we will dump all stats to the configured file

Example Output

$ cat $LP_PROJECT_ROOT/build/lcd-stats.json| head -n 20 || true # on CI the first run will have no such file
{
    "Filtered_0_Values": 484,
    "Global.LPPlugin.on_config.LP_env_vars": 5,
    "Global.MDFindPagesPlugin.on_config.find_terms": 1,
    "Global.MDFindPagesPlugin.on_config.matching": 37,
    "Log.debug": 533,
    "Log.info": 92,
    "Log.warning": 2,
    "Pages.LPPlugin.on_page_markdown.Overview.blocks_evaled": 4,
    "Pages.LPPlugin.on_page_markdown.Overview.blocks_max_time": 0.45,
    "Pages.LPPlugin.on_page_markdown.Overview.blocks_total": 4,
    "Pages.LPPlugin.on_page_markdown.Overview.dt": 0.52,
    "Pages.LPPlugin.on_page_markdown.about/changelog/Changelog.blocks_skipped_prev_result": 1,
    "Pages.LPPlugin.on_page_markdown.about/changelog/Changelog.blocks_total": 1,
    "Pages.LPPlugin.on_page_markdown.about/coverage/Coverage.blocks_evaled": 2,
    "Pages.LPPlugin.on_page_markdown.about/coverage/Coverage.blocks_total": 2,
    "Pages.LPPlugin.on_page_markdown.about/credits/Credits.blocks_evaled": 1,
    "Pages.LPPlugin.on_page_markdown.about/credits/Credits.blocks_max_time": 1.85,
    "Pages.LPPlugin.on_page_markdown.about/credits/Credits.blocks_total": 1,
    "Pages.LPPlugin.on_page_markdown.about/credits/Credits.dt": 1.86,

Diff Output

We keep the stats from the last run and compare at every build, logging the diff.

Example: Post build output diff after adding an LP block

INFO     -  Stats changes since last run   [StatsPlugin]
{
    "added": {
        "Pages.LPPlugin.on_page_markdown.features/mypage/My Page.blocks_evaled": 1,
        "Pages.LPPlugin.on_page_markdown.features/mypage/My Page.blocks_total": 1
    },
    "changed": {
        "Log.debug": [
            144,
            145
        ],
        "Log.info": [
            8,
            9
        ]
    }
}

Logging

$ignore_err

• mkdocs build will fail at error levels including and above error.
• You can lower error logs by setting $ignore_err, matching the log message to be lowered from error (or higher) to just warning.

Example: In ci.yml:

env:
    ignore_err: "No coverage files"

Log Dumps

On config setting dump_logs, we will dump all created logs in line-sep json form to the given file (backing up the previous one)

Tip

The output of debug level logging is often overwhelming and inter build changes are hard to see, but you'll see a change of log statement counts in the diff.

If you are interested in why log counts changed you can diff the previous logs output with the current one.

Warning

Some "expensive" debug logs are not even sent to the logging system, when level is info or higher. Those logs cannot occur in the log dumps then. In order to get really all possible debug logs, you have to run mkdocs with -v.

Example Output

$ cat $LP_PROJECT_ROOT/build/lcd-logs.json| head -n 20 || true # on CI the first run will have no such file
[0, 20, "lcd", "Ran reset, cleared stats", {"reset": true, "utc": "20211004T210256Z", "unix": 1633381376}]
[185, 10, "Blacklist", "BlacklistPlugin.on_config"]
[185, 10, "Blacklist", "No $blacklisted_words to check for blacklisted words"]
[186, 10, "LP", "LPPlugin.on_config"]
[202, 20, "LP", "Plugs doc symlink created"]
[210, 30, "LP", "Linking", {"frm": "/home/runner/work/docutools/docutools/src/lcdoc/mkdocs/lp/assets", "to": "/home/runner/work/docutools/docutools/docs/lcd/lp"}]
[212, 10, "LP", "Added assets", {"typ": "css", "count": 1, "dir": "/home/runner/work/docutools/docutools/docs/lcd/lp/css"}]
[212, 10, "LP", "Added assets", {"typ": "javascript", "count": 8, "dir": "/home/runner/work/docutools/docutools/docs/lcd/lp/javascript"}]
[212, 20, "LP", "Assets for git changelog linked", {"frm": "/home/runner/work/docutools/docutools/src/lcdoc/mkdocs/lp/assets", "to": "/home/runner/work/docutools/docutools/docs/lcd/lp"}]
[213, 10, "LP", "From environ", {"key": "eval", "value": "always"}]
[213, 10, "LP", "From environ", {"key": "DOCU_FILE", "value": "init"}]
[213, 10, "LP", "From environ", {"key": "DOCU_DIR", "value": ""}]
[213, 10, "LP", "From environ", {"key": "DOCU_ROOT", "value": "/home/runner/work/docutools/docutools/docs"}]
[213, 10, "LP", "From environ", {"key": "PROJECT_ROOT", "value": "/home/runner/work/docutools/docutools"}]
[213, 10, "MDFindPages", "MDFindPagesPlugin.on_config"]
[233, 20, "MDFindPages", "Inserting 22 pages into nav"]
[241, 10, "PageTree", "PageTreePlugin.on_config"]
[242, 20, "PageTree", "footer.html alrady replaced"]
[286, 10, "MDReplace", "MDReplacePlugin.on_pre_build"]
[286, 20, "MDReplace", "Loading replacement file", {"fn": "/home/runner/work/docutools/docutools/docs/mdreplace.py"}]