CFEngine Documentation System

Note: This repository was merged into the generator sub-directory of the documentation repository on [2022-08-16 Tue]. It is archived and should not be used.

Writing Documentation

Documentation is written in Markdown. Our Markdown Cheatsheet shows useful formatting examples including usage of our custom extensions.

Site-wide Variables

For a given branch it can be useful to define variables that can be expanded anywhere in the documentation. Variables can be defined in =_config.yml= and then referenced in markdown. Jekyll uses the Liquid templating language to process templates.

This commit shows liquid variables in use.

Building Documentation

The documentation is rendered using jekyll. Jekyll is a ruby based static site generator and is the engine used by GitHub Pages.

Build Environment Setup

Note: The documentation build environment has very particular requirements for old libraries and environment.

Installation

This installation was tested on ubuntu 10.04 with vagrant, ruby 1.9.3 and Java 7. Make sure your machine has everything according to jekyll requirements: https://github.com/mojombo/jekyll.

git is required to checkout documentation.

You also need:

• java - for closure compiler
• python-pygments 1.5 or later for the code highlighting

Gems:

• jekyll-asset-pipeline
• closure-compiler
• yui-compressor
• redcarpet
• albino
• uglifier
• execjs
• sanitize

If pandoc is used:

• pandoc-ruby
• rdiscount

• =_setup/start.sh= is a shell script to install these dependencies. It needs to

be reviewed and adjusted to your environment accordingly. Clean setup

• Review steps described in file =_setup/step2.txt=
• Create pages folder and checkout CFEngine documentation inside.
• All content files should be inside =pages= folder.
• All files must have =meta= data on top. See (Page meta data section).
• Templates are saved in the =_layouts= folder.
• Images and other materials saved in media folder.
• JS and css saved in _assets folder. To include new files open =_includes/head.html= file.
• Make sure you set correct settings in _config.yml for CFE_OUTPUT and CFE_DIR

To build the project change into to the project root folder and run jekyll. Your can find the finished rendered files inside =pages= folder

Run local server and automatically build changes to pages: Change into the project root folder and run jekyll --auth --server 4000 where 4000 is the local port number to run the webserver on.

Note: you must restart server after changes in _config.yml.

To configure jekyll edit _config.yml. See the jekyll Configuration Wiki for a description of all available options.

CI Pipelines for Documentation

The documentation is built automatically by Jenkins. For details see the Poll SCM setting in each bootstrap-documentation-* job configuration.

You can trigger a documentation build for a branch by starting the appropriate bootstrap-documentation-* job manually.

Build Preparation

The official documentation uses multiple sources in its build.

• cfengine/buildscripts.git
• cfengine/core.git
• cfenigne/core.git
• cfengine/design-center.git
• cfengine/masterfiles.git
• cfengine/nova.git
• cfengine/enterprise.git
• cfengine/mission-portal.git
• cfengine/documentation.git
• cfengine/documentation-generator.git

After all repositories are updated and available the following is executed.

export WRKDIR=`pwd`
$WRKDIR/documentation-generator/_scripts/cfdoc_bootstrap.py master
cd $WRKDIR/core
NO_CONFIGURE=1 ./autogen.sh

Building

Documentation builds are performed by the build-documentation-* jobs. See Execute shell in one of the build-documentation-* jobs.

For example:

#!/bin/bash

set -x

export WRKDIR=`pwd`
cd $WRKDIR/core

./configure --with-lmdb=/usr/local --without-pam || exit 1

make || exit 2

cd $WRKDIR/documentation-generator

bash -x ./_regenerate_json.sh || exit 3

./_scripts/cfdoc_preprocess.py master || exit 4

bash -x ./_scripts/_run_jekyll.sh master || exit 5

Parts of the documentation rely on agent output (syntax description from cf-promises --syntax-description json) so compiling the agent is the first step in building the documentation.

After the agent has been built the syntax map is regenerated by the _regenerate_json.sh script.

Once the syntax map has been regenerated cfdoc_preprocess.py is executed. The cfdoc_preprocess.py script appears to be responsible for resolving metadata (whatever that means), resolving links, expanding our custom macros and creating printable (dead trees) versions of the documentation.

After pre-processing is complete _run_jekyll.sh performs the jekyll build, post processes the build with cfdoc_postprocess.py

Publishing

At the end of a successful documentation build the artifacts are published to a public server with the _publish.sh script. See Post build task for one of the build-documentation-* jobs.