Skip to content

How to Write a New Sketch

Jump to bottom Edit New page
Ted Zlatanov edited this page Jun 3, 2014 · 1 revision

TODO: merge into howto/etch_a_sketch.md.

The following document will walk you through all of the information you will need to start writing your own Sketches.

Sketch Writing Tutorial: Message of the Day

In this document, we introduce Sketches using a very simple example - changing the Message of the Day on a Linux system. motd is a Unix command which is displayed after a successful login. It is a simple mechanism to communicate with the users that log into a machine, and it is configured by changing the contents of the /etc/motd file.

Sketch Structure

A good place to start is the structure of a Sketch. What files does a Sketch have to have to supply metadata and policy? Each sketch must be contained in its own directory, and must contain at least the three following files:

  • A JSON file that must be named sketch.json, which contains the description and metadata for the sketch.
  • A CFEngine policy file (.cf) that contains the actual CFEngine code to execute. The file can be named anything you want. We will describe its contents in detail below.
  • A README (plain text) or README.md (GitHub-Flavored Markdown format) file, containing human-readable documentation for the sketch.

Sketch Policy - policy.cf

The centerpiece of a sketch's functionality is a bundle. As an example we will use the following simple bundle, which adds one or more lines to the /etc/motd file using the insert_lines bundle from the CFEngine Standard Library.

Here's the Sketch policy file for this bundle:

bundle agent edit_motd(clear, lines)
{
  meta:
      "vars[clear][type]" string => "BOOLEAN";
      "vars[clear][default]" string => "true";
      "vars[lines][type]" string => "LIST()";
  vars:
    debug::
      "motd_file" string => "/tmp/motd",
        policy => "free";
    !debug::
      "motd_file" string => "/etc/motd",
        policy => "free";
  files:
      "$(motd_file)"
        create => "true",
        edit_defaults => cond_empty("$(clear)"),
        edit_line => insert_lines(@(edit_motd.lines));
}

body edit_defaults cond_empty(x)
{
        empty_file_before_editing => "$(x)";
}

What is this bundle doing? Let's walk through each section in detail:

Entry Point: "bundle agent edit_motd"

Every Sketch needs an entry point, and this is the entry point for the motd Sketch. If you are trying to dig into the details of a particular sketch, the best place to start is by scanning the Sketch policy bundle for the key words: "bundle agent". This policy takes two arguments "clear" and "lines".

Parameter Metadata "meta:"

The first step is to declare the types and default values of the parameters (if a parameter does not have a declared optional value, then it will be required when configuring the sketch). This is done by declaring an array metavariable (using meta: promises), indexed by parameter name, and containing two subelements:

  • type for the parameter type, and
  • optionally default to store its default value.

In our case, we want to declare clear as a "BOOLEAN"which defaults to true, and lines as an array (a LIST()) that has no default value.

Variable Definitions "vars:"

This vars section defines a variable used to refer to the /etc/motd file. As you can see in this example policy there are two classes that define two alternative locations:

  • debug - defines the location of the motd configuration file in /tmp
  • !debug - defines the location of the motd configuration file in /etc

Note that classes are a foundational concept in CFEngine and most policies are written to use define a different set of variables depending on the set of classes activated on a particular node. All decisions in CFEngine are made with classes, for more information see the CFEngine Reference.

File Definitions "files:"

This is the section that controls how the Message of the Day Sketch is going to populate the motd_config file. Here you can see that we're telling CFEngine to create the file if it doesn't exist, and we're telling CFEngine to use the @(edit_motd.lines) to populate this file.

"cond_empty"

This bundle tells CFEngine to empty the file before adding the contents of the line array to the motd_config file.

Defining sketch metadata

While the bundle is the essential building block for everything that happens in a CFEngine installation, it is the metadata that controls how a Sketch is used in the CFEngine Design Center. The metadata describes the Sketch's contents providing human-readable information such as the author and the license as well as an accounting of the contents of the Sketch - which files hold policy and which files provide documentation for the Sketch.

The sketch.json file contains the following:

{ 
    "manifest":
    {
        "main.cf": { "desc": "main file" },
        "README.md": { "documentation": true },
    },

    "metadata":
    {
        "name": "Misc::mysketch",
        "version": 3.14,
        "license": "MIT",
        "tags": [ "cfdc" ],
        "authors": [ "Diego Z", "Ted Z" ],
        "depends": { "CFEngine::stdlib": { "version": 105 }, "cfengine": { "version": "3.3.0" }, "os": [ "linux" ] },
    },

    "entry_point": "main.cf",
    "interface": [ "main.cf" ]
}

Let us look at the different fields defined in this file. For a full reference of all the fields allowed and required in a sketch.json file, see Sketch metadata specification.

The Manifest

First is the manifest. This is a mandatory element, and contains an array with one key for each file that is part of the sketch. Each value is a key-value array with keys like:

  • desc for description,
  • version for versioning individual files, and
  • documentation for indicating files that should be considered as documentation.

The Metadata

Next comes the metadata, also mandatory. This structure indicates the:

  • name of the sketch,
  • its version,
  • the tags,
  • the authors as a list, and
  • its dependencies in depends

In this example the tags are justcfdc which stands for CFEngine Design Center. If you are creating a collection of Sketches, you can define your own tags to use when searching for new Sketches. They are just strings.

The dependencies are a list of other Sketches and bundle libraries as well as a set of criteria. In this case we can see that this Sketch is only relevant to Linux systems with a minimum version of CFEngine being 3.3.0. The Sketch also depends on the CFEngine standard library being installed on the target CFEngine Agent.

Important about versioning: version fields are considered as strings and not as numbers. The only requirement is that versions should sort lexicographically, that is, "1.2.3 look out llamas" is before 2.3.4 which is before "7.8.9 beta of my soul".

The name of the sketch can be specified with namespaces, separated by double colons. Namespaces will be converted to directories when installing the sketch. For example, Misc::mysketch might be installed under /var/cfengine/masterfiles/sketches/Misc/mysketch/.

The dependencies can be the literal strings cfengine for the CFEngine version; os for the OS type (this is a regular expression that is matched against the output of the uname -s command); or any other sketch name with a specific version, if needed.

Sketch Entry Point

The entry_point indicates the file in which the main "entry bundle" for the sketch can be found, as well as its metadata specification (more on that in a moment). If the entry_point is set to null, then cf-sketch considers this sketch as a library, without the need to invoke a particular bundle during execution.

Sketch Interface

The interface indicates the files that need to be loaded for the sketch to function. In most cases it should be the same as entry_point, but if your sketch contains more than one .cf file that needs to be loaded, it should be specified here.

Defining entry point metadata

main.cf is your normal every day CFEngine configuration file, except that it has to contain two special bundles. Here is an example:

bundle agent mysketch_main_bundle(prefix)
{
  reports:
    cfengine_3::
      "myint = $($(prefix)myint); mystr = $($(prefix)mystr); os_special_path = $($(prefix)os_special_path); denied host = $($(prefix)hosts_deny)";
}

bundle agent meta_mysketch_main_bundle
{
  vars:
      "argument[mybool]"          string => "context"; # boolean
      "argument[myint]"           string => "string";
      "argument[mystr]"           string => "string";
      "optional_argument[myopt]"  string => "string";
      "argument[os_special_path]" string => "string";
      "argument[hosts_allow]"     string => "slist";

      "default[os_special_path]"  string => "/no/such/path";
      "default[hosts_allow]"      slist => { "a", "b", "c" };

      # we support array parameters too! 
      "argument[arrayparams]" string => "array";

      "optional_argument[extras]" string => "array";
      "default[extras][number1]" string => "123";
      "default[extras][number2]" string => "456";
}

The bundle whose name starts with meta_ establishes two things:

  • The name of the "entry bundle" for the sketch, which is defined to be the one with the same name except for the meta_ at the beginning. In this case, the existence of meta_mysketch_main_bundle indicates that mysketch_main_bundle is the main entry point for the sketch.
  • The metadata for the entry bundle: the types and names of its arguments, as well as their default values, if any:
    • The keys of the argument and optional_argument arrays are the parameter names, and their values are the type of each parameter. The types can be context (a boolean, which gets represented by a CFEngine class), string, slist or array.
    • The default array stores default parameter values, indexed by parameter name.

Writing the body of a sketch

The bundle to which the meta_ bundle points, as described above, is considered as the "entry bundle" of the sketch. To adhere to cf-sketch conventions, it must be a bundle of type agent, which receives a single string argument, as seen in this example:

bundle agent mysketch_main_bundle(prefix)
{
  reports:
    cfengine_3::
      "myint = $($(prefix)myint); mystr = $($(prefix)mystr); os_special_path = $($(prefix)os_special_path); denied host = $($(prefix)hosts_deny)";
}

All the parameters that the sketch receives must be referenced by prepending the string $(prefix) to the variable names. This makes it possible for cf-sketch to instantiate multiple instances of a single sketch at the same time, while keeping their parameters separate. Thus, any parameter defined in the sketch meta_ bundle needs to be always addressed like this:

$($(prefix)paramname)

Other than this, the bundle can have any structure you want. It may call other bundles or it may call libraries such as the CFEngine standard library. The sketch may contain more than one .cf file, as long as it is declared in the interface metadata element, so that cf-sketch arranges for it to be loaded before execution.

Sketch coding guidelines

Here are some suggestions for making sketches easy to use, reusable and play nicely with others.

  • The sketch directory must be inside a top-level category directory inside the sketches directory of design-center repository. Follow the list of suggested categories when possible. If something definitely does not fit in one of those categories, feel free to suggest a new one.
  • The name of the sketch, as specified in the metadata, can include arbitrary namespaces, separated by double colons. For example, the sketch stored under utilities/vcs_mirror/ is named VCS::vcs_mirror, and when installed, will be located under VCS/vcs_mirror in the installation target directory.
  • Restrict sketch functionality to a single topic/software/system, so that it is easy to explain what it does. If a sketch starts growing too much, it may be better to split it in two.
  • A sketch may have only one entry point that will be called by the cf-sketch-generated runfile (using the --generate option). Of course, a sketch can include other bundles, and they can be called manually from the CFEngine policy, but those calls will not be manageable through cf-sketch.
  • A sketch may have no entry points (define entry_point as null in the cfsketch.json file). In this case the sketch is considered as a "library sketch". It does not need to be activated, and can be loaded from other CFEngine policy files.
  • Avoid global classes and variables whenever possible, to avoid conflicts with other sketches. See "Naming conventions" below.
  • Make bundles configurable. See "How to make bundles configurable" below.

Naming conventions

To avoid conflicts among sketches and with other CFEngine policy files, we strongly suggest you follow these naming guidelines:

  • Avoid global classes and variables whenever possible, to avoid conflicts with other sketches. When unavoidable (for example, all classes set by the classes attribute to promises are global), prefix them with a unique string, preferably the $(prefix) string passed as argument to the entry point bundle. For example:

      "/tmp/somefile"
    create => "true",
    classes => if_repaired("$(prefix)file_created");

    This ensures that each class created in this way will be unique to each activated instance of a sketch.

    If a classname needs to be hardcoded, we suggest you prefix it with the string cfdc_ (for CFEngine Design Center), followed by the name of the sketch, and then an identifier. Keep in mind that in this case the class will be shared among all activated instances of the same sketch. For example:

      reports:
    cfdc_wordpress_tarfile_exists::
      "File already exists.";

  • All bundle names in CFEngine are global, so similar naming conventions should be applied. We recommend naming bundles with cfdc_<sketchname>_<bundlename>. If the sketch name and the bundlename are the same, then one of them can be omitted. For example:

      bundle agent cfdc_wordpress_wp_install
  
  bundle agent cfdc_vcs_mirror

How to make bundles configurable

Sketches in the Design Center should be as generic and reusable as possible. For this reason, it is good practice to make most (or all) information in the sketch configurable through parameters, so that users of your sketch can adapt it to their specific needs without having to modify the source files.

The preferred technique for this is to declare all configurable pieces of information as parameters in the meta_ bundle for the sketch. You can also store "private" (not meant for user modification) parameters in this way (our advice is to use a naming scheme for them, for example start their names with an underscore) - this gives the user the possibility, if so desired, to modify those internal parameters as well.

To provide default values that can be overridden by the user, you can do the following:

  • Declare the parameters in the optional_argument array of the meta_ bundle. For example:

      "optional_argument[_more_cmd]"  string => "string";

  • Declare its default value in the default array of the meta_ bundle:

      "default[_more_cmd]" string => "/bin/more";

  • Use the parameters as you would any other parameter:

      commands:
    "$($(prefix)_more_cmd) /some/file";

If the sketch has a large or unknown number of configurable parameters, you can use another technique to set default values. Declare an array parameter to the sketch, containing named parameters, that gets used throughout the sketch bundles. In this case, do the following:

  • Declare an array parameter, with the corresponding default values:

      "optional_argument[wp_params]"      string => "array";

  • Within the sketch, store all parameters with their default values in an internal array, setting them with policy => "free".

  • Copy values from the array passed by the user into the internal array.

  • Use the values from the internal array.

An example may make this technique clearer:

bundle agent wp_config(prefix)
{
  methods:
      # wp_vars receives the passed "params" array, and stores things
      # in an internal array.
      "wp_vars"  usebundle => wp_vars("$($(prefix)wp_params)");
      # All other bundles use "wp_vars.conf" instead of "params"
      "wp_cfgcp" usebundle => wp_config_exists("wp_vars.conf");
      "wp_cfg"   usebundle => wp_is_properly_configured("wp_vars.conf");
}
 
# This bundle merges the user-provided parameters (params) with the
# default values for each one, and leaves the result in the
# wp_vars.conf array, which is then used by the other bundles.
bundle agent wp_vars(params)
{
  vars:
      "wp_dir"             string => "$($(params)[_wp_dir])";
      # Default configuration values. Internal parameters start with _
      "conf[_tarfile]"      string => "/root/wordpress-latest.tar.gz",
                            policy => "overridable";
      "conf[_downloadurl]"  string => "http://wordpress.org/latest.tar.gz",
                            policy => "overridable";
      "conf[_wp_config]"    string => "$(wp_dir)/wp-config.php",
                            policy => "overridable";
      "conf[_wp_cfgsample]" string => "$(wp_dir)/wp-config-sample.php",
                            policy => "overridable";
    debian::
      "conf[_sys_servicecmd]" string => "/usr/sbin/service",
                              policy => "overridable";
      "conf[_sys_apachesrv]"  string => "apache2",
                              policy => "overridable";
    redhat::
      "conf[_sys_servicecmd]" string => "/sbin/service",
                              policy => "overridable";
      "conf[_sys_apachesrv]"  string => "httpd",
                              policy => "overridable";
    any::
      # Copy configuration parameters passed, into a local array
      "param_keys"          slist  => getindices("$(params)");
      "conf[$(param_keys)]" string => "$($(params)[$(param_keys)])",
                            policy => "overridable";
      ## At this point, wp_vars.conf contains all the default values,
      ## overwritten by any parameters passed in the "params" array.
}

Asking Questions

As a community of CFEngine users start to contribute to the CFEngine Design Center, you'll often find an answer from another Sketch author that has similar requirements. If you have any questions, feel free to ask in the CFEngine Help forum, in the IRC channel, or send an email to [email protected].

Contributing Sketches

Once you have your sketch ready, look at Contributing to the CFEngine Design Center for instructions on submitting it for inclusion in the repository. That document outlines the criteria used when evaluating the quality of submitted sketchs and selecting which Sketches to add to the CFEngine Design Center.

Add a custom footer

A public repository for customizable CFEngine design patterns and code.

Clone this wiki locally