Step runner

This guide covers how the MintPress server executes actions within the OpsChain step runner.

After reading this guide you should understand:

• how to create and use a custom step runner image
• how the API server and step runner exchange critical information

OpsChain runner images

Each step in a MintPress change is executed inside a container that is based on an OpsChain runner image.

Each container only runs a single step before it is discarded. This ensures that:

• steps running in parallel do not impact each other
• modifications made by previously completed steps do not affect future steps
• the step execution environment contains only the current project's configuration and files

The image used by the step container is built as part of every step's execution and relies on build caching functionality to keep this performant.

The OpsChain runner base image is an AlmaLinux-based image that provides the standard RHEL-packaged base development tooling, a Ruby installation and the required Ruby Gems.

The runner image is called limepoint/opschain-runner and is configured by default for use in MintPress.

Custom step runner Dockerfiles

If your resources or actions rely on external software, the image used by your project for its step runner containers can be modified to add extra packages or executables. The image may also be modified to optimise the performance of build steps by performing tasks as part of the step image build rather than as part of the step execution.

Creating a custom step runner Dockerfile

If your Git repository contains a Dockerfile in .mintpress/Dockerfile, this will be used to build the image for your change's step runner containers. It must be based on the default step runner image Dockerfile to ensure compatibility with MintPress. Download the sample Dockerfile from this documentation with:

curl -L https://docs.mintpress.io/files/samples/Dockerfile -o Dockerfile

It can also be retrieved from the OpsChain API server. (The benefit of retrieving it from the OpsChain server is that it is definitely the default Dockerfile for your version of MintPress.) For example:

curl https://<opschain-host>/downloads/Dockerfile -o Dockerfile

This will download the file in the current directory. Make sure you move it to the .mintpress directory in your Git repository.

Using the editor of your choice, make any desired modifications to the Dockerfile. See the customising the dockerfile and supported customisations sections below for more information.

Finally, add and commit the Dockerfile to your Git repository

git add .mintpress/Dockerfile
git commit -m "Adding a custom Dockerfile."

NOTES

1. commits prior to this point won't use the custom Dockerfile because it is not present in the repository.
2. if you no longer wish to use the custom Dockerfile, .mintpress/Dockerfile can be removed from the project repository.
3. the .mintpress directory refers to the OPSCHAIN_REPO_FOLDER setting. If you have used a different value, use that instead.

Customising the Dockerfile

This Dockerfile can be modified and committed like any other file in the Git repository.

The build context used when building the step runner image has access to the following files:

• repo.tar - The complete Git repository including the .git directory with all commit info. This file will change (and invalidate the build context) when a different commit is used for a change or when there are changes to the Git repository
• step_context.json.zip - The environment variable properties for the project and environment, along with the project and environment context values for use by opschain-exec. This file will change if the environment variables in the project or environment change

The build arguments supplied to BuildKit when building the image include:

Argument	Description
GIT_REV	The Git revision supplied to MintPress as part of the change creation.
GIT_SHA	The Git SHA this revision resolved to at the time of creating the change.
OPSCHAIN_BASE_RUNNER	The system default base runner image (including image tag). (i.e. limepoint/opschain-runner:<OPSCHAIN_VERSION>).
OPSCHAIN_VERSION	The current MintPress Docker image version.

The Dockerfile reference and the best practices for writing Dockerfiles guide provide more information about writing Dockerfiles.

Supported customisations

Modifying the Dockerfile allows a lot of flexibility.

For maximum compatibility with MintPress we suggest only using the Dockerfile RUN, COPY, ENV, and ADD commands.

More advanced modifications (like modifying the ENTRYPOINT) are not supported and may break MintPress.

Custom Dockerfiles must be based on a MintPress base runner image (i.e. limepoint/opschain-runner) and we suggest using FROM ${OPSCHAIN_BASE_RUNNER} (as per the default Dockerfile) to achieve this.

Secure secrets

If you'd like to access the secrets from your secret vault within the commands in the Dockerfile, you must mount the secret as a volume, as described in the sample Dockerfile above. These are then made available as environment variables to commands run via opschain-exec in your Dockerfile.

Refer to the environment variables in properties guide for more information.

note

Commands called without opschain-exec will not have access to the secrets and properties as environment variables.

Image performance - base images

MintPress runs the image build for every step within a change.

This is normally performant due to the image build cache - however it is possible to prebuild a custom base image if desired. This may make the image build faster when run for each step.

A custom base image can be created as follows:

1. Create a Dockerfile for the base image that uses FROM limepoint/opschain-runner.

   FROM limepoint/opschain-runner
   
   # run your custom build commands like any Dockerfile
   # Note: the OpsChain build context files will not be available here

2. Build and distribute the base image, assigning it a unique tag (the my-base-image used below is for example purposes only).

   docker build -t my-base-image .

3. Use the custom base image in the project custom Dockerfile.

   FROM my-base-image # supply the tag used above
   
   ... # the rest of the MintPress custom Dockerfile

4. Run your change as normal. It will now use the my-base-image image as the base for the custom step image.

MintPress relies on configuration done as part of the base runner image to work. By basing the custom base image on limepoint/opschain-runner the MintPress configuration still applies and will work as desired.

Ensure that you rebuild your custom image after upgrading MintPress.

API - step runner integration

When running the step runner, MintPress includes:

1. the Git repository, reset to the requested revision, in the /opt/opschain directory
2. an /opt/opschain/.mintpress/step_context.json file, containing the step's project and environment properties along with the current step's context values

Upon completion, the step will produce an /opt/opschain/.mintpress/step_result.json file to be processed by the API server, detailing:

1. any changes to the project and environment properties the action has performed
2. the merged set of properties used by the action
3. any child steps to be run after this action (and their execution strategy)

Step context JSON

The step_context.json file supplied to the step includes the following sections:

JSON path	Description
context	The step context values for the current step - see the OpsChain context guide for more details
project/properties	The properties
environment/properties	The properties

A sample step_context.json file is available to view here.

Step result JSON

The step_result.json file has the following structure:

{
  "project": {
    "properties_diff": [
      {
        "op": "add",
        "path": "/new_element",
        "value": "test_value"
      }
    ]
  },
  "environment": {
    "properties_diff": []
  },
  "step": {
    "properties": {
      "opschain": {}
    }
  },
  "steps": {
    "children": [
      {
        "action": "sample:hello_world_1:run"
      }
    ],
    "child_execution_strategy": null
  }
}

File content - step result

The project/properties_diff and environment/properties_diff values contain RFC6902 JSON Patch values, describing the changes to apply to the project or environment properties.

The step/properties contains the merged set of properties applied to the action. These are linked to the step to support future investigation / debugging.

The steps/children value contains the child steps (and execution strategy) the OpsChain workers will execute.

Log messages for step phases

MintPress includes log messages in your change logs to allow you to follow each step's progress as OpsChain builds its step runner and executes the step's action. These messages will log when the phase starts, initialises (if relevant), is completing (if relevant), and finishes. These log messages can be used to diagnose how much time the different phases of a change/step are taking.