Skip to main content

Patching guide

This guide takes you through patching MintPress to the latest release.

caution

Before patching MintPress, make sure you check the Release Notes for the relevant version - making note of any breaking changes and related pre-patch steps.

Prerequisites​

To patch MintPress, we must refer to the values.yaml file used for installation. The patching process should be performed in the same server as the one used for installation.

If you're patching a high availability topology, ensure you follow the patch sequence instructions.

Validate Helm and K3s are installed​

Before you proceed, please ensure Helm and K3s are installed and configured. You can quickly verify this by running the following commands:

helm version
k3s --version

If you see the version numbers, you are good to go. If you don't, please refer to the Installing K3s guide for more information.

Validate Helm registry access​

With the credentials you received as part of your licence, ensure Helm is connected to DockerHub's registry.

helm registry login docker.io

The command will prompt you for your DockerHub credentials.

MintPress chart version​

Identify the version of MintPress you want to patch to from the Release Notes page and set it to your bash profile with the Helm chart version format, for example:

vi ~/.bash_profile

And then add or modify the following line with the version you want to upgrade to:

export MINTPRESS_CHART_VERSION=2025.11.12

Then source it for the changes to take effect:

source ~/.bash_profile

Update the values.yaml file​

You can obtain the new values.yaml file in the sample values.yaml files section of this documentation. You can download it with:

curl -L https://docs.mintpress.io/files/config_file_examples/values.yaml.example -o /limepoint/values_new.yaml

Some MintPress updates might change the settings in the values.yaml file. To ensure you don't lose any of your custom settings, merge the new values_new.yaml file with the one you used for installation (or previous patches), ensuring the new settings are applied and your custom settings are preserved. Save the merged file as values.yaml so that it can be used in the next step.

Refer to the release notes for the version you're patching to for any new settings that need to be applied or breaking changes. If you are unsure, refer to the configuration guide for more information.

To see the settings the release ships with, download the full values.yaml for this version of the documentation and compare it against your own file.

MintPress version

Ensure all the settings that have an image tag match the MintPress version you're patching to, otherwise your installation will be running with outdated images.

Patch MintPress​

If upgrading from 2026-07-09

Ensure theopschain-api-worker deployment is scaled to 0 prior to running the helm upgrade command. This will avoid error events being generated in the audit history during the upgrade process. You can scale the deployment down with the following command:

kubectl scale -n <opschain namespace> deploy/opschain-api-worker --replicas=0

With the environment variables configured and the values.yaml file updated, patch MintPress by running the following command:

helm upgrade --install mintpress "oci://docker.io/limepoint/mintpress" --version ${MINTPRESS_CHART_VERSION} --create-namespace -n ${KUBERNETES_NAMESPACE} -f /limepoint/values.yaml --wait --timeout 30m --insecure-skip-tls-verify --debug
Patching time

The command may take several minutes (up to 30 mins) to start as the MintPress images are downloaded, especially with slower internet connections.

Do not close or end the shell session while the patching is still ongoing, doing so might render your installation unusable.

Just like with installation, this command will start the MintPress server and its dependent services in separate Kubernetes pods. For more information on these containers see the architecture overview.

The kubectl command can be used to see the patching progress:

kubectl get deployments.apps/opschain-api -n ${KUBERNETES_NAMESPACE}

The patching will be complete when the READY column of the opschain-api deployment is set to 1/1. You can verify the current MintPress version in the version info section of the GUI.

Patching a high availability topology​

The steps above patch a single cluster. In a high availability topology several clusters share one replicated database, and every MintPress instance — including those running in replica clusters — connects to the primary cluster's database. Each MintPress API applies any outstanding database migrations automatically as it starts, against that shared database.

Two consequences shape how you patch:

  • The schema is migrated once, by the first patched API to start. Every other running instance then uses that same, already-migrated schema.
  • Running two different MintPress versions against the shared database at the same time is unsafe — an older instance can encounter a schema it does not expect. Patch the primary first, and keep the window in which clusters run different versions as short as possible.
Patch the whole topology in one maintenance window

Treat the topology's MintPress instances as a single application. Do not leave some clusters on the old version and others on the new version for longer than the patching itself takes.

Before you patch a topology​

  • The release notes breaking changes and any Before upgrading steps apply to the whole topology — carry them out before you start.
  • Move every cluster's values.yaml to the same new version together, keeping the encryption keys, database credentials and image registry password identical across clusters, as high availability already requires.
  • Do not failover during patching.
  • If the release changes the PostgreSQL major version, streaming replication cannot run between clusters on different majors — plan this separately and follow the release notes' guidance. See db.cnpg.imageName.

Patch sequence​

  1. Update the values.yaml on every cluster to the new version, as described above.

  2. Reduce the running application to only the primary cluster. Scale the API and workers to 0 in every replica cluster and, if you run MintPress instances (API, workers) in replica clusters, put those clusters into stopped mode (stopped: true) so only the primary cluster runs the application while the database is migrated. Their databases keep replicating.

  3. Patch the primary cluster with the patch command. Its API applies the database migration against the shared database. Wait until opschain-api reports 1/1.

  4. Patch each replica cluster to the same version with the same command. If you run MintPress instances there, bring them back by removing stopped: true. Because the schema is already migrated, the new code matches it.

  5. If you use the OpsChain secret vault, roll its pods in every cluster so they run the new image and reconnect to the primary database — delete them and let the StatefulSet recreate them:

    kubectl delete pod -l app.kubernetes.io/name=openbao -n ${KUBERNETES_NAMESPACE}

  6. Update the runner image version in each cluster if it hasn't rolled automatically.

  7. Verify every database cluster is healthy and replicating (kubectl get cluster <cluster-name> -n ${KUBERNETES_NAMESPACE}), confirm the version info, and make a small write — create a change or edit a project's properties — to confirm the primary is serving.

Minimising database downtime

Patching the primary cluster might restart its database instance in some scenarios, which briefly interrupts the database. If you run more than one database instance in the primary cluster (via db.cnpg.primary.replicas), set db.cnpg.primaryUpdateMethod to switchover to reduce that interruption.

If opschain-api does not become ready and the patching times out while you are using the OpsChain secret vault, see helm upgrade fails with context deadline exceeded.

Update the runner image version​

After a successful patch, you may want to update your runner image version to the same version as MintPress. This can be done via system configuration, modifying the Resolved image tag to the version you're patching to.

Old images in the MintPress registry​

MintPress will not automatically remove old images in the internal image registry during the patching process. This means that old runner images may still exist in the registry, using up disk space. If you need to remove these old images, refer to the container image cleanup guide.

Troubleshooting​

If the Helm commands take too long to complete, you can check the status of the deployments with the following command:

kubectl get deployments.apps -n ${KUBERNETES_NAMESPACE}

And verify that all deployments have the READY column set to a matching value, for example:

NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/opschain-api          1/1     1            1           10m
deployment.apps/opschain-api-worker   2/2     2            2           10m
...

To check the status of the pods, you can run the following command:

kubectl get pods -n ${KUBERNETES_NAMESPACE}

Common issues​

Build service pod stuck in Pending state​

If the buildService.rootless setting is set to true in your values.yaml file, it might occur that the build service pod gets stuck in a Pending state. This is usually related to the fuse-device-plugin-daemonset not running or requiring a restart. This pod is responsible for making the required FUSE devices on the node, available to the opschain-build-service pod when running in rootless mode.

If this is the case, the opschain-build-service pod will show events similar to the following:

Warning  FailedScheduling  22m                  default-scheduler  0/1 nodes are available: 1 Insufficient github.com/fuse. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.
Warning  FailedScheduling  6m23s (x7 over 22m)  default-scheduler  0/1 nodes are available: 1 Insufficient github.com/fuse. preemption: 0/1 nodes are available: 1 No preemption victims found for incoming pod.
Solution - restart the fuse-device-plugin-daemonset daemonset​

To restart the fuse-device-plugin-daemonset daemonset, run the following command:

kubectl rollout restart daemonset/fuse-device-plugin-daemonset -n ${KUBERNETES_NAMESPACE}

This will restart the daemonset and make the FUSE devices available to the opschain-build-service pod.