Skip to main content

K3s Installation guide

This guide takes you through steps for installing self hosted K3s platform in order to host MintPress.

Linux Certification​

K3s for MintPress is tested and supported on RHEL/OEL/AlmaLinux 8.x and 9.x on a VM.

Access to software media and licence​

You must have acquired a license from LimePoint before you begin to install. If you don't have one, please reach out to your account manager or drop a note on support@limepoint.com. As part of license acquisition you would also get credentials to download the installers.

You must have access to the following URLs from your VM network either directly or via a proxy.

Infrastructure requirements​

The infrastructure requirements includes minumum configuration required for K3s along with requirements for MintPress. The requirements for MintPress are calculated based on the number of parallel executions you wish to do.

# of Parallel StepsMemory (GB)CPUStorage (GB)
1042100
10084100
1000328300

You can add more storage if you intend to save logs for longer within MintPress. We recommend externalizing logs to Splunk or a file system for better management.

Kernel version

To support native rootless image builds, you must have a Kernel version newer than 5.11. In systems older than 5.11, you can have rootless image builds by enabling the FUSE device plugin in the settings.

More details are provided later in the image build settings documentation.

Supported Platforms​

MintPress needs a container platform to run. It can be deployed on any of the following container platforms:

  • Azure Kubernetes Services (AKS)
  • Elastic Kubernetes Services (EKS)
  • OpenShift Container Platform
  • Self hosted Kubernetes (k8s or k3s) either on bare metal or a VM

The following setup guides will assume that you are installing MintPress on a self hosted K3s cluster.

Endpoint Security Software Compatibility​

MintPress is deployed on K3s with containerd as the container runtime. Endpoint security products running on the host including Antivirus (AV), Endpoint Protection Platform (EPP), Endpoint Detection and Response (EDR), and Extended Detection and Response (XDR) solutions are known to interfere with the correct operation and performance of K3s and containerd.

This is a prerequisite for installation. MintPress cannot function if the paths described below are subject to active filesystem scanning. Configuring the exclusions in this section is mandatory before proceeding.

Background​

K3s manages temporary mounts as part of normal container lifecycle operations. Active scanning of these mount points by endpoint security software causes failures in containerd that cannot be mitigated at the application level. This behaviour is documented in the upstream containerd issue tracker (containerd#5538).

Required Path Exclusions​

The following directories must be excluded from real-time or on-access scanning in your endpoint security product. Ensure there are no trailing spaces in the configured paths.

PathPurpose
/limepoint/k3s/MintPress K3s root
/limepoint/k3s/agent/containerd/containerd data directory
/var/lib/rancher/k3s/K3s runtime state

Example: Microsoft Defender for Endpoint (Linux):

sudo mdatp exclusion folder add --path "/limepoint/k3s/"
sudo mdatp exclusion folder add --path "/limepoint/k3s/agent/containerd/"
sudo mdatp exclusion folder add --path "/var/lib/rancher/k3s/"

The exact procedure for configuring exclusions will differ depending on your endpoint security product. Consult your vendor's documentation for the equivalent configuration steps.

Performance Baseline​

If MintPress is operational but exhibiting degraded performance, endpoint security software should be considered a potential contributing factor. To establish a valid performance baseline, temporarily disable your endpoint security product and retest. If performance improves, re-enable the product and work with your security team to tune the exclusion configuration.

Installation user​

Create a user named opschain on the Linux VM. The user need not be called opschain, it can be whatever name you want. For purpose of this guide, we will assume the linux user is opschain. If you decide to use any other username, please replace all occurrences of the opschain user in this guide to the name of your choice.

Root user

The following commands require root user access. Once the sudorules have been granted, you can switch to the installation user for further steps. Do not proceed if you haven't been able to provide sudo access to the user you created.

groupadd --gid 1001 opschain
useradd --uid 1001 --gid opschain --create-home opschain
UID & GID

If there are any existing users with the same UID or GID, please change them to unique values.

Sudo rules​

K3s needs full sudo access for installation and operations purposes. Modify the opschain user's sudo rules by running the following command:

vi /etc/sudoers.d/opschain

And pasting in the following line:

opschain ALL=(ALL) NOPASSWD:ALL

Once the user has been created and configured as a sudoer, switch to the opschain user by running su - opschain.

Installation user

The rest of the commands from here on should be run with the installation user you created.

Kernel & Ulimit settings​

Update the kernel and ulimit settings to ensure the system has enough resources to run MintPress.

tip

The opschain in the lines below refer to the installation user.

Update the limits by running the following command and pasting in the following lines:

vi /etc/security/limits.d/opschain.conf
root soft nofile 131072
root hard nofile 131072
opschain soft nofile 131072
opschain hard nofile 131072

Reboot the system to apply the changes:

reboot

Update the kernel settings by running the following command and pasting in the following lines:

vi /etc/sysctl.d/99-k3s-inotify.conf
# for k3s
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.inotify.max_queued_events=32768
fs.file-max = 2097152

# for fluentd
net.core.somaxconn = 1024
net.core.netdev_max_backlog = 5000
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_wmem = 4096 12582912 16777216
net.ipv4.tcp_rmem = 4096 12582912 16777216
net.ipv4.tcp_max_syn_backlog = 8096
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 10240 65535
net.ipv4.ip_local_reserved_ports = 24224

Installation & configuration​

Follow these steps to install K3s. All the steps below must be run by the installation user you created in the previous steps.

Setup proxy​

If you are using a proxy, please set the following variables on the shell before starting the next steps. This would be used only during the installation process. See Proxy Configuration later in this document for persisting proxy settings.

export http_proxy=<your_proxy_address>
export https_proxy=<your_proxy_address>

Directory setup​

All K3s configuration including data directory will be stored under this path. opschain here refers to the installation user.

sudo mkdir -p /limepoint/k3s /limepoint/rancher
sudo ln -s /limepoint/rancher /etc/rancher
sudo chown -R opschain:opschain /limepoint

Firewall setup​

If you are using the local Linux firewall, the following rules need to be added. First, check if the Linux firewall is running:

ps -ef|grep -i firewall

If you see a process, run the following commands, otherwise skip this step.

sudo firewall-cmd --permanent --add-port=6443/tcp # API server - # skip if no firewall is running
sudo firewall-cmd --permanent --add-port=30432/tcp # database replication - # skip if no firewall is running
sudo firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16 # pods - # skip if no firewall is running
sudo firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16 # services - # skip if no firewall is running
sudo firewall-cmd --reload # skip if no firewall is running

Download & install K3s​

K3s version

We recommend installing K3s version v1.35.3+k3s1 or later, but this requires that your operating system uses cgroups v2, given cgroups v1 is deprecated in Kubernetes versions beyond v1.35. If your operating system does not support cgroups v2, you can proceed with installing K3s version v1.34.6+k3s1 instead.

In most Linux distributions, you can check if your system uses cgroups v2 by running the following command:

stat -fc %T /sys/fs/cgroup/

If the output contains cgroup2fs, your system uses cgroups v2. If the command does not work, refer to your operating system's documentation for how to check if your system uses cgroups v2.

Learn more about cgroups.

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.35.3+k3s1" INSTALL_K3S_EXEC="--disable traefik --write-kubeconfig-mode 644 --data-dir /limepoint/k3s --secrets-encryption true" sh -

Validate K3s:

k3s --version

Download Helm​

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | DESIRED_VERSION="v3.20.2" bash

Validate Helm:

helm version

Setup shell​

Update your login shell to add the following aliases to make life simpler.

vi ~/.bash_profile
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
export KUBE_EDITOR=vim
alias crictl='sudo env "PATH=$PATH" crictl --config /limepoint/k3s/agent/etc/crictl.yaml'
alias ctr='sudo env "PATH=$PATH" ctr'

And then source the file for changes to take effect:

source ~/.bash_profile

Proxy Configuration​

MintPress requires outbound network access in two distinct contexts: during installation, to pull images from the MintPress container registry; and at runtime, when MintPress executes changes that reach external networks (this is optional). If the host VM does not have direct internet access, a proxy must be configured to serve both contexts.

Note: If your environment requires a more complex proxy topology (e.g. selective proxy bypass), refer to Advanced Proxy Setup before proceeding.

K3s Service Proxy Configuration​

Note: These steps are only required if you need proxy during runtime (from within the pod network). Otherwise, setting the proxy variables on the command line before running helm would suffice during the install time.

Once K3s is installed, proxy settings are configured via the K3s service environment file at /etc/systemd/system/k3s.service.env. These settings govern both host-level traffic during installation and pod-level traffic at runtime.

The example below assumes the proxy is reachable at proxy-host.com:4125 and the host subnet is 10.100.22.0/24. Adjust these values to match your environment.

# /etc/systemd/system/k3s.service.env

HTTPS_PROXY="http://proxy-host:4125"
HTTP_PROXY="http://proxy-host:4125"
https_proxy="http://proxy-host:4125"
http_proxy="http://proxy-host:4125"

NO_PROXY="127.0.0.1,localhost,10.42.0.0/16,10.43.0.0/16,10.100.22.0/24,kubernetes,kubernetes.default,.svc,.cluster.local"
no_proxy="127.0.0.1,localhost,10.42.0.0/16,10.43.0.0/16,10.100.22.0/24,kubernetes,kubernetes.default,.svc,.cluster.local"

Both the uppercase and lowercase variants of each variable must be set, as different tools and runtimes observe different conventions.

The NO_PROXY / no_proxy values are required and must not be omitted. They ensure that traffic to internal cluster addresses including the pod CIDR (10.42.0.0/16), service CIDR (10.43.0.0/16), host subnet, and Kubernetes DNS names bypasses the proxy entirely.

Applying the Configuration​

After saving the file, reload the systemd daemon and restart K3s:

sudo systemctl daemon-reload
sudo systemctl restart k3s

What to do next​