Upgrade Kommander
This section describes how to upgrade your Kommander Management cluster and all Platform Applications to their supported versions in air-gapped, non-air-gapped and on-prem environments. To prevent compatibility issues, you must first upgrade Kommander on your Management Cluster before upgrading to DKP.
It is important you upgrade Kommander BEFORE upgrading the Kubernetes version (or Konvoy version for Managed Konvoy clusters) in attached clusters. This ensures that any changes required for new or changed Kubernetes API’s are already present.
Page Contents
Prerequisites
REQUIRED Before upgrading, create an on-demand backup of your current configuration with Velero.
Download and install the supported DKP CLI binary of this release on your computer.
Ensure you are attempting to run a supported DKP upgrade.
Ensure you are on Kubernetes version 1.23 or higher.
If you have attached clusters, ensure they are on Kubernetes versions 1.22 or higher.
Review the DKP 2.4.0 Components and Applications versions that are part of this upgrade.
For air-gapped environments with DKP Catalog Applications in a multi-cluster environment: Load the Docker images into your Docker registry
For air-gapped environments without DKP Catalog Applications: Load the Docker images into your Docker registry
DKP Upgrade with Insights Installed
If your environment has Insights installed, you must uninstall Insights BEFORE upgrading DKP.
Prerequisites for Configurations with NVIDIA GPU Nodes
If your cluster is running GPU nodes and has deployed the nvidia
Platform Application, you must run additional steps prior to upgrading Kommander. In DKP 2.4, the nvidia
application is replaced by nvidia-gpu-operator
during upgrade, and the configuration for the new application must be created prior to the upgrade to ensure that the new application is deployed with the proper configuration, or it may not deploy successfully. If your cluster is not running GPU nodes, then skip these steps and proceed directly to the Upgrade Kommander section.
The GPU nodes will experience downtime during the upgrade due to the new version of the Nvidia application.
Create the ConfigMap with the necessary configuration overrides to set the correct Toolkit version. For example, if you’re using Centos 7.9 or RHEL 7.9 as the base operating system for your GPU enabled nodes, set the
toolkit.version
parameter:cat <<EOF | kubectl apply -f - apiVersion: v1 kind: ConfigMap metadata: namespace: kommander name: nvidia-gpu-operator-overrides data: values.yaml: | toolkit: version: v1.10.0-centos7 EOF
CODECreate an
AppDeployment
namednvidia-gpu-operator
in thekommander
namespace:cat <<EOF | kubectl apply -f - apiVersion: apps.kommander.d2iq.io/v1alpha3 kind: AppDeployment metadata: name: nvidia-gpu-operator namespace: kommander spec: appRef: name: nvidia-gpu-operator-1.11.1 kind: ClusterApp configOverrides: name: nvidia-gpu-operator-overrides EOF
CODEYou may now proceed to the Kommander upgrade steps.
Upgrade Kommander
Before running the following command, ensure that your dkp
configuration references the Kommander Management cluster, otherwise it attempts to run the upgrade on the bootstrap cluster. You can do this by setting the KUBECONFIG
environment variable to the appropriate kubeconfig file’s location.
In this version of DKP, pre-provisioned environments require a special upgrade path. Refer to Upgrade Kommander in a Pre-provisioned Environment for more information.
If you have configured a custom domain, running the upgrade command
could result in an inaccessibility of your services via your custom domain for a few minutes.
As stated earlier, an alternative to initializing the KUBECONFIG environment variable is to use the –kubeconfig=cluster_name.conf
flag. This ensures that Kommander upgrades on the workload cluster.
Use the DKP CLI to upgrade Kommander and all the Platform Applications in the Management Cluster:
For air-gapped:
dkp upgrade kommander --charts-bundle ./application-charts/dkp-kommander-charts-bundle-v2.4.0.tar.gz --kommander-applications-repository ./application-repositories/kommander-applications-v2.4.0.tar.gz
CODEFor air-gapped with DKP Catalog Applications in a multi-cluster environment:
dkp upgrade kommander --charts-bundle ./application-charts/dkp-kommander-charts-bundle-v2.4.0.tar.gz --charts-bundle ./application-charts/dkp-catalog-applications-charts-bundle-v2.4.0.tar.gz --kommander-applications-repository ./application-repositories/kommander-applications-v2.4.0.tar.gz
CODEAfter the upgrade, follow the DKP Catalog Applications configuration page to update the Git repository.
For non-air-gapped:
dkp upgrade kommander
CODEIf you have DKP Catalog Applications deployed, follow the DKP Catalog Applications configuration page to update the Git repository after the upgrade.
The output looks similar to this:✓ Ensuring upgrading conditions are met ✓ Ensuring application definitions are updated ✓ Ensuring helm-mirror implementation is migrated to chartmuseum ...
CODE
If the upgrade fails, run the following command to get more information on the upgrade process:
dkp upgrade kommander -v 6
CODEIf you find any
HelmReleases
in a “broken” release state such as “exhausted” or “another rollback/release in progress”, you can trigger a reconciliation of theHelmRelease
using the following commands:kubectl -n kommander patch helmrelease <HELMRELEASE_NAME> --type='json' -p='[{"op": "replace", "path": "/spec/suspend", "value": true}]' kubectl -n kommander patch helmrelease <HELMRELEASE_NAME> --type='json' -p='[{"op": "replace", "path": "/spec/suspend", "value": false}]'
CODEFor Enterprise customers (multi-cluster environment): Upgrade your additional Workspaces on a per-Workspace basis to upgrade the Platform Applications on other clusters than the Management Cluster.
For Essential customers (single-cluster environment): Proceed with the Konvoy Upgrade.
You can always go back to the DKP Upgrade overview, to review the next steps depending on your environment and license type.