Skip to main content
Skip table of contents

Enterprise: Upgrade Workspace DKP Catalog Applications

Upgrade catalog applications using the CLI and UI

These pages refer to DKP Enterprise and DKP Gov Advanced products.

Considerations for Upgrading DKP Catalog Applications with Spark Operator Post DKP 2.7

Starting in DKP 2.7, the Spark operator app is removed, resulting in the complete uninstallation of the app after upgrading the DKP Catalog Applications GitRepository.
This section provides instructions on how you can continue using Spark in DKP 2.7 and after.

If you do not plan on using Spark or if you’re content with having Spark be uninstalled automatically, you can skip this section and proceed to the Update the DKP Catalog Applications GitRepository section.

How to orphan existing instances of Spark so that it is unmanaged by DKP or Flux
  1. Set the WORKSPACE_NAMESPACE variable to the namespace where the spark-operator is deployed:

    CODE
    export WORKSPACE_NAMESPACE=<WORKSPACE_NAMESPACE>
  2. Set suspend: true on the cluster Kustomization
    NOTE: This needs to be reverted in the last step.

    CODE
    kubectl -n ${WORKSPACE_NAMESPACE} patch kustomization cluster --type='json' -p='[{"op": "replace", "path": "/spec/suspend", "value": true}]'
  3. Set suspend: true and prune: false on the spark-operator Kustomizations

    CODE
    export SPARK_OPERATOR_APPDEPLOYMENT_NAME=<spark operator AppDeployment name>
    export SPARK_OPERATOR_APPDEPLOYMENT_VERSION=<spark operator AppDeployment version>
    kubectl -n ${WORKSPACE_NAMESPACE} patch kustomization ${SPARK_OPERATOR_APPDEPLOYMENT_NAME} --type='json' -p='[{"op": "replace", "path": "/spec/suspend", "value": true},{"op": "replace", "path": "/spec/prune", "value": false}]'
    kubectl -n ${WORKSPACE_NAMESPACE} patch kustomization spark-operator-${SPARK_OPERATOR_APPDEPLOYMENT_VERSION}-defaults --type='json' -p='[{"op": "replace", "path": "/spec/suspend", "value": true},{"op": "replace", "path": "/spec/prune", "value": false}]'
  4. Update the dkp-catalog-applications GitRepository with the new version

  5. Unsuspend the cluster Kustomization
    NOTE: If the Kustomization is left suspended, Kommander will be unable to function properly.

    CODE
    kubectl -n ${WORKSPACE_NAMESPACE} patch kustomization cluster --type='json' -p='[{"op": "replace", "path": "/spec/suspend", "value": false}]'

Remove Spark from the UI

Follow these steps to remove the spark-operator AppDeployment from the DKP UI:

  1. From the top menu bar, select your target workspace.

  2. Select Applications from the sidebar menu.

  3. Select the three dot button from the bottom-right corner of the Spark application tile, and then select Uninstall.

  4. Select Save.

OR

Remove Spark with CLI

  1. Execute the following command to get the WORKSPACE_NAMESPACE of your workspace:

    CODE
    dkp get workspaces

    And copy the values under the NAMESPACE column for your workspace.

  2. Export the WORKSPACE_NAMESPACE variable:

    CODE
    export WORKSPACE_NAMESPACE=<WORKSPACE_NAME>
  3. Run the following command to delete the spark-operatorAppDeployment:

    CODE
    kubectl delete AppDeployment <spark operator appdeployment name> -n ${WORKSPACE_NAMESPACE}

This results in the spark-operator Kustomizations being deleted, but the HelmRelease and default ConfigMap will remain cluster.

From here, you can continue to manage the spark-operator via the HelmRelease.

Update the DKP Catalog Applications GitRepository

Follow the instructions in this section to update the GitRepository dkp-catalog-applications.

For air-gapped environments

Update the GitRepository (for air-gapped environments)

Update the GitRepository for dkp-catalog-applications.

For the following section, ensure you modify the most recent kommander.yaml configuration file. It must be the file that reflects the current state of your environment. Reinstalling Kommander with an outdated kommander.yaml overwrites the list of platform applications that are currently running in your cluster.

In the kommander.yaml you are currently using for your environment, update the DKP Catalog Applications by setting the correct DKP version:

CODE
...
# The list of enabled/disabled apps here should reflect the current state of the environment, including configuration overrides!
...
catalog:
  repositories:
    - name: dkp-catalog-applications
      labels:
        kommander.d2iq.io/project-default-catalog-repository: "true"
        kommander.d2iq.io/workspace-default-catalog-repository: "true"
        kommander.d2iq.io/gitapps-gitrepository-type: "dkp"
      path: ./dkp-catalog-applications-v2.7.1.tar.gz # modify this version to match the DKP upgrade version
...

Refresh the kommander.yaml to apply the updated tarball:

Ensure the kommander.yaml is the Kommander Installer Configuration file you are currently using for your environment. Otherwise, your configuration will be overwritten and previous configuration lost.

CODE
dkp install kommander --installer-config kommander.yaml

For a final cleanup of Spark, delete its AppDeployment from all workspaces where it was deployed:

  1. Execute the following command to get the WORKSPACE_NAMESPACE of your workspace:

    CODE
    dkp get workspaces

    And copy the values under the NAMESPACE column for your workspace.

  2. Export the WORKSPACE_NAMESPACE variable:

    CODE
    export WORKSPACE_NAMESPACE=<WORKSPACE_NAMESPACE>
  3. Run the following command to delete the spark-operatorAppDeployment:

    CODE
    kubectl delete AppDeployment <spark operator appdeployment name> -n ${WORKSPACE_NAMESPACE}
For non-air-gapped environments

Update the GitRepository (for non-air-gapped environments)

Upgrade the workspace Catalog Applications in kommander on each additional workspace on which catalog apps are deployed. In air-gapped scenarios, the images are included in the air-gapped bundle. In non-air-gapped scenarios, refresh the Git Repository as shown below.

Update the GitRepository with the tag of your updated DKP version on the kommander workspace:

CODE
kubectl patch gitrepository -n kommander dkp-catalog-applications --type merge --patch '{"spec":{"ref":{"tag":"v2.7.1"}}}'

This command updates the catalog application repositories for all workspaces.

For any additional Catalog GitRepository resources created outside the kommander.yaml configuration (e.g. such as in a project or workspace namespace), follow these steps:
Set the WORKSPACE_NAMESPACE environment variable to the namespace of the workspace:

CODE
export WORKSPACE_NAMESPACE=<workspace namespace>

Update the GitRepository for the additional workspace:

CODE
kubectl patch gitrepository -n ${WORKSPACE_NAMESPACE} dkp-catalog-applications --type merge --patch '{"spec":{"ref":{"tag":"v2.7.1"}}}'

For a final cleanup of Spark, delete its AppDeployment from all workspaces where it was deployed:

  1. Execute the following command to get the WORKSPACE_NAMESPACE of your workspace:

    CODE
    dkp get workspaces

    And copy the values under the NAMESPACE column for your workspace.

  2. Export the WORKSPACE_NAMESPACE variable:

    CODE
    export WORKSPACE_NAMESPACE=<WORKSPACE_NAMESPACE>
  3. Run the following command to delete the spark-operatorAppDeployment:

    CODE
    kubectl delete AppDeployment <spark operator appdeployment name> -n ${WORKSPACE_NAMESPACE}

Upgrade Catalog Applications

Before upgrading also, keep in mind the distinction between Platform applications and Catalog applications. Platform applications are deployed and upgraded as a set for each cluster or workspace. Catalog applications are deployed separately, so that you can deploy and upgrade them individually for each project.

Upgrade with UI

Follow these steps to upgrade an application from the DKP UI:

  1. From the top menu bar, select your target workspace.

  2. Select Applications from the sidebar menu.

  3. Select the three dot button from the bottom-right corner of the desired application tile, and then select Edit.

  4. Select the Version drop-down, and select a new version. This drop-down will only be available if there is a newer version to upgrade to.

  5. Select Save.

  6. Repeat this process for on each workspace where you have deployed the application.

Upgrade with CLI

Please note that the below commands are using the workspace name and not namespace.
You can retrieve the workspace name by running the command:
dkp get workspaces
To view a list of the deployed apps to your workspace, you can run the command:
dkp get appdeployments --workspace=<workspace-name>

  1. To see what app(s) and app versions are available to upgrade, run the following command:
    NOTE: You can reference the app version by going into the app name (e.g. <APP ID>-<APP VERSION>)

    CODE
    kubectl get apps -n ${WORKSPACE_NAMESPACE}

    You can also use this command to display the apps and app versions, for example:

    CODE
    kubectl get apps -n ${WORKSPACE_NAMESPACE} -o jsonpath='{range .items[*]}{@.spec.appId}{"----"}{@.spec.version}{"\n"}{end}'

    Below is an example of an output that shows the different apps and apps versions.

    CODE
    kafka-operator----0.20.0
    kafka-operator----0.20.2
    kafka-operator----0.23.0-dev.0
    zookeeper-operator----0.2.13
    zookeeper-operator----0.2.14
  2. Run the following command to upgrade an application from the DKP CLI:

    CODE
    dkp upgrade catalogapp <appdeployment-name> --workspace=${WORKSPACE_NAME} --to-version=<version.number>

    As an example, the following command upgrades the Kafka Operator application, named kafka-operator-abc, in a workspace to version 0.25.1:

    CODE
    dkp upgrade catalogapp kafka-operator-abc --workspace=${WORKSPACE_NAME} --to-version=0.25.1
  3. Repeat this process for on each workspace where you have deployed the application.

Platform applications cannot be upgraded on a one-off basis, and must be upgraded in a single process for each workspace. If you attempt to upgrade a platform application with these commands, you receive an error and the application is not upgraded.

As of DKP 2.7, the versions of Catalog applications other than these listed below are deprecated:

  • kafka-operator-0.25.1

  • zookeeper-operator-0.2.16-dkp.1

If you plan on upgrading to DKP 2.7 or above, ensure that you upgrade these applications to the latest compatible version.

  • To find what versions of applications are available for upgrade, use kubectl:

    CODE
    kubectl get apps -n ${WORKSPACE_NAMESPACE}

For more information, see Workspace DKP Catalog Applications | Workspace DKP Catalog Applications.

To ensure you do not install images with known CVE’s, you should specify a custom image for kafka and zookeeper by following these instructions:

Next Step:

Enterprise: Upgrade Project Catalog Applications

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.