Skip to main content
Skip table of contents

Pre-provisioned Air-gapped GPU: Install Kommander

Prerequisites

Create your Kommander Installer Configuration File

  1. Set the environment variable for your cluster:

    CODE
    export CLUSTER_NAME=<your-management-cluster-name>
  2. Copy the kubeconfig file of your Management cluster to your local directory:

    CODE
    dkp get kubeconfig -c ${CLUSTER_NAME} >> ${CLUSTER_NAME}.conf
  3. Create a configuration file for the deployment:

    CODE
    dkp install kommander --init --airgapped > kommander.yaml
  4. Edit the installer file to include configuration overrides for the rook-ceph-cluster. DKP’s default configuration ships Ceph with PVC based storage which requires your CSI provider to support PVC with type volumeMode: Block. As this is not possible with the default local static provisioner, you can install Ceph in host storage mode.

    You can choose whether Ceph’s object storage daemon (osd) pods should consume all or just some of the devices on your nodes. Include one of the following Overrides:

    1. To automatically assign all raw storage devices on all nodes to the Ceph cluster:

      CODE
      rook-ceph-cluster:
          enabled: true
          values: |
            cephClusterSpec:
              storage:
                storageClassDeviceSets: []
                useAllDevices: true
                useAllNodes: true
                deviceFilter: "<<value>>"
    2. To assign specific storage devices on all nodes to the Ceph cluster:

      CODE
      rook-ceph-cluster:
          enabled: true
          values: |
            cephClusterSpec:
              storage:
                storageClassDeviceSets: []
                useAllNodes: true
                useAllDevices: false
                deviceFilter: "^sdb."

      (info) Note: If you want to assign specific devices to specific nodes using the deviceFilter option, refer to Specific Nodes and Devices. For general information on the deviceFilter value, refer to Storage Selection Settings.

  5. If required: Customize your kommander.yaml.
    (info) See Kommander Customizations for customization options. Some of them include:
    Custom Domains and Certificates, HTTP proxy, External Load Balancer, GPU utilization, etc.

Enable GPU Resources

  1. In the same kommander.yaml file, enable Nvidia platform services.

    CODE
    apps:
      nvidia-gpu-operator:
       enabled: true
  2. Append the correct Toolkit version based on your OS:

    The NVIDIA Container Toolkit allows users to run GPU accelerated containers. The toolkit includes a container runtime library and utilities to automatically configure containers to leverage NVIDIA GPU and must be configured correctly according to your base operating system.

    Centos 7.9/RHEL 7.9:
    If you’re using Centos 7.9 or RHEL 7.9 as the base operating system for your GPU enabled nodes, set the toolkit.version parameter in your Kommander Installer Configuration file or <kommander.yaml> to the following:

    CODE
    kind: Installation
    apps:
      nvidia-gpu-operator:
       enabled: true
       values: |
         toolkit:
           version: v1.13.1-centos7

    RHEL 8.4/8.6 and SLES 15 SP3
    If you’re using RHEL 8.4/8.6 or SLES 15 SP3 as the base operating system for your GPU enabled nodes, set the toolkit.version parameter in your Kommander Installer Configuration file or <kommander.yaml> to the following:

    CODE
    kind: Installation
    apps:
      nvidia-gpu-operator:
       enabled: true
       values: |
         toolkit:
           version: v1.13.1-ubi8

    Ubuntu 18.04 and 20.04
    If you’re using Ubuntu 18.04 or 20.04 as the base operating system for your GPU enabled nodes, set the toolkit.version parameter in your Kommander Installer Configuration file or <kommander.yaml> to the following:

    CODE
    kind: Installation
    apps:
      nvidia-gpu-operator:
       enabled: true
       values: |
         toolkit:
           version: v1.13.1-ubuntu20.04

Enable DKP Catalog Applications and Install Kommander in an Air-gapped Environment

If you want to enable DKP Catalog applications after installing DKP, see Enable DKP Catalog Applications after Installing DKP.

  1. In the same kommander.yaml of the previous section, add the following values to enable DKP Catalog Applications:

    CODE
    apiVersion: config.kommander.mesosphere.io/v1alpha1
    kind: Installation
    ...
    catalog:
      repositories:
        - name: dkp-catalog-applications
          labels:
            kommander.d2iq.io/project-default-catalog-repository: "true"
            kommander.d2iq.io/workspace-default-catalog-repository: "true"
            kommander.d2iq.io/gitapps-gitrepository-type: "dkp"
          path: ./dkp-catalog-applications-v2.6.1.tar.gz

    (warning) If you only want to enable catalog applications to an existing configuration, add these values to an existing installer configuration file to maintain your Management cluster’s settings.

  2. Use the customized kommander.yaml to install DKP:

    CODE
    dkp install kommander --installer-config kommander.yaml --kubeconfig=${CLUSTER_NAME}.conf \
    --kommander-applications-repository ./application-repositories/kommander-applications-v2.6.1.tar.gz \
    --charts-bundle ./application-charts/dkp-kommander-charts-bundle-v2.6.1.tar.gz \
    --charts-bundle ./application-charts/dkp-catalog-applications-charts-bundle-v2.6.1.tar.gz

Tips and recommendations

  • The --kubeconfig=${CLUSTER_NAME}.conf flag ensures that you install Kommander on the correct cluster. For alternatives, see Provide Context for Commands with a kubeconfig File.

  • Applications can take longer to deploy, and time out the installation. Add the --wait-timeout <time to wait> flag and specify a period of time (for example, 1h) to allocate more time to the deployment of applications.

  • If the Kommander installation fails, or you wish to reconfigure applications, rerun the install command to retry.

Next Step:

Pre-provisioned Air-gapped GPU: Verify Install and Log in to UI

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.