Pre-provisioned Prerequisites Air-gapped

Fulfill the prerequisites for using a pre-provisioned infrastructure when Air-Gapped

The instructions below outline how to fulfill the prerequisites for using pre-provisioned infrastructure when using air-gapped. In DKP 2.4.0, there is a new complete DKP air-gapped bundle available to download which contains all the DKP components needed for air-gapped installation. (i.e. dkp-air-gapped-bundle_v2.4.0_linux_amd64.tar.gz)

Air-Gapped Registry Prerequisites

JFrog Artifactory

If you use Jfrog Artifactory or Jfrog Container Registry, you must update to a new version of the software. Any build newer than version 7.11 will work, as we have confirmed that older versions are not compatible.

Nexus Registry

If you use Nexus Registry, there was an issue that prevented usage with DKP 2.X and OCI Images, but support for OCI Images was added here in this publicly available Jira ticket:

[NEXUS-21087] Support OCI registry format - Sonatype JIRA

Harbor Registry

Any newer version than Harbor Registry v2.1.1-5f52168e will support OCI images.

Load the bootstrap image

Assuming you have downloaded dkp-air-gapped-bundle_v2.4.0_linux_amd64.tar.gz from the download site mentioned above, extract the tarball to a local directory:
```
tar -xzvf dkp-air-gapped-bundle_v2.4.0_linux_amd64.tar.gz && cd dkp-v2.4.0
```
CODE
Load the bootstrap Docker image on your bastion machine:
```
docker load -i konvoy-bootstrap-image-v2.4.0.tar
```
CODE

Copy air-gapped artifacts onto cluster hosts

Using the Konvoy Image Builder, you can copy the required artifacts onto your cluster hosts.

Assuming you have downloaded dkp-air-gapped-bundle_v2.4.0_linux_amd64.tar.gz , extract the tarball to a local directory:
```
tar -xzvf dkp-air-gapped-bundle_v2.4.0_linux_amd64.tar.gz && cd dkp-v2.4.0/kib
```
CODE

The kubernetes image bundle will be located in kib/artifacts/images and you will want to verify image and artifacts.

Verify the image bundles exist in artifacts/images:

$ ls artifacts/images/
kubernetes-images-1.24.6-d2iq.1.tar kubernetes-images-1.24.6-d2iq.1-fips.tar

CODE

Verify the artifacts for your OS exist in the artifacts/ directory and export the appropriate variables:

$ ls artifacts/
1.24.6_centos_7_x86_64_fips.tar.gz  1.24.6_redhat_8_x86_64_fips.tar.gz                      containerd-1.4.13-d2iq.1-rhel-7.9-x86_64_fips.tar.gz  containerd-1.4.13-d2iq.1-rhel-8.4-x86_64_fips.tar.gz  images
1.24.6_centos_7_x86_64.tar.gz       1.24.6_redhat_8_x86_64.tar.gz                           containerd-1.4.13-d2iq.1-rhel-7.9-x86_64.tar.gz       containerd-1.4.13-d2iq.1-rhel-8.4-x86_64.tar.gz       NVIDIA-Linux-x86_64-470.82.01.run
1.24.6_redhat_7_x86_64_fips.tar.gz  containerd-1.4.13-d2iq.1-centos-7.9-x86_64_fips.tar.gz  containerd-1.4.13-d2iq.1-rhel-8.2-x86_64_fips.tar.gz  containerd-1.4.13-d2iq.1-rhel-8.6-x86_64_fips.tar.gz  pip-packages.tar.gz
1.24.6_redhat_7_x86_64.tar.gz       containerd-1.4.13-d2iq.1-centos-7.9-x86_64.tar.gz       containerd-1.4.13-d2iq.1-rhel-8.2-x86_64.tar.gz       containerd-1.4.13-d2iq.1-rhel-8.6-x86_64.tar.gz

CODE

For example, for RHEL 8.4 you would set:

export OS_PACKAGES_BUNDLE=1.24.6_redhat_8_x86_64.tar.gz
export CONTAINERD_BUNDLE=containerd-1.4.13-d2iq.1-rhel-8.4-x86_64.tar.gz

CODE

Export the following environment variables, ensuring that all control plane and worker nodes are included:

export CONTROL_PLANE_1_ADDRESS="<control-plane-address-1>"
export CONTROL_PLANE_2_ADDRESS="<control-plane-address-2>"
export CONTROL_PLANE_3_ADDRESS="<control-plane-address-3>"
export WORKER_1_ADDRESS="<worker-address-1>"
export WORKER_2_ADDRESS="<worker-address-2>"
export WORKER_3_ADDRESS="<worker-address-3>"
export WORKER_4_ADDRESS="<worker-address-4>"
export SSH_USER="<ssh-user>"
export SSH_PRIVATE_KEY_FILE="<private key file>"

CODE

SSH_PRIVATE_KEY_FILE must be either the name of the SSH private key file in your working directory or an absolute path to the file in your user’s home directory.

Generate an inventory.yaml which is automatically picked up by the konvoy-image upload in the next step. This inventory.yaml should exclude any GPU workers, which will be handled in steps #6-7.

cat <<EOF > inventory.yaml
all:
  vars:
    ansible_user: $SSH_USER
    ansible_port: 22
    ansible_ssh_private_key_file: $SSH_PRIVATE_KEY_FILE
  hosts:
    $CONTROL_PLANE_1_ADDRESS:
      ansible_host: $CONTROL_PLANE_1_ADDRESS
    $CONTROL_PLANE_2_ADDRESS:
      ansible_host: $CONTROL_PLANE_2_ADDRESS
    $CONTROL_PLANE_3_ADDRESS:
      ansible_host: $CONTROL_PLANE_3_ADDRESS
    $WORKER_1_ADDRESS:
      ansible_host: $WORKER_1_ADDRESS
    $WORKER_2_ADDRESS:
      ansible_host: $WORKER_2_ADDRESS
    $WORKER_3_ADDRESS:
      ansible_host: $WORKER_3_ADDRESS
    $WORKER_4_ADDRESS:
      ansible_host: $WORKER_4_ADDRESS
EOF

CODE

Upload the artifacts onto cluster hosts with the following command:

konvoy-image upload artifacts \
              --container-images-dir=./artifacts/images/ \
              --os-packages-bundle=./artifacts/$OS_PACKAGES_BUNDLE \
              --containerd-bundle=artifacts/$CONTAINERD_BUNDLE \
              --pip-packages-bundle=./artifacts/pip-packages.tar.gz

BASH

KIB uses variable overrides to specify base image and container images to use in your new machine image. The variable overrides files for NVIDIA and FIPS can be ignored unless adding an overlay feature.

Use the --overrides flag and reference either fips.yaml or offline-fips.yaml manifests located in the overrides directory or see these pages in the documentation:
- FIPS Overrides
- Create FIPS 140 Images

GPU Only Steps

If the NVIDIA runfile installer has not been downloaded, then retrieve and install the download first by running the following command. The first line in the command below downloads and installs the runfile and the second line places it in the artifacts directory.

curl -O https://download.nvidia.com/XFree86/Linux-x86_64/470.82.01/NVIDIA-Linux-x86_64-470.82.01.run
mv NVIDIA-Linux-x86_64-470.82.01.run artifacts

CODE

6. Create an inventory for GPU Nodes.

cat <<EOF >  gpu_inventory.yaml 
all:
  vars:
    ansible_port: 22
    ansible_ssh_private_key_file: $SSH_PRIVATE_KEY_FILE
    ansible_user: $SSH_USER

  hosts:
    $GPU_WORKER_1_ADDRESS:
      ansible_host: $GPU_WORKER_1_ADDRESS
EOF

CODE

7. Upload the artifacts to the gpu nodepool with the nvidia-runfile flag

konvoy-image upload artifacts --inventory-file=gpu_inventory.yaml \
              --container-images-dir=./artifacts/images/ \
              --os-packages-bundle=./artifacts/$OS_PACKAGES_BUNDLE \
              --containerd-bundle=artifacts/$CONTAINERD_BUNDLE \
              --pip-packages-bundle=./artifacts/pip-packages.tar.gz \
              --nvidia-runfile=./artifacts/NVIDIA-Linux-x86_64-470.82.01.run

CODE

KIB uses variable overrides to specify base image and container images to use in your new machine image. The variable overrides files for NVIDIA and FIPS can be ignored unless adding an overlay feature.

Use the --overrides overrides/fips.yaml,overrides/offline-fips.yaml flag with manifests located in the overrides directory or see these pages in the documentation:
- FIPS Overrides
- Create FIPS 140 Images

Seed your docker registry

Before creating a Kubernetes cluster you must have the required images in a local docker registry. This registry must be accessible from both the bastion machine and the machines that will be created for the Kubernetes cluster.

Assuming you have downloaded dkp-air-gapped-bundle_v2.4.0_linux_amd64.tar.gz , extract the tarball to a local directory:
```
tar -xzvf dkp-air-gapped-bundle_v2.4.0_linux_amd64.tar.gz && cd dkp-v2.4.0
```
CODE

Set an environment variable with your registry address:

export DOCKER_REGISTRY_ADDRESS=<registry-address>:<registry-port>

CODE

Run the following command to load the air-gapped image bundle into your private Docker registry:

./dkp push image-bundle --image-bundle ./container-images/konvoy-image-bundle-v2.4.0.tar --to-registry $DOCKER_REGISTRY_ADDRESS

CODE

It may take a while to push all the images to your image registry, depending on the performance of the network between the machine you are running the script on and the Docker registry.

Then begin creating the bootstrap cluster.