vSphere Create a New Air-gapped Cluster

Prerequisites

Before you begin, be sure that you have created a Bootstrap Cluster

Name your Cluster

1. Give your cluster a unique name suitable for your environment. The cluster name may only contain the following characters: a-z, 0-9, ., and -. Cluster creation will fail if the name has capital letters. See Kubernetes for more naming information.

2. When specifying the cluster-name, you must use the same cluster-name as used when defining your inventory objects.

3. Set the CLUSTER_NAME environment variable with the command:

   CODE

   export CLUSTER_NAME=<my-vsphere-cluster>

IMPORTANT: Ensure your subnets do not overlap with your host subnet because they cannot be changed after cluster creation. If you need to change the kubernetes subnets, you must do this at cluster creation. The default subnets used in DKP are:

• CODE

  spec:
    clusterNetwork:
      pods:
        cidrBlocks:
        - 192.168.0.0/16
      services:
        cidrBlocks:
        - 10.96.0.0/12

Create a New vSphere Kubernetes Cluster

Use the following steps to create a new, air-gapped vSphere cluster.

IMPORTANT: The image must be created by the konvoy-image-builder project in order to use the registry mirror feature.

1. Configure your cluster to use an existing registry as a mirror when attempting to pull images as done on the vSphere Air-gapped Seed the Registry page previously.

2. Create the Kubernetes cluster objects by copying the following command and substituting the valid values for your environment:

   CODE

   dkp create cluster vsphere \
     --cluster-name ${CLUSTER_NAME} \
     --network <NETWORK_NAME> \
     --control-plane-endpoint-host <CONTROL_PLANE_IP> \
     --data-center <DATACENTER_NAME> \
     --data-store <DATASTORE_NAME> \
     --folder <FOLDER_NAME> \
     --server <VCENTER_API_SERVER_URL> \
     --ssh-public-key-file </path/to/key.pub> \
     --resource-pool <RESOURCE_POOL_NAME> \
     --vm-template konvoy-ova-vsphere-os-release-k8s_release-vsphere-timestamp \
     --virtual-ip-interface eth0 \
     --extra-sans "127.0.0.1" \
     --registry-mirror-url=${REGISTRY_URL} \
     --registry-mirror-cacert=${REGISTRY_CA} \
     --registry-mirror-username=${REGISTRY_USERNAME} \
     --registry-mirror-password=${REGISTRY_PASSWORD} \
     --dry-run \
     --output=yaml \
   > ${CLUSTER_NAME}.yaml

Expand the drop-downs for HTTP, FIPS and other flags to use during the cluster creation step above.

--os-hint flatcar

--kubernetes-version=v1.27.11+fips.0 \
--etcd-version=3.5.10+fips.0 \
--kubernetes-image-repository=docker.io/mesosphere \

--http-proxy <<http proxy list>>
--https-proxy <<https proxy list>>
--no-proxy <<no proxy list>>

--output-directory=<existing-directory>

DKP uses the vsphere CSI driver as the default storage provider. Use a Kubernetes CSI compatible storage that is suitable for production. See the Kubernetes documentation called Changing the Default Storage Class for more information.

If you’re not using the default, you cannot deploy an alternate provider until after the dkp create cluster is finished. However, it must be determined before Kommander installation.

3. Inspect the created cluster resources with the command:

   CODE

   kubectl get clusters,kubeadmcontrolplanes,machinedeployments

4. (Optional) Edit the cluster objects and familiarize yourself with Cluster API before editing the cluster objects as edits can prevent the cluster from deploying successfully. See Customizing CAPI Components for a Cluster .

5. Create the cluster from the objects generated in the dry run. A warning will appear in the console if the resource already exists and will require you to remove the resource or update your YAML.

   CODE

   kubectl create -f ${CLUSTER_NAME}.yaml

   NOTE: If you used the --output-directory flag in your dkp create .. --dry-run step above, create the cluster from the objects you created by specifying the directory:

   CODE

   kubectl create -f <existing-directory>/

6. Use the wait command to monitor the cluster control-plane readiness:

   CODE

   kubectl wait --for=condition=ControlPlaneReady "clusters/${CLUSTER_NAME}" --timeout=20m

   Output:

   CODE

   cluster.cluster.x-k8s.io/${CLUSTER_NAME} condition met

   The READY status becomes True after the cluster control-plane becomes Ready in one of the following steps.

   After DKP creates the objects on the API server, the Cluster API controllers reconcile them, creating infrastructure and machines. As the controllers progress, they update the Status of each object.
   

7. Run the DKP describe command to monitor the current status of the cluster:

   CODE

   dkp describe cluster -c ${CLUSTER_NAME}

NAME                                                                READY  SEVERITY  REASON  SINCE  MESSAGE
Cluster/e2e-airgapped-1                                             True                     13h
├─ClusterInfrastructure - VSphereCluster/e2e-airgapped-1            True                     13h
├─ControlPlane - KubeadmControlPlane/e2e-airgapped-1-control-plane  True                     13h
│ ├─Machine/e2e-airgapped-1-control-plane-7llgd                     True                     13h
│ ├─Machine/e2e-airgapped-1-control-plane-vncbl                     True                     13h
│ └─Machine/e2e-airgapped-1-control-plane-wbgrm                     True                     13h
└─Workers
    └─MachineDeployment/e2e-airgapped-1-md-0                          True                     13h
    ├─Machine/e2e-airgapped-1-md-0-74c849dc8c-67rv4                 True                     13h
    ├─Machine/e2e-airgapped-1-md-0-74c849dc8c-n2skc                 True                     13h
    ├─Machine/e2e-airgapped-1-md-0-74c849dc8c-nkftv                 True                     13h
    └─Machine/e2e-airgapped-1-md-0-74c849dc8c-sqklv                 True                     13h

6. As they progress, the controllers also create Events, which you can list using the command:

   CODE

   kubectl get events | grep ${CLUSTER_NAME}

   For brevity, this example uses grep. You can also use separate commands to get Events for specific objects, such as kubectl get events --field-selector involvedObject.kind="VSphereCluster" and kubectl get events --field-selector involvedObject.kind="VSphereMachine".

Known Limitations

NOTE: Be aware of these limitations in the current release of DKP Konvoy.

• The DKP Konvoy version used to create a bootstrap cluster must match the DKP Konvoy version used to create a workload cluster.

• DKP Konvoy supports deploying one workload cluster.

• DKP Konvoy generates a set of objects for one Node Pool.

• DKP Konvoy does not validate edits to cluster objects.

Next Step

vSphere Make your Air-gapped Cluster Self-Managed