Prepare to deploy Kubernetes clusters

To create Kubernetes clusters, Konvoy uses Cluster API (CAPI) controllers. These controllers run on a Kubernetes cluster. To get started, you need a bootstrap cluster. By default, Konvoy creates a bootstrap cluster for you in a Docker container using the Kubernetes-in-Docker (KIND) tool.


Before you begin, you must:

  • Complete the steps in Prerequisites.
  • Ensure the dkp binary can be found in your $PATH.

Bootstrap Cluster Lifecycle Services

  1. If an HTTP proxy is required for the bootstrap cluster, set the local http_proxy, https_proxy, and no_proxy environment variables. Konvoy copies them into the bootstrap cluster.

  2. Create a bootstrap cluster:

    dkp create bootstrap --kubeconfig $HOME/.kube/config

    The output appears similar to:

    INFO[2021-11-23T15:54:07-08:00] Creating bootstrap cluster                    src="bootstrap/bootstrap.go:148"
    INFO[2021-11-23T15:55:01-08:00] Initializing bootstrap controllers            src="bootstrap/controllers.go:94"
    INFO[2021-11-23T15:56:05-08:00] Created bootstrap controllers                 src="bootstrap/controllers.go:106"
    INFO[2021-11-23T15:56:05-08:00] Bootstrap controllers are ready               src="bootstrap/controllers.go:110"
    INFO[2021-11-23T15:56:05-08:00] Initializing Tigera operator                  src="bootstrap/clusterresourceset.go:37"
    INFO[2021-11-23T15:56:05-08:00] Created/Updated Tigera operator               src="bootstrap/clusterresourceset.go:42"
    INFO[2021-11-23T15:56:05-08:00] Initializing AWS EBS CSI CustomResourceSet    src="bootstrap/clusterresourceset.go:95"
    INFO[2021-11-23T15:56:05-08:00] Created/Updated AWS EBS CSI CustomResourceSet  src="bootstrap/clusterresourceset.go:100"
    INFO[2021-11-23T15:56:05-08:00] Initializing Azure Disk CSI CustomResourceSet  src="bootstrap/clusterresourceset.go:102"
    INFO[2021-11-23T15:56:05-08:00] Created Azure Disk CustomResourceSet          src="bootstrap/clusterresourceset.go:107"
    INFO[2021-11-23T15:56:05-08:00] Initializing Local Volume Provisioner CustomResourceSet  src="bootstrap/clusterresourceset.go:109"
    INFO[2021-11-23T15:56:05-08:00] Created/Updated Local Volume Provisioner CustomResourceSet  src="bootstrap/clusterresourceset.go:114"
    INFO[2021-11-23T15:56:05-08:00] Initializing Cluster Autoscaler CustomResourceSet  src="bootstrap/clusterresourceset.go:181"
    INFO[2021-11-23T15:56:05-08:00] Created/Updated Cluster Autoscaler CustomResourceSet  src="bootstrap/clusterresourceset.go:186"
    INFO[2021-11-23T15:56:05-08:00] Initializing Node Feature Discovery CustomResourceSet  src="bootstrap/clusterresourceset.go:239"
    INFO[2021-11-23T15:56:05-08:00] Created/Updated Node Feature Discovery CustomResourceSet  src="bootstrap/clusterresourceset.go:244"
    INFO[2021-11-23T15:56:06-08:00] Initializing NVIDIA GPU Feature Discovery CustomResourceSet  src="bootstrap/clusterresourceset.go:297"
    INFO[2021-11-23T15:56:06-08:00] Created/Updated NVIDIA GPU Feature Discovery CustomResourceSet  src="bootstrap/clusterresourceset.go:302"

    Konvoy creates a bootstrap cluster using KIND as a library, and then deploys the following Cluster API providers on the cluster:

    Konvoy waits until the controller-manager and webhook deployments of these providers are ready.

    List these deployments using the command:

    kubectl get --all-namespaces deployments

    The output appears similar to

    NAMESPACE                           NAME                                            READY   UP-TO-DATE   AVAILABLE   AGE
    capa-system                         capa-controller-manager                         1/1     1            1           5m24s
    capi-kubeadm-bootstrap-system       capi-kubeadm-bootstrap-controller-manager       1/1     1            1           5m28s
    capi-kubeadm-control-plane-system   capi-kubeadm-control-plane-controller-manager   1/1     1            1           5m26s
    capi-system                         capi-controller-manager                         1/1     1            1           5m29s
    cappp-system                        cappp-controller-manager                        1/1     1            1           5m20s
    capz-system                         capz-controller-manager                         1/1     1            1           5m21s
    cert-manager                        cert-manager                                    1/1     1            1           5m52s
    cert-manager                        cert-manager-cainjector                         1/1     1            1           5m52s
    cert-manager                        cert-manager-webhook                            1/1     1            1           5m52s

    Konvoy then creates additional resources for Cluster API to apply to every new cluster. The resources, called ClusterResourceSets, contain complete YAML manifests to deploy essential cluster applications, such as the Calico Container Networking Interface (CNI) implementation, and Container Storage Interface (CSI) implementations for various infrastructure APIs. List the ClusterResourceSets using this command:

    kubectl get clusterresourceset

    The output appears similar to:

    NAME                       AGE
    aws-ebs-csi                5m38s
    azure-disk-csi             5m38s
    cluster-autoscaler         5m38s
    local-volume-provisioner   5m38s
    node-feature-discovery     5m38s
    nvidia-feature-discovery   5m37s
    tigera-operator            5m38s

    A ClusterResourceSet object defines selectors that match against cluster labels, and a reference to a ConfigMap. The ConfigMap contains a YAML manifest. When Konvoy creates a cluster with matching labels, it applies the YAML manifest to the cluster. Konvoy applies the manifest only once, during cluster creation.

    For example, this is the azure-disk-csi ClusterResourceSet, deployed by Konvoy from the actions described previously:

    kind: ClusterResourceSet
      name: azure-disk-csi
      - kind: ConfigMap
        name: azure-disk-csi
      strategy: ApplyOnce

    Konvoy defines the selectors and sets the correct labels on the Cluster objects. For a more detailed explanation of how ClusterResourceSets work, see the Extension Proposal.

(Optional) Create identity secret for Azure

If your bootstrap cluster resides on a Virtual machine inside Azure, create an identity secret that uses the cappz-controller:

export AZURE_CLUSTER_IDENTITY_SECRET_NAME="cluster-identity-secret"
export CLUSTER_IDENTITY_NAME="cluster-identity"

kubectl create secret generic "${AZURE_CLUSTER_IDENTITY_SECRET_NAME}" --from-literal=clientSecret="${AZURE_CLIENT_SECRET}"

When complete, move on to the Create a New Cluster section.