Pre-provisioned Create a New Cluster

After you have defined the infrastructure and control plane endpoints, you create a Kubernetes cluster using the infrastructure definition. Create the cluster by following these steps to spin up a new pre-provisioned cluster.

Before you create a new DKP cluster below, you may choose an external load balancer or virtual IP and use the corresponding dkp create cluster command example from that page in the docs from the links below. Other customizations are available, but require different flags during dkp create cluster command also. Refer to Pre-provisioned Cluster Creation Customization Choices for more cluster customizations.

DKP uses localvolumeprovisioner as the default storage provider. However, localvolumeprovisioner is not suitable for production use. You should use a Kubernetes CSI compatible storage that is suitable for production.

You can choose from any of the storage options available for Kubernetes. To disable the default that Konvoy deploys, set the default StorageClasslocalvolumeprovisioner as non-default. Then set your newly created StorageClass to be the default by following the commands in the Kubernetes documentation called Changing the Default Storage Class.

NOTE: When specifying the cluster-name, you must use the same cluster-name as used when defining your inventory objects.

NOTE: To increase Docker Hub's rate limit use your Docker Hub credentials when creating the cluster, by setting the following flag --registry-mirror-url=https://registry-1.docker.io --registry-mirror-username= --registry-mirror-password= on the dkp create cluster command.
NOTE: Ensure your subnets do not overlap with your host subnet because they cannot be changed after cluster creation. If you need to change the kubernetes subnets, you must do this at cluster creation. The default subnets used in DKP are:

CODE

spec:
  clusterNetwork:
    pods:
      cidrBlocks:
      - 192.168.0.0/16
    services:
      cidrBlocks:
      - 10.96.0.0/12

Generate the Kubernetes cluster objects:

The following command relies on the pre-provisioned cluster API infrastructure provider to initialize the Kubernetes control plane and worker nodes on the hosts defined in the inventory. It uses the default external load balancer.
1. (Optional) If you have overrides for your clusters, you must specify the secret as part of the create cluster command. If these are not specified, the overrides for your nodes will not be applied.
  --override-secret-name=$CLUSTER_NAME-user-overrides
2. (Optional) Use a registry mirror. Configure your cluster to use an existing local registry as a mirror when attempting to pull images previously pushed to your registry. Instructions in the expandable section:

Export Registry Variables and Flags for Cluster Creation:

If you have a local registry, you must provide additional arguments when creating the cluster. These tell the cluster where to locate the local registry to use by defining the URL. Set the needed environment variable(s) with your registry information:

CODE

export REGISTRY_URL="<https/http>://<registry-address>:<registry-port>"
export REGISTRY_CA=<path to the CA on the bastion>
export REGISTRY_USERNAME=<username>
export REGISTRY_USERNAME=<password>

REGISTRY_URL: the address of an existing container registry accessible in the VPC that the new cluster nodes will be configured to use a mirror registry when pulling images.
REGISTRY_CA: (optional) the path on the bastion machine to the container registry CA. Konvoy will configure the cluster nodes to trust this CA. This value is only needed if the registry is using a self-signed certificate and the AMIs are not already configured to trust this CA.
REGISTRY_USERNAME: optional, set to a user that has pull access to this registry.
REGISTRY_PASSWORD: optional if username is not set.

When creating the cluster, apply the variables you defined above during the dkp create cluster command with the flags needed for your environment:

CODE

--registry-mirror-url=${REGISTRY_URL} \
--registry-mirror-cacert=${REGISTRY_CA} \
--registry-mirror-username=${REGISTRY_USERNAME} \
--registry-mirror-password=${REGISTRY_PASSWORD}

Create cluster command - Depending on the cluster size, it will take a few minutes to create the Kubernetes cluster objects:

CODE

dkp create cluster preprovisioned \
  --cluster-name ${CLUSTER_NAME} \
  --control-plane-endpoint-host <control plane endpoint host> \
  --control-plane-endpoint-port <control plane endpoint port, if different than 6443> \
  --override-secret-name=$CLUSTER_NAME-user-overrides \
  --dry-run \ 
  --output=yaml \ 
  > ${CLUSTER_NAME}.yaml

Virtual IP ALTERNATIVE - if you don’t have an external LB, and wish to use a VIRTUAL IP provided by kube-vip, specify these flags example below:

CODE

dkp create cluster preprovisioned \
    --cluster-name ${CLUSTER_NAME} \
    --control-plane-endpoint-host 196.168.1.10 \
    --virtual-ip-interface eth1 \
    --dry-run \ 
    --output=yaml \ 
    > ${CLUSTER_NAME}.yaml

HTTP only:

If your environment uses HTTP/HTTPS proxies, you must include the flags --http-proxy, --https-proxy, and --no-proxy and their related values in this command for it to be successful. More information is available in Configuring an HTTP/HTTPS Proxy.

CODE

--http-proxy <<http proxy list>>
--https-proxy <<https proxy list>>
--no-proxy <<no proxy list>>

To configure the Control Plane and Worker nodes to use an HTTP proxy:

CODE

export CONTROL_PLANE_HTTP_PROXY=http://example.org:8080
export CONTROL_PLANE_HTTPS_PROXY=http://example.org:8080
export CONTROL_PLANE_NO_PROXY="example.org,example.com,example.net,localhost,127.0.0.1,10.96.0.0/12,192.168.0.0/16,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,.svc,.svc.cluster,.svc.cluster.local,169.254.169.254"

export WORKER_HTTP_PROXY=http://example.org:8080
export WORKER_HTTPS_PROXY=http://example.org:8080
export WORKER_NO_PROXY="example.org,example.com,example.net,localhost,127.0.0.1,10.96.0.0/12,192.168.0.0/16,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local,.svc,.svc.cluster,.svc.cluster.local,169.254.169.254"

Replace:
- example.org,example.com,example.net with you internal addresses
- localhost and 127.0.0.1 addresses should not use the proxy
- 10.96.0.0/12 is the default Kubernetes service subnet
- 192.168.0.0/16 is the default Kubernetes pod subnet
- kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.default.svc.cluster.local is the internal Kubernetes kube-apiserver service
- .svc,.svc.cluster,.svc.cluster.local is the internal Kubernetes services

Use the flags in the dkp create cluster command:

CODE

--control-plane-http-proxy="${CONTROL_PLANE_HTTP_PROXY}" \
--control-plane-https-proxy="${CONTROL_PLANE_HTTPS_PROXY}" \
--control-plane-no-proxy="${CONTROL_PLANE_NO_PROXY}" \
--worker-http-proxy="${WORKER_HTTP_PROXY}" \
--worker-https-proxy="${WORKER_HTTPS_PROXY}" \
--worker-no-proxy="${WORKER_NO_PROXY}" \

FIPS Requirements

To create a cluster in FIPS mode, inform the controllers of the appropriate image repository and version tags of the official D2iQ FIPS builds of Kubernetes by adding those flags to dkp create cluster command:

CODE

--kubernetes-version=v1.27.11+fips.0 \
--etcd-version=3.5.10+fips.0 \
--kubernetes-image-repository=docker.io/mesosphere \

Individual manifests using the Output Directory flag:

You can create individual files with different smaller manifests for ease in editing using the --output-directory flag. This will create multiple files in the specified directory which must already exist:

CODE

--output-directory=<existing-directory>

Refer to the Cluster Creation Customization Choices section for more information on how to use optional flags such as the --output-directory flag.

Cluster creation results:

Output

The output from this command is shortened here for reading clarity, but should start like this:

CODE

Generating cluster resources
cluster.cluster.x-k8s.io/preprovisioned-example created
cont.........

Inspect or edit the cluster objects and familiarize yourself with Cluster API before editing the cluster objects as edits can prevent the cluster from deploying successfully. See Pre-provisioned Customizing CAPI Clusters .

Familiarize yourself with Cluster API before editing the cluster objects as edits can prevent the cluster from deploying successfully.

Create the cluster from the objects generated in the dry run. A warning will appear in the console if the resource already exists and will require you to remove the resource or update your YAML.
CODE
```
kubectl create -f ${CLUSTER_NAME}.yaml
```
NOTE: If you used the --output-directory flag in your dkp create .. --dry-run step above, create the cluster from the objects you created by specifying the directory:
CODE
```
kubectl create -f <existing-directory>/
```

Use the wait command to monitor the cluster control-plane readiness:

CODE

kubectl wait --for=condition=ControlPlaneReady "clusters/${CLUSTER_NAME}" --timeout=30m

Output:

CODE

cluster.cluster.x-k8s.io/preprovisioned-example condition met

After the creation, use this command to get the Kubernetes kubeconfig for the new cluster and begin deploying workloads:
CODE
```
dkp get kubeconfig -c ${CLUSTER_NAME} > ${CLUSTER_NAME}.conf
```

Azure requires changing the CNI encapsulation type of Calico from the default of IPtoIP to VXlan. If changing the Calico encapsulation, D2iQ recommends changing it after cluster creation, but before production.

Audit logs

To modify Control Plane Audit logs settings using the information contained in the page Configuring the Control Plane.

Further Optional Steps

Next Step

When you complete this procedure, move on to Pre-provisioned Make your Cluster Self-managed to continue the process.