Pre-provisioned Azure only Configurations

After your bootstrap is running and your cluster is created, you will need to install the Azure Disk CSI Driver on your pre-provisioned Azure Kubernetes cluster. The DKP pre-provisioned provider installs by default the storage-local-static-provisioner CSI driver, which is not suitable for production environments. For this reason, it needs to be replaced by the Azure Disk CSI Driver.

Prerequisites:

Before you begin using DKP you must have:

An x86_64-based Linux or macOS machine.
Download the dkp binary for Linux, or macOS. To check which version of DKP you installed for compatibility reasons, run the dkp version -o command (dkp version).
A container engine:
- Version 4.0 of Podman or higher for Linux
- Version 18.09.2 of Docker or higher for Linux or MacOS
kubectl for interacting with the running cluster.
Azure CLI.
A valid Azure account with credentials configured.
Create a custom Azure image using KIB.
For air-gapped environments only -
- Ability to download artifacts from the internet and then copy those onto your bastion machine.
- Download the Complete DKP Air-gapped Bundle for this release - dkp-air-gapped-bundle_v2.5.0_linux_amd64.tar.gz.
- An existing local registry to seed the air-gapped environment.

On macOS, Docker runs in a virtual machine. Configure this virtual machine with at least 8GB of memory.

Set Environment Variables with Credentials:

An Azure Service Principal is needed for deploying resources. To configure your Azure environment, follow below:

Log in to Azure:

CODE

az login

CODE

[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "a1234567-b132-1234-1a11-1234a5678b90",
    "id": "b1234567-abcd-11a1-a0a0-1234a5678b90",
    "isDefault": true,
    "managedByTenants": [],
    "name": "Mesosphere Developer Subscription",
    "state": "Enabled",
    "tenantId": "a1234567-b132-1234-1a11-1234a5678b90",
    "user": {
      "name": "user@azuremesosphere.onmicrosoft.com",
      "type": "user"
    }
  }
]

Create an Azure Service Principal (SP) by running the following command:
Note: If an SP with the name exists, this command will rotate the password.
CODE
```
az ad sp create-for-rbac --role contributor --name "$(whoami)-konvoy" --scopes=/subscriptions/$(az account show --query id -o tsv)
```
CODE
```
{
  "appId": "7654321a-1a23-567b-b789-0987b6543a21",
  "displayName": "azure-cli-2021-03-09-23-17-06",
  "password": "Z79yVstq_E.R0R7RUUck718vEHSuyhAB0C",
  "tenant": "a1234567-b132-1234-1a11-1234a5678b90"
}
```
For air-gapped environments, you need to create a resource management private link with a private endpoint to ensure the Azure CSI driver will run correctly in further steps. Private links enable you to access Azure services over a private endpoint in your virtual network.
To set up a private link resource, use the following process.
1. Create the resource management private link using Azure CLI.
2. Create a private link association for the root management group which also references the resource ID for the resource management private link.
3. Add a private endpoint that references the resource management private link using the Azure Documentation.

Set the required environment variables using that output:

CODE

export AZURE_SUBSCRIPTION_ID="<id>"       # b1234567-abcd-11a1-a0a0-1234a5678b90
export AZURE_TENANT_ID="<tenant>"         # a1234567-b132-1234-1a11-1234a5678b90
export AZURE_CLIENT_ID="<appId>"          # 7654321a-1a23-567b-b789-0987b6543a21
export AZURE_CLIENT_SECRET="<password>"   # Z79yVstq_E.R0R7RUUck718vEHSuyhAB0C
export AZURE_RESOURCE_GROUP="<resource group name>" # set to the name of the resource group
export AZURE_LOCATION="westus"            # set to the location you are using

Set your KUBECONFIG environment variable:
CODE
```
export kubeconfig=${CLUSTER_NAME}.conf
```

Create the Secret with the Azure credentials, this will be used by the Azure CSI driver:

Create an azure.json file:

CODE

cat <<EOF > azure.json
{
  "cloud": "AzurePublicCloud",
  "tenantId": "$AZURE_TENANT_ID",
  "subscriptionId": "$AZURE_SUBSCRIPTION_ID",
  "aadClientId": "$AZURE_CLIENT_ID",
  "aadClientSecret": "$AZURE_CLIENT_SECRET",
  "resourceGroup": "$AZURE_RESOURCE_GROUP",
  "location": "$AZURE_LOCATION"
}
EOF

Create the Secret:

CODE

kubectl create secret generic azure-cloud-provider --namespace=kube-system --type=Opaque --from-file=cloud-config=azure.json

Install the Azure Disk CSI driver:

CODE

$ curl -skSL https://raw.githubusercontent.com/kubernetes-sigs/azuredisk-csi-driver/v1.26.2/deploy/install-driver.sh | bash -s v1.26.2 snapshot –

Check the status to see if the driver is ready for use:

CODE

kubectl -n kube-system get pod -o wide --watch -l app=csi-azuredisk-controller
kubectl -n kube-system get pod -o wide --watch -l app=csi-azuredisk-node

Now Kubernetes knows that this is Azure disk, and will create clusters on Azure. You are ready to create the StorageClass for the Azure Disk CSI Driver:
CODE
```
kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/azuredisk-csi-driver/master/deploy/example/storageclass-azuredisk-csi.yaml
```

Change the default storage class to this new StorageClass so that every new disk will be created in the Azure environment:

CODE

kubectl patch sc/localvolumeprovisioner -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
kubectl patch sc/managed-csi -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Verify that the StorageClass chosen is currently the default:
CODE
```
kubectl get storageclass
```

For more information about Azure Disk CSI for persistent storage and changing the default StorageClass, refer to that page in the documentation: Default Storage Providers in DKP

Next Step:

Pre-provisioned Modify the Calico Installation