Skip to main content
Skip table of contents

Post Conversion Cleanup: Clusters run on Different Cloud Platforms

For a DKP Enterprise Management cluster to manage a cluster hosted in another cloud provider, you must ensure the Management cluster has all the necessary permissions.

Prerequisites

Prior to running these commands, you must ensure that the DKP Management Enterprise cluster is configured with the necessary platform specific permissions to manage the incoming CAPI objects that backs the infrastructure resources in the target cloud platform.

For example, for the DKP Enterprise Managed cluster to manage CAPI clusters in AWS, refer tohttps://cluster-api-aws.sigs.k8s.io/topics/iam-permissions.html.

DKP supports expanding your platform in the following scenarios:

DKP Enterprise Management cluster host provider

DKP Enterprise Management cluster IAM permissions

DKP Essential cluster host provider

AWS

https://cluster-api-aws.sigs.k8s.io/topics/iam-permissions.html

AWS, GCP, vSphere, Pre-provisioned

GCP

https://cloud.google.com/iam/docs/overview

AWS, GCP, vSphere, Pre-provisioned

vSphere

https://docs.vmware.com/en/vRealize-Operations/Cloud/com.vmware.vcom.config.doc/GUID-F85638E3-937E-4E31-90D0-9D4A5E479292.html

AWS, GCP, vSphere, Pre-provisioned

Azure

https://learn.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-ops-guide-iam

Azure

Pre-provisioned

NA

AWS, GCP, vSphere, Pre-provisioned

Moving the CAPI Resources

  1. Following the conversion into a DKP Enterprise managed cluster, run the following command to move the CAPI Objects:

    CODE
    dkp move capi-resources --from-kubeconfig <essential_cluster_kubeconfig> --to-kubeconfig <enterprise_cluster_kubeconfig> --to-namespace ${WORKSPACE_NAMESPACE}
  2. Verify that the output looks similar to the following:

    CODE
    ✓ Moving cluster resources 
    You can now view resources in the moved cluster by using the --kubeconfig flag with kubectl. For example: kubectl --kubeconfig=<enterprise_cluster_kubeconfig> get nodes
  3. Since cert-manager is not managed by the Kommander component of DKP in an Essential cluster, you must change the ownership of the cert-manager application and give ownership to Kommander.
    On the expanded, now considered a Managed Cluster, annotate the elements so they are managed by the Kommander installation of cert-manager. First run the command to annotate release-namespace:

    CODE
    kubectl annotate deployment,sa,clusterrole,clusterrolebinding,role,service,MutatingWebhookConfiguration,ValidatingWebhookConfiguration,configmap,RoleBinding -l app.kubernetes.io/instance=cert-manager meta.helm.sh/release-namespace=cert-manager --all-namespaces

    Next run the command to annotate release-name:

    CODE
    kubectl annotate deployment,sa,clusterrole,clusterrolebinding,role,service,MutatingWebhookConfiguration,ValidatingWebhookConfiguration,configmap,RoleBinding -l app.kubernetes.io/instance=cert-manager meta.helm.sh/release-name=cert-manager --all-namespaces
  4. On the expanded (Managed Cluster), label the cert-manager to be managed by the Kommander installation of cert-manager by running:

    CODE
    kubectl label deployment,sa,clusterrole,clusterrolebinding,role,service,MutatingWebhookConfiguration,ValidatingWebhookConfiguration,configmap,RoleBinding -l app.kubernetes.io/instance=cert-manager app.kubernetes.io/managed-by=Helm --all-namespaces
  5. While still using the Managed Cluster, check to see if the other cert-manager objects are still ready:

    CODE
    kubectl get Issuers,ClusterIssuers,Certificates,CertificateRequests,Orders,Challenges --all-namespaces

    Output will be similar to this:

    CODE
    NAMESPACE   NAME                                              READY   AGE
                clusterissuer.cert-manager.io/kommander-ca        True    111m
                clusterissuer.cert-manager.io/selfsigned-issuer   True    111m
    
    NAMESPACE      NAME                                                READY   SECRET                       AGE
    cert-manager   certificate.cert-manager.io/kommander-ca            True    kommander-ca                 111m
    kommander      certificate.cert-manager.io/dex                     True    dex-tls                      101m
    kommander      certificate.cert-manager.io/git-tls                 True    git-tls                      111m
    kommander      certificate.cert-manager.io/kommander-traefik-tls   True    kommander-traefik-tls        102m
    kommander      certificate.cert-manager.io/kube-oidc-proxy         True    kube-oidc-proxy-server-tls   101m
    
    NAMESPACE      NAME                                                             APPROVED   DENIED   READY   ISSUER              REQUESTOR                                         AGE
    cert-manager   certificaterequest.cert-manager.io/kommander-ca-kk9zw            True                True    selfsigned-issuer   system:serviceaccount:cert-manager:cert-manager   111m
    kommander      certificaterequest.cert-manager.io/git-tls-dgcxv                 True                True    kommander-ca        system:serviceaccount:cert-manager:cert-manager   111m
    kommander      certificaterequest.cert-manager.io/kommander-traefik-tls-lqn8z   True                True    kommander-ca        system:serviceaccount:cert-manager:cert-manager   102m
    kommander      certificaterequest.cert-manager.io/kube-oidc-proxy-j5st6         True                True    kommander-ca        system:serviceaccount:cert-manager:cert-manager   101m
  6. The cert-manager application will still be running, but you want to enable the Workspace application for the cluster. This ensures that any updates you make to your Workspace will be enforced by Kommander and will upgrade as expected in the future.

    1. In the DKP UI, go to the Workspace you attached your Essential cluster.

    2. Select Applications from the left navigation menu.

    3. Locate the cert-manager in the Security section and select the three button menu at the bottom of the card and select Edit.

    4. From the edit cert-manager Workspace Application page, select the checkbox for the newly managed cluster and select the Enable/Save button.

  7. After moving the resources, run the following command to remove the CAPI controller manager deployments:

    CODE
    dkp delete capi-components --kubeconfig <essential_cluster_kubeconfig>

Next Step

Troubleshooting

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.