Cluster IAM Policies, Roles, and Artifacts

This guides a DKP user in creating IAM Policies and Instance Profiles used by the cluster’s control plane and worker nodes using the provided AWS CloudFormation Stack. For multi-tenancy, every tenant should be in a different AWS account to ensure they are truly independent of other tenants in order to enforce security.

Prerequisites

Before applying the IAM Policies, verify the following:

You have a valid AWS account with credentials configured that can manage CloudFormation Stacks, IAM Policies, IAM Roles, and IAM Instance Profiles.
You have the AWS CLI utility installed.

Below is information regarding the set up for policies and roles. After reading the information for each of these areas, you will find the CloudFormation Stack that creates these policies and roles in the IAM Artifacts drop-down section below:

Policies

AWSIAMManagedPolicyCloudProviderControlPlane enumerates the Actions required by the workload cluster control plane machines. It is attached to the AWSIAMRoleControlPlane Role.
AWSIAMManagedPolicyCloudProviderNodes enumerates the Actions required by the workload cluster worker machines. It is attached to the AWSIAMRoleNodes Role.
AWSIAMManagedPolicyControllers enumerates the Actions required by the workload cluster worker machines. It is attached to the AWSIAMRoleControlPlane Role.

Roles

AWSIAMRoleControlPlane is the Role associated with the AWSIAMInstanceProfileControlPlane Instance Profile.
AWSIAMRoleNodes is the Role associated with the AWSIAMInstanceProfileNodes Instance Profile.

For more information on learning how to grant cluster access to IAM users and roles, see the official AWS Documentation.

IAM Artifacts

Instance Profiles

AWSIAMInstanceProfileControlPlane, assigned to workload cluster control plane machines.

If the name is changed from the default, used below, it must be passed to dkp create cluster with the --control-plane-iam-instance-profile flag.

2. AWSIAMInstanceProfileNodes, assigned to workload cluster worker machines.

If the name is changed from the default, used below, it must be passed to dkp create cluster with the --worker-iam-instance-profile flag.

YAML

AWSTemplateFormatVersion: 2010-09-09
Resources:
  AWSIAMInstanceProfileControlPlane:
    Properties:
      InstanceProfileName: control-plane.cluster-api-provider-aws.sigs.k8s.io
      Roles:
      - Ref: AWSIAMRoleControlPlane
    Type: AWS::IAM::InstanceProfile
  AWSIAMInstanceProfileNodes:
    Properties:
      InstanceProfileName: nodes.cluster-api-provider-aws.sigs.k8s.io
      Roles:
      - Ref: AWSIAMRoleNodes
    Type: AWS::IAM::InstanceProfile
  AWSIAMManagedPolicyCloudProviderControlPlane:
    Properties:
      Description: For the Kubernetes Cloud Provider AWS Control Plane
      ManagedPolicyName: control-plane.cluster-api-provider-aws.sigs.k8s.io
      PolicyDocument:
        Statement:
        - Action:
          - autoscaling:DescribeAutoScalingGroups
          - autoscaling:DescribeLaunchConfigurations
          - autoscaling:DescribeTags
          - ec2:DescribeInstances
          - ec2:DescribeImages
          - ec2:DescribeRegions
          - ec2:DescribeRouteTables
          - ec2:DescribeSecurityGroups
          - ec2:DescribeSubnets
          - ec2:DescribeVolumes
          - ec2:CreateSecurityGroup
          - ec2:CreateTags
          - ec2:CreateVolume
          - ec2:ModifyInstanceAttribute
          - ec2:ModifyVolume
          - ec2:AttachVolume
          - ec2:AuthorizeSecurityGroupIngress
          - ec2:CreateRoute
          - ec2:DeleteRoute
          - ec2:DeleteSecurityGroup
          - ec2:DeleteVolume
          - ec2:DetachVolume
          - ec2:RevokeSecurityGroupIngress
          - ec2:DescribeVpcs
          - elasticloadbalancing:AddTags
          - elasticloadbalancing:AttachLoadBalancerToSubnets
          - elasticloadbalancing:ApplySecurityGroupsToLoadBalancer
          - elasticloadbalancing:CreateLoadBalancer
          - elasticloadbalancing:CreateLoadBalancerPolicy
          - elasticloadbalancing:CreateLoadBalancerListeners
          - elasticloadbalancing:ConfigureHealthCheck
          - elasticloadbalancing:DeleteLoadBalancer
          - elasticloadbalancing:DeleteLoadBalancerListeners
          - elasticloadbalancing:DescribeLoadBalancers
          - elasticloadbalancing:DescribeLoadBalancerAttributes
          - elasticloadbalancing:DetachLoadBalancerFromSubnets
          - elasticloadbalancing:DeregisterInstancesFromLoadBalancer
          - elasticloadbalancing:ModifyLoadBalancerAttributes
          - elasticloadbalancing:RegisterInstancesWithLoadBalancer
          - elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer
          - elasticloadbalancing:AddTags
          - elasticloadbalancing:CreateListener
          - elasticloadbalancing:CreateTargetGroup
          - elasticloadbalancing:DeleteListener
          - elasticloadbalancing:DeleteTargetGroup
          - elasticloadbalancing:DescribeListeners
          - elasticloadbalancing:DescribeLoadBalancerPolicies
          - elasticloadbalancing:DescribeTargetGroups
          - elasticloadbalancing:DescribeTargetHealth
          - elasticloadbalancing:ModifyListener
          - elasticloadbalancing:ModifyTargetGroup
          - elasticloadbalancing:RegisterTargets
          - elasticloadbalancing:SetLoadBalancerPoliciesOfListener
          - iam:CreateServiceLinkedRole
          - kms:DescribeKey
          - kms:CreateGrant
          Effect: Allow
          Resource:
          - '*'
        Version: 2012-10-17
      Roles:
      - Ref: AWSIAMRoleControlPlane
    Type: AWS::IAM::ManagedPolicy
  AWSIAMManagedPolicyCloudProviderNodes:
    Properties:
      Description: For the Kubernetes Cloud Provider AWS nodes
      ManagedPolicyName: nodes.cluster-api-provider-aws.sigs.k8s.io
      PolicyDocument:
        Statement:
        - Action:
          - ec2:DescribeInstances
          - ec2:DescribeRegions
          - ecr:GetAuthorizationToken
          - ecr:BatchCheckLayerAvailability
          - ecr:GetDownloadUrlForLayer
          - ecr:GetRepositoryPolicy
          - ecr:DescribeRepositories
          - ecr:ListImages
          - ecr:BatchGetImage
          Effect: Allow
          Resource:
          - '*'
        - Action:
          - secretsmanager:DeleteSecret
          - secretsmanager:GetSecretValue
          Effect: Allow
          Resource:
          - arn:*:secretsmanager:*:*:secret:aws.cluster.x-k8s.io/*
        - Action:
          - ssm:UpdateInstanceInformation
          - ssmmessages:CreateControlChannel
          - ssmmessages:CreateDataChannel
          - ssmmessages:OpenControlChannel
          - ssmmessages:OpenDataChannel
          - s3:GetEncryptionConfiguration
          Effect: Allow
          Resource:
          - '*'
        Version: 2012-10-17
      Roles:
      - Ref: AWSIAMRoleControlPlane
      - Ref: AWSIAMRoleNodes
    Type: AWS::IAM::ManagedPolicy
  AWSIAMManagedPolicyControllers:
    Properties:
      Description: For the Kubernetes Cluster API Provider AWS Controllers
      ManagedPolicyName: controllers.cluster-api-provider-aws.sigs.k8s.io
      PolicyDocument:
        Statement:
        - Action:
          - ec2:AllocateAddress
          - ec2:AssociateRouteTable
          - ec2:AttachInternetGateway
          - ec2:AuthorizeSecurityGroupIngress
          - ec2:CreateInternetGateway
          - ec2:CreateNatGateway
          - ec2:CreateRoute
          - ec2:CreateRouteTable
          - ec2:CreateSecurityGroup
          - ec2:CreateSubnet
          - ec2:CreateTags
          - ec2:CreateVpc
          - ec2:ModifyVpcAttribute
          - ec2:DeleteInternetGateway
          - ec2:DeleteNatGateway
          - ec2:DeleteRouteTable
          - ec2:DeleteSecurityGroup
          - ec2:DeleteSubnet
          - ec2:DeleteTags
          - ec2:DeleteVpc
          - ec2:DescribeAccountAttributes
          - ec2:DescribeAddresses
          - ec2:DescribeAvailabilityZones
          - ec2:DescribeInstanceTypes
          - ec2:DescribeInternetGateways
          - ec2:DescribeImages
          - ec2:DescribeNatGateways
          - ec2:DescribeNetworkInterfaces
          - ec2:DescribeNetworkInterfaceAttribute
          - ec2:DescribeRouteTables
          - ec2:DescribeSecurityGroups
          - ec2:DescribeSubnets
          - ec2:DescribeVpcs
          - ec2:DescribeVpcAttribute
          - ec2:DescribeVolumes
          - ec2:DetachInternetGateway
          - ec2:DisassociateRouteTable
          - ec2:DisassociateAddress
          - ec2:ModifyInstanceAttribute
          - ec2:ModifyInstanceMetadataOptions
          - ec2:ModifyNetworkInterfaceAttribute
          - ec2:ModifySubnetAttribute
          - ec2:ReleaseAddress
          - ec2:RevokeSecurityGroupIngress
          - ec2:RunInstances
          - ec2:TerminateInstances
          - tag:GetResources
          - elasticloadbalancing:AddTags
          - elasticloadbalancing:CreateLoadBalancer
          - elasticloadbalancing:ConfigureHealthCheck
          - elasticloadbalancing:DeleteLoadBalancer
          - elasticloadbalancing:DescribeLoadBalancers
          - elasticloadbalancing:DescribeLoadBalancerAttributes
          - elasticloadbalancing:DescribeTargetGroups
          - elasticloadbalancing:ApplySecurityGroupsToLoadBalancer
          - elasticloadbalancing:DescribeTags
          - elasticloadbalancing:ModifyLoadBalancerAttributes
          - elasticloadbalancing:RegisterInstancesWithLoadBalancer
          - elasticloadbalancing:DeregisterInstancesFromLoadBalancer
          - elasticloadbalancing:RemoveTags
          - autoscaling:DescribeAutoScalingGroups
          - autoscaling:DescribeInstanceRefreshes
          - ec2:CreateLaunchTemplate
          - ec2:CreateLaunchTemplateVersion
          - ec2:DescribeLaunchTemplates
          - ec2:DescribeLaunchTemplateVersions
          - ec2:DeleteLaunchTemplate
          - ec2:DeleteLaunchTemplateVersions
          Effect: Allow
          Resource:
          - '*'
        - Action:
          - autoscaling:CreateAutoScalingGroup
          - autoscaling:UpdateAutoScalingGroup
          - autoscaling:CreateOrUpdateTags
          - autoscaling:StartInstanceRefresh
          - autoscaling:DeleteAutoScalingGroup
          - autoscaling:DeleteTags
          Effect: Allow
          Resource:
          - arn:*:autoscaling:*:*:autoScalingGroup:*:autoScalingGroupName/*
        - Action:
          - iam:CreateServiceLinkedRole
          Condition:
            StringLike:
              iam:AWSServiceName: autoscaling.amazonaws.com
          Effect: Allow
          Resource:
          - arn:*:iam::*:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling
        - Action:
          - iam:CreateServiceLinkedRole
          Condition:
            StringLike:
              iam:AWSServiceName: elasticloadbalancing.amazonaws.com
          Effect: Allow
          Resource:
          - arn:*:iam::*:role/aws-service-role/elasticloadbalancing.amazonaws.com/AWSServiceRoleForElasticLoadBalancing
        - Action:
          - iam:CreateServiceLinkedRole
          Condition:
            StringLike:
              iam:AWSServiceName: spot.amazonaws.com
          Effect: Allow
          Resource:
          - arn:*:iam::*:role/aws-service-role/spot.amazonaws.com/AWSServiceRoleForEC2Spot
        - Action:
          - iam:PassRole
          Effect: Allow
          Resource:
          - arn:*:iam::*:role/*.cluster-api-provider-aws.sigs.k8s.io
        - Action:
          - secretsmanager:CreateSecret
          - secretsmanager:DeleteSecret
          - secretsmanager:TagResource
          Effect: Allow
          Resource:
          - arn:*:secretsmanager:*:*:secret:aws.cluster.x-k8s.io/*
        Version: 2012-10-17
      Roles:
      - Ref: AWSIAMRoleControlPlane
    Type: AWS::IAM::ManagedPolicy
  AWSIAMRoleControlPlane:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action:
          - sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
            - ec2.amazonaws.com
        Version: 2012-10-17
      RoleName: control-plane.cluster-api-provider-aws.sigs.k8s.io
    Type: AWS::IAM::Role
  AWSIAMRoleNodes:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
        - Action:
          - sts:AssumeRole
          Effect: Allow
          Principal:
            Service:
            - ec2.amazonaws.com
        Version: 2012-10-17
      RoleName: nodes.cluster-api-provider-aws.sigs.k8s.io
    Type: AWS::IAM::Role

To create the resources in the cloudformation stack copy the contents above into a file replacing MYFILENAME.yaml and MYSTACKNAME with the intended values. Then execute the following command:

CODE

 aws cloudformation create-stack --template-body=file://MYFILENAME.yaml --stack-name=MYSTACKNAME --capabilities  CAPABILITY_NAMED_IAM

If your organization uses encrypted AMIs, then you will need to add additional permissions to the control plane policy control-plane.cluster-api-provider-aws.sigs.k8s.io to allow access to the Amazon Key Management Services. The code snippet shows how to add a particular key ARN that is used to encrypt and decrypt AMIs.

CODE

---
Action:
- kms:CreateGrant
- kms:DescribeKey
- kms:Encrypt
- kms:Decrypt
- kms:ReEncrypt*
- kms:GenerateDataKey*
Resource:
- arn:aws:kms:us-west-2:111122223333:key/key-arn
Effect: Allow

EKS IAM Policies and Instance Profiles that govern who has access to the cluster are found here: EKS Cluster IAM Permissions and Roles. If attaching an EKS cluster, that CloudStack Formation will also need to be run to create the ARN of the bootstrapper role.

Next Steps:

AWS Cluster Creation Choices