Credential Provider Package Configuration

• 1: Credential Provider Package with IAM Roles Anywhere
• 2: Credential Provider Package
• 3: v0.1.0

1 - Credential Provider Package with IAM Roles Anywhere

This tutorial demonstrates how to configure the credential provider package to authenticate using IAM Roles Anywhere to pull from a private AWS Elastic Container Registry (ECR).
IAM Roles Anywhere enables workloads outside of AWS to access AWS resources by using X.509 digital certificates to obtain temporary AWS credentials. A trust anchor is used to reference a certificate authority with IAM Roles Anywhere. For this use case, the Kubernetes Cluster CA can be registered and each kubelet client’s x509 cert can be used to authenticate to get temporary AWS credentials.

Prerequisites

1. For setting up the certificate authority later, you will need to obtain your cluster’s CA. This can be obtain by:

   # Assuming CLUSTER_NAME and KUBECONFIG are set:
   kubectl get secret -n eksa-system ${CLUSTER_NAME}-ca -o yaml | yq '.data."tls.crt"' | base64 -d
   

2. A role should be created to allow read access for curated packages. This role can be extended to include private registries that you would also like to pull from into your cluster. A sample policy for curated packages would be.

   {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ECRRead",
            "Effect": "Allow",
            "Action": [
                "ecr:DescribeImageScanFindings",
                "ecr:GetDownloadUrlForLayer",
                "ecr:DescribeRegistry",
                "ecr:DescribePullThroughCacheRules",
                "ecr:DescribeImageReplicationStatus",
                "ecr:ListTagsForResource",
                "ecr:ListImages",
                "ecr:BatchGetImage",
                "ecr:DescribeImages",
                "ecr:DescribeRepositories",
                "ecr:BatchCheckLayerAvailability"
            ],
            "Resource": "arn:aws:ecr:*:783794618700:repository/*"
        },
        {
            "Sid": "ECRLogin",
            "Effect": "Allow",
            "Action": [
                "ecr:GetAuthorizationToken"
            ],
            "Resource": "*"
        }
     ]
    }
   

3. Next create a trust anchor and profile. The trust anchor will be a reference to the CA certificate from step 1 and the profile should point to the role created in step 2. See here for instructions on creating the trust anchor and profile.

4. Create a secret that will be referenced by the credential-provider-package to authenticate the kubelet with ECR.

   # Set PROFILE_ARN, ROLE_ARN, and TRUST_ANCHOR_ARN obtained in previous step
   # Set AWS_REGION to region to pull images from
   # This will create a file credfile which will then be turned into a secret
   cat << EOF >> credfile
   [default]
   region = $AWS_REGION
   credential_process = aws_signing_helper credential-process --certificate /var/lib/kubelet/pki/kubelet-client-current.pem --private-key /var/lib/kubelet/pki/kubelet-client-current.pem --profile-arn $PROFILE_ARN --role-arn $ROLE_ARN --trust-anchor-arn $TRUST_ANCHOR_ARN
   EOF
   
   # Create secret, for this example the secret name aws-config is used and the package will be installed in eksa-packages
   kubectl create secret generic aws-config --from-file=config=credfile -n eksa-packages
   

5. Either edit the existing package or delete and create a new credential-provider-package that points towards the new secret. For more information on specific configuration option refer to installation guide for details]
   The example below changes the default secret name from aws-secret to newly created aws-config. It also changes the match images to pull from multiple regions as well as across multiple accounts. Make sure to change cluster-name to match your CLUSTER_NAME

   apiVersion: packages.eks.amazonaws.com/v1alpha1
   kind: Package
   metadata:
     name: my-credential-provider-package
     namespace: eksa-packages-<clusterName>
     annotations:
       "helm.sh/resource-policy": keep
       "anywhere.eks.aws.com/internal": "true"
   spec:
     packageName: credential-provider-package
     targetNamespace: eksa-packages
     config: |-
       tolerations:
         - key: "node-role.kubernetes.io/master"
           operator: "Exists"
           effect: "NoSchedule"
         - key: "node-role.kubernetes.io/control-plane"
           operator: "Exists"
           effect: "NoSchedule"
       sourceRegistry: public.ecr.aws/eks-anywhere
       credential:
         - matchImages:
           - *.dkr.ecr.*.amazonaws.com
           profile: "default"
           secretName: aws-config
           defaultCacheDuration: "5h"    
   

2 - Credential Provider Package

If you have not already done so, make sure your cluster meets the package prerequisites. Be sure to refer to the troubleshooting guide in the event of a problem.

Install

By default an instance of this package is installed with the controller to help facilitate authentication for other packages. The following are instructions in case you want to tweak the default values.

1. Generate the package configuration

   eksctl anywhere generate package credential-provider-package --cluster <cluster-name> > credential-provider-package.yaml
   

2. Add the desired configuration to credential-provider-package.yaml Please see complete configuration options for all configuration options and their default values. Example default package using IAM User Credentials installed with the controller

   apiVersion: packages.eks.amazonaws.com/v1alpha1
   kind: Package
   metadata:
     name: my-credential-provider-package
     namespace: eksa-packages-<clusterName>
     annotations:
       "helm.sh/resource-policy": keep
       "anywhere.eks.aws.com/internal": "true"
   spec:
     packageName: credential-provider-package
     targetNamespace: eksa-packages
     config: |-
       tolerations:
         - key: "node-role.kubernetes.io/master"
           operator: "Exists"
           effect: "NoSchedule"
         - key: "node-role.kubernetes.io/control-plane"
           operator: "Exists"
           effect: "NoSchedule"
       sourceRegistry: public.ecr.aws/eks-anywhere
       credential:
         - matchImages:
           - 783794618700.dkr.ecr.us-west-2.amazonaws.com
           profile: "default"
           secretName: aws-secret
           defaultCacheDuration: "5h"
   

3. Create the secret. If you are changing the secret, see complete configuration options for the format of the secret.

4. Create the namespace (if not installing to eksa-packages). If you are overriding targetNamespace, change eksa-packages to the value of targetNamespace.

   kubectl create namespace <namespace-name-here>
   

5. Install the credential-provider-package

   eksctl anywhere create packages -f credential-provider-package.yaml
   

6. Validate the installation

   eksctl anywhere get packages --cluster <cluster-name>
   

Update

To update package configuration, update credential-provider-package.yaml file and run the following command:

eksctl anywhere apply package -f credential-provider-package.yaml

Upgrade

Credential-Provider-Package will automatically be upgraded when a new bundle is activated.

Uninstall

To uninstall credential-provider-package, simply delete the package:

eksctl anywhere delete package --cluster <cluster-name> my-credential-provider-package

3 - v0.1.0

Configuring Credential Provider Package in EKS Anywhere package spec

Example

The following is the sample configuration for the credential provider package that is installed by default with the package controller. Please refer to Credential Provider Package with IAM Roles Anywhere.

apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  name: ecr-credential-provider-package
  namespace: eksa-packages-<clusterName>
  annotations:
    "helm.sh/resource-policy": keep
    "anywhere.eks.aws.com/internal": "true"
spec:
  packageName: credential-provider-package
  targetNamespace: eksa-packages
  config: |-
    tolerations:
      - key: "node-role.kubernetes.io/master"
        operator: "Exists"
        effect: "NoSchedule"
      - key: "node-role.kubernetes.io/control-plane"
        operator: "Exists"
        effect: "NoSchedule"
    sourceRegistry: public.ecr.aws/eks-anywhere
    credential:
      - matchImages:
        - 783794618700.dkr.ecr.us-west-2.amazonaws.com
        profile: "default"
        secretName: aws-secret
        defaultCacheDuration: "5h"

In this example, the credential provider will use the secret provided in aws-secret (created automatically on cluster creation) to authenticate to the repository from which curated package images are pulled. Tolerations were also added so that the control plane nodes would also be configured with authentication.

The secret can exist in two forms: either a base64 encoding of a credential config or individual keys for fields.

Example credential

[default]
aws_access_key_id=EXAMPLE_ACCESS_KEY
aws_secret_access_key=EXAMPLE_SECRET_KEY
region=us-west-2

Example secret with separate keys

apiVersion: v1
kind: Secret
metadata:
  name: aws-secret
  namespace: eksa-packages
data:
  AWS_ACCESS_KEY_ID: "QUtJQUlPU0ZPRE5ON0VYQU1QTEUK"
  AWS_SECRET_ACCESS_KEY: "d0phbHJYVXRuRkVNSS9LN01ERU5HL2JQeFJmaUNZRVhBTVBMRUtFWQo="
  REGION: dXMtd2VzdC0yCg==

Example secret in config form

apiVersion: v1
kind: Secret
metadata:
  name: aws-secret
  namespace: eksa-packages
data:
  config: W2RlZmF1bHRdCmF3c19hY2Nlc3Nfa2V5X2lkPUFLSUFJT1NGT0ROTjdFWEFNUExFCmF3c19zZWNyZXRfYWNjZXNzX2tleT13SmFsclhVdG5GRU1JL0s3TURFTkcvYlB4UmZpQ1lFWEFNUExFS0VZCnJlZ2lvbj11cy13ZXN0LTI=
type: Opaque

Configurable parameters and default values under spec.config

credential: 
  - secretName: “aws-secret”
    matchImages: []
    defaultCacheDuration: “1h”
    profile: “default