Manage cluster with GitOps

Use Flux to manage clusters with GitOps

GitOps Support (optional)

EKS Anywhere supports a GitOps workflow for the management of your cluster.

When you create a cluster with GitOps enabled, EKS Anywhere will automatically commit your cluster configuration to the provided GitHub repository and install a GitOps toolkit on your cluster which watches that committed configuration file. You can then manage the scale of the cluster by making changes to the version controlled cluster configuration file and committing the changes. Once a change is detected by the GitOps controller running in your cluster, the scale of the cluster will be adjusted to match the committed configuration file.

If you’d like to learn more about GitOps and the associated best practices, check out this introduction from Weaveworks .

NOTE: Installing a GitOps controller needs to be done during cluster creation. In the event that GitOps installation fails, EKS Anywhere cluster creation will continue.

Supported Cluster Properties

Currently, you can manage a subset of cluster properties with GitOps:

Cluster:

  • Cluster.workerNodeGroupConfigurations[0].count
  • Cluster.workerNodeGroupConfigurations[0].machineGroupRef.name

Worker Nodes:

  • VSphereMachineConfig.datastore
  • VSphereMachineConfig.diskGiB
  • VSphereMachineConfig.folder
  • VSphereMachineConfig.memoryMiB
  • VSphereMachineConfig.numCPUs
  • VSphereMachineConfig.resourcePool
  • VSphereMachineConfig.template

Any other changes to the cluster configuration in the git repository will be ignored. If an immutable immutable field is changed in Git repsitory, there are two ways to find the error message:

  1. If a notification webhook is set up, check the error message in notification channel.
  2. Check the Flux Kustomization Controller log: kubectl logs -f -n flux-system kustomize-controller-****** for error message containing text similar to Invalid value: 1: field is immutable

Getting Started with EKS Anywhere GitOps

In order to use GitOps to manage cluster scaling, you need a couple of things:

Create a GitHub Personal Access Token

Create a Personal Access Token (PAT) to access your provided github repository. It must be scoped for all repo permissions.

NOTE: GitOps configuration only works with hosted github.com and will not work on self hosted GitHub Enterprise instances.

This PAT should have at least the following permissions:

GitHub PAT permissions

NOTE: The PAT must belong to the owner of the repository or, if using an organization as the owner, the creator of the PAT must have repo permission in that organization.

You need to set your PAT as the environment variable $EKSA_GITHUB_TOKEN to use it during cluster creation:

export EKSA_GITHUB_TOKEN=ghp_MyValidPersonalAccessTokenWithRepoPermissions

Create GitOps configuration repo

If you have an existing repo you can set that as your repository name in the configuration. If you specify a repo in your GitOpsConfig which does not exist EKS Anywhere will create it for you. If you would like to create a new repo you can click here to create a new repo.

If your repository contains multiple cluster specification files, store them in subfolders and specify the configuration path in your cluster specification.

Example repository structure:

clusters
├── dev
│   └── eksa-system
│       └── eksa-cluster.yaml
├── prod
│   └── eksa-system
│       └── eksa-cluster.yaml
└── stage
    └── eksa-system
        └── eksa-cluster.yaml

Example GitOps cluster configuration

apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
  name: mynewgitopscluster
spec:
... # collapsed cluster spec fields
# Below added for gitops support
  gitOpsRef:
    kind: GitOpsConfig
    name: my-cluster-name
---
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: GitOpsConfig
metadata:
  name: my-cluster-name
spec:
  flux:
    github:
      personal: true
      repository: mygithubrepository
      owner: mygithubusername

Create a GitOps enabled cluster

Generate your cluster configuration and add the GitOps configuration. For a full spec reference see the Cluster Spec reference .

NOTE: After your cluster is created the cluster configuration will automatically be commited to your git repo.

  1. Create an EKS Anywhere cluster with GitOps enabled.

    CLUSTER_NAME=gitops
    eksctl anywhere create cluster -f ${CLUSTER_NAME}.yaml
    

Test GitOps controller

After your cluster is created you can test the GitOps controller by modifying the cluster specification.

  1. Clone your git repo and modify the cluster specification. The default path for the cluster file is:

    clusters/$CLUSTER_NAME/eksa-system/eksa-cluster.yaml
    
  2. Modify the workerNodeGroupsConfigurations[0].count field with your desired changes.

  3. Commit the file to your git repository

    git add eksa-cluster.yaml
    git commit -m 'Scaling nodes for test'
    git push origin main
    
  4. The flux controller will automatically make the required changes.

    If you updated your node count you can use this command to see the current node state.

    kubectl get nodes