Flux Gitops in Gitlab

GitOps is mode of operation where you keep Kubernetes manifests in a repo as source of truth and you use some way to sync that to your cluster. There are several ways and tools you can use, ie. ArgoCD or Flux.

Gitlab is using Flux by default. I believe it is more lightweight than ArgoCD and requires les resources in your cluster. This is why it is more suitable for small clusters like k3s or microk8s. You may want to use it if you do not need fancy UI. If you have big team of developers and they do not want to invest any effort to understand k8s logic, use ArgoCD.

Flux will install a few small PODs into your cluster (flux-system namespace). These POds will connect to your Gitlab repo, check for changes, apply kustomizatins or update Helm charts. You can run only PODs you need. Minimal setup will use just 2 PODs: source-controller and kustomization-controller.

Install and Bootstrap

You need to allow Flux to access your Gitlab repo, so you should create and save PAT. Clone your repo and run the following commands from it because Flux will create and commit some files, ie: clusters//flux-system.

export GITLAB_TOKEN=<your-token>
flux bootstrap gitlab --deploy-token-auth --owner=<Gitlab Repo Owner> --repository=<Repo Name>  --branch=main --path=clusters/<name>

This will run the PODs in your cluster, connect them to your Repo and add config files into your repo. These files are used to configure Flux. You can control what components are running by editing clusters//flux-system/gotk-components.yaml. For example, you can scale to 0 helm-controller.

Repo Organization

You should make some kind of hierarchy in your repo to organize files related to app and separate from flux system. Here sample:

├── clusters // flux-system
│   └── dev
│       ├── flux-system
│       │   ├── gotk-components.yaml
│       │   ├── gotk-sync.yaml
│       │   └── kustomization.yaml
│       ├── notes-front
│           └── kustomization.yaml
│      
└── notes-front // your app
    ├── docker-registry.yaml
    ├── kustomization.yaml
    ├── notes-front-deployment.yaml
    ├── notes-front-ingress.yaml
    ├── notes-front-namespace.yaml
    └── notes-front-service.yaml

Sample App

I have dev and prod Kubernetes clusters in different clouds. Both are using different DNS domains, so my app must use different ENV variables for configuration. I’m using Kustomize to overwrite the Deployments and set correct ENV. App is running as single POD in notes-front namespace. There is service and Ingress used to expose the app on the http://dev-domain or http://prod-domain. In real life, I’m using cert-manager and HTTPs.

Kustomize

clusters/dev/notes-front/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
metadata:
  name: notes-front
resources:
  - ../../../notes-front

patches:
  - target:
      kind: Deployment
      name: notes-front-app
    patch: |-
      - op: add
        path: /spec/template/spec/containers/0/env/-
        value:
          name: ENV
          value: dev
      - op: add
        path: /spec/template/spec/containers/0/env/-
        value:
          name: GITOPS
          value: SOME_VALUE

  # Change Ingress class for K3s
  - target:
      kind: Ingress
      name: notes-front-ingress
    patch: |-
      - op: add
        path: /spec/ingressClassName
        value: traefik

notes-front/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - notes-front-namespace.yaml
  - docker-registry.yaml
  - notes-front-deployment.yaml
  - notes-front-service.yaml
  - notes-front-ingress.yaml

Sync and Troubleshoot

After you commit and push to your repo, Flux will sync after around 1 minute and try to apply your manifests. You can follow up using the commands:

# Check the events
flux logs

# Get info about kustomizations
kubectl get kustomizations -n flux-system

# Reply
NAME          AGE     READY   STATUS
flux-system   3d23h   True    Applied revision: main@sha1:5cf7f78c6f8d5629767566279dc618b86a116ac4