GitOps Unlocked: Building a Declarative Delivery Pipeline for Kubernetes

Kubernetes has become the default operating system for cloud-native applications, but with great power comes great complexity. Deploying directly via kubectl, ad-hoc scripts, and imperatively updated Helm releases can quickly turn a cluster into a snowflake environment. GitOps offers a different philosophy: use Git as the single source of truth, describe the entire desired cluster state in declarative manifests, and let an automated sync process enforce that state. This article explains why GitOps is a natural fit for Kubernetes and how you can build a robust delivery pipeline around it.

What Is GitOps?

GitOps is a set of practices and tools that applies software engineering workflows to infrastructure and application operations. In a GitOps model, a Git repository contains the complete desired state of a system. That state is declarative: it describes what should exist, not how to make it exist. A reconciliation loop continuously compares the desired state in Git to the actual state in the cluster. If they diverge, the loop takes action to converge them.

Modern GitOps follows a few core principles:

• Declarative configuration: The system is described by manifests, not imperative commands.
• Versioned and immutable: Every change is a commit, which creates an immutable, audited history.
• Automated delivery: No manual kubectl or dashboard clicks are needed to deploy.
• Continuous reconciliation: A controller continuously works to make the live environment match Git.
• Pull-based deployment: The cluster agent pulls changes from Git, rather than CI pushing to the cluster.

This approach is often summarized by the slogan: Your infrastructure is a Git repository, and Git is your control plane.

Why GitOps Matters for Kubernetes

Kubernetes is natively declarative. A Deployment object specifies a desired replica count, and the controller ensures that many replicas exist. Yet, many teams still treat Kubernetes as a scripting platform: they run kubectl apply, delete pods, and patch services on a whim. That creates unmanageable drift and no clear source of truth.

GitOps addresses this in several important ways:

• No configuration drift: If someone changes something in the cluster manually, GitOps will revert it during the next reconciliation.
• Fast and safe rollbacks: Reverting to a known good state is just adding a commit or resuming a sync to a previous revision.
• Audit trail: Every deployment and every change is linked to a commit, a Pull Request, and a human reviewer.
• Multi-cluster consistency: The same Git repository can be applied to multiple clusters, producing reproducible environments.
• Developer autonomy: Developers can self-service deploy by opening a Pull Request, without needing cluster credentials.

Core Components of a GitOps Pipeline

A typical GitOps pipeline consists of four parts:

• Source repository: Contains Kubernetes manifests, Helm charts, Kustomize overlays, or other declarative configs.
• CI pipeline: Builds artifacts and runs tests. It does not need to deploy directly to the cluster.
• GitOps operator: A controller in the cluster that watches Git and applies the desired state. Popular tools include Argo CD and Flux.
• Destination cluster: The Kubernetes environment that is continuously reconciled with the source repository.

In a mature GitOps workflow, the CI pipeline produces a container image and then updates the Kubernetes manifest in the Git repo to point to the new image. The GitOps operator detects the change in Git, pulls it, and applies the manifest to the cluster. This separation is powerful: CI is responsible for build, and CD is responsible for delivery via pull.

Choosing a GitOps Tool: Argo CD vs. Flux

Argo CD and Flux are the two leading open-source GitOps operators. Both are excellent, but they have different strengths.

Argo CD

Argo CD is built around the concept of an Application resource. You define a Git repository, a path, and a cluster destination. Argo CD watches the repository, compares the live state with the desired state, and displays the difference in its UI and CLI. It supports sync hooks, health checks, and automatic sync with prune. Argo CD is widely loved for its user-friendly web UI and tight integration with the broader Argo ecosystem, including Argo Workflows and Argo Rollouts.

Flux

Flux is the CNCF project that originally popularized GitOps. The current Flux v2 is a collection of controllers: source controller, kustomize controller, helm controller, and notification controller. Its design is more modular and focuses on reconciling at a lower level. Flux also supports image automation, which can update manifests automatically when a new image is pushed. It is an excellent choice for teams that want fine-grained control and a strong Kubernetes-native architecture.

Both tools support Kubernetes Secrets management via SOPS or Sealed Secrets, but they do not store raw secrets in a readable file unless you explicitly choose that pattern. Teams should evaluate their existing workflows and choose the tool that feels most natural.

Building a GitOps Pipeline Step by Step

Let us walk through a realistic GitOps implementation using Argo CD. This is not a vendor endorsement; the same pattern works with Flux.

1. Create a Git Repository

Create a repository for your deployment manifests, for example myapp-config. Inside it, you might have this structure:

config/
  base/
    deployment.yaml
    service.yaml
    namespace.yaml
  overlays/
    development/
      kustomization.yaml
    production/
      kustomization.yaml

Using Kustomize or Helm allows the same base manifest to be reused across environments. This repository is the source of truth, so protect it with branch rules, code owners, and signed commits if necessary.

2. Write Declarative Manifests

Here is a minimal example of a Deployment manifest. Notice that there is no deployment command, only a desired state:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  namespace: myapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
        - name: myapp
          image: myregistry/myapp:1.2.3
          ports:
            - containerPort: 8080

The image tag is pinned. It is a best practice to use immutable tags such as commit SHAs or semantic versions that you never overwrite.

3. Bootstrap the Cluster with Argo CD

Install Argo CD into the cluster using its manifests or Helm chart. Then define an Application resource that points to your repository:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp-development
  namespace: argocd
spec:
  destination:
    namespace: myapp-development
    server: https://kubernetes.default.svc
  project: default
  source:
    repoURL: https://github.com/acme/myapp-config
    targetRevision: main
    path: config/overlays/development
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

The automated block enables automatic sync. prune: true deletes resources that are removed from Git. selfHeal: true reverts manual modifications. CreateNamespace ensures the target namespace is created if it does not already exist.

4. Configure Rollout Strategy

You should also define a rollout strategy in the Deployment. The default RollingUpdate strategy is good for many apps, but production systems may want a safer canary or blue-green approach. Argo Rollouts can canary a new version while analyzing metrics from Prometheus or Datadog. This integrates with GitOps by storing the Rollout manifest in Git and using Argo CD to sync it.

5. Optimize the CI Pipeline

A GitOps CI pipeline does not push directly to Kubernetes. Instead, it should do the following:

• Run unit tests and static analysis.
• Build a container image.
• Scan the image for vulnerabilities.
• Push the image to a registry with an immutable tag.
• Update the image tag in the Git repository.

Here is a simplified pipeline structure in pseudocode:

test:
  - run unit tests
  - run integration tests
build:
  - docker build -t myregistry/myapp:$CI_COMMIT_SHA
  - docker push myregistry/myapp:$CI_COMMIT_SHA
update-config:
  - sed the image tag in the manifests
  - git commit and push to main

If the Git repository is protected, the CI service will need a token or SSH key to push changes. Teams often use a bot account and require a Pull Request instead of direct commits. This triggers an automated config preview, which is another opportunity for validation.

Managing Secrets in GitOps

Putting secrets in Git is a common concern. The answer is not to abandon GitOps, but to encrypt secrets before committing them. Two widely used tools are:

• Sealed Secrets: You create a SealedSecret object that is encrypted with a public key. The controller in the cluster decrypts it into a normal Secret. The sealed data is safe to commit to Git.
• SOPS: SOPS encrypts values in YAML, JSON, or env files. It supports AWS KMS, GCP KMS, Azure Key Vault, and age. Argo CD and Flux can decrypt SOPS-encrypted files before applying them.

You can also use external secret management services like External Secrets Operator to pull secrets from Vault or AWS Secrets Manager. The Git repository stores only a reference to the secret, not the secret itself. This is a robust pattern for environments with strict compliance requirements.

Progressive Delivery with GitOps

GitOps shines when combined with progressive delivery. The goal is to release changes gradually while monitoring impact. Tools such as Argo Rollouts and Flagger automate the analysis and promotion of new versions.

Imagine a new image tag appears in Git. The GitOps operator syncs the Rollout resource, which creates a canary version. The controller then shifts a small percentage of traffic to the canary. It checks error rates, latency, and other metrics. If everything looks good, it gradually increases the traffic. If not, it automatically rolls back and marks the deployment as failed. This entire process is declarative and auditable through Git history.

Image Automation

One of the more advanced GitOps features is image automation. Flux can watch a container registry and update manifests when a new image is tagged. This removes the need for a CI system to commit manifest changes, although many teams prefer to keep that step explicit. Image automation is a trade-off: it makes the pipeline faster, but it also means that Git is updated by an automated process rather than a human-initiated commit. If you use image automation, ensure you have strict tag naming policies and sufficient automated tests before production.

Multi-Environment and Multi-Cluster GitOps

A single state repository can support multiple clusters by using the same source path with different override values. For example, the development cluster watches the development overlay, and the production cluster watches the production overlay. This pattern lets you promote a release by merging a change in the base or overlay.

However, there are practical limits. If you have dozens of clusters, a monorepo with all cluster configs can become unwieldy. In that case, you may want separate repositories per cluster or per domain. The key is to maintain a clear promotion path and avoid duplicating configuration that can drift.

Measuring Success: Metrics and Outcomes

When you adopt GitOps, you should track a few operational signals:

• Deployment frequency: How often can you release? GitOps should increase this.
• Change failure rate: How often does a deployment require a hotfix or rollback?
• Mean time to recovery: How long does it take to restore service after an incident?
• Drift frequency: How often does the live cluster differ from Git?
• Time to rollback: How long does it take to revert to a previous known-good state?

Automated reconciliation should make manual drift rare. If you find frequent drift, inspect your controllers and external mutation paths. Something outside Git is modifying the cluster, and GitOps should be alerting you to it.

Common Pitfalls and How to Avoid Them

Treating GitOps as a Simple kubectl Replacement

A common mistake is to commit a single Deployment YAML and call it GitOps. While that is a start, GitOps also requires a reconciliation operator, a review process, and a strategy for handling secrets and rollouts. Without those, you are still doing manual deployments with a versioned config file.

Ignoring Non-Kubernetes Resources

Many systems depend on resources outside the cluster, such as managed databases, DNS records, or service mesh configuration. GitOps tools typically only manage Kubernetes resources. To extend GitOps to external resources, you need custom controllers or tools like Crossplane. Otherwise, your actual state can still drift from what you think you have.

Overusing Auto-Sync in Day-2 Operations

Automatic sync is great but can be dangerous if the manifests are invalid or if they destroy data. Use infrastructure as code tests, pre-sync validation, and dry-run in CI. You should also enable automated self-healing, but combine it with proper RBAC so that only approved roles can mutate the cluster or the Git repo.

Secret Management Complexity

Encrypting secrets is mandatory, but it adds complexity. Make sure the encryption keys are accessible only to the GitOps controller. Define a clear rotation policy and test restoring secrets from backup. Do not assume that a sealed secret is automatically safe forever; long-lived encryption keys are a risk.

Monorepo vs. Many Small Repositories

Both approaches work. A monorepo simplifies change coordination and cross-service refactoring. Many small repos make it easier to isolate failure domains and permissions. Choose a model based on your team size and deployment boundary. In large organizations, a config repo per team plus a central platform repo often works well.

GitOps and Platform Engineering

GitOps is a natural enabler for platform engineering. By exposing environment promotion as Pull Requests, platform teams can provide self-service deployment while keeping guardrails. An internal developer portal can generate application scaffolding and open a PR to the config repository. The GitOps operator does the rest. This reduces cognitive load and makes infrastructure changes reviewable, reversible, and repeatable.

Security and Compliance Advantages

Git history provides an immutable audit log. You can see who changed what, when, and why. Signed commits and branch protection prevent unauthorized changes. If a cluster is compromised, the GitOps operator can help re-establish the desired state, but only if Git itself is secure. Protect your Git environment with strong authentication, branch protection, and mandatory code review for production paths.

For compliance frameworks like SOC 2 or ISO 27001, GitOps helps demonstrate that changes are documented and reviewed. Automated drift correction also reduces the chance of configuration compliance failures. However, GitOps is not a silver bullet. You still need to secure the supply chain, the container registry, and the GitOps tool itself.

Getting Started

Start small. Pick a low-risk service and move its deployment to a GitOps operator. Use a separate Git repository and a test cluster. Define a simple Argo CD Application or Flux Kustomization, then automate the sync. Practice rollbacks and observe how the system reacts to drift. Once you trust the workflow, expand to more services.

To further deepen your knowledge, explore the OpenGitOps specification and the official documentation for Argo CD or Flux. Both communities are active and provide excellent examples. Implementing GitOps is not just a tool change; it is a cultural shift toward treating infrastructure with the same rigor as application code.

Conclusion

Kubernetes gives us the abstraction to run distributed systems, but it also demands a disciplined way to manage configuration and delivery. GitOps supplies that discipline. By making Git the source of truth, using pull-based deployments, and continuously reconciling the live cluster, teams can achieve faster, safer, and more auditable software delivery. The transition takes careful planning, but the payoff in reliability and developer experience is enormous. Whether you choose Argo CD, Flux, or another operator, the principles remain the same: declarative state, Git-driven change, and automated convergence.