streams-manifests/kafka-resources

Kustomize and Kubernetes Commands

Generate and Apply Manifests

To build a Kubernetes manifest using Kustomize and apply it to the kafka namespace:

kustomize build overlay/test/ > manifest.yaml | kubectl kustomize overlay/test/ > manifest.yaml
kubectl -n kafka apply -f manifest.yaml

Viewing the Kustomize Directory

You can view the configuration for a Kustomize directory as follows:

kubectl kustomize argo-cd-work/base/kafka-core

---

Kustomize Documentation

For detailed documentation on Kustomization, visit: Kustomize Documentation

---

Applying Overlays

To apply a specific overlay (e.g., dev), first build the configuration and apply it to the kafka namespace:

kustomize build overlay/dev/ > dev.yml
kubectl -n kafka apply -f dev.yml

---

Preparing for Kafka Connect

Before setting up Kafka Connect, ensure the following steps are completed:

1. Create a Docker Image
   Build and push the Docker image required for Kafka Connect.

2. Create Docker Secrets for Authentication
   Generate secrets for S3 login or other credentials.

3. Create Platform and S3 Credentials
   Create the following Kubernetes secrets for authentication:

   • docker-platform-creds
   • s3-minio-creds

   Verify the secrets are present in the kafka namespace:

   kubectl -n kafka get secrets
   

   Example output:

   NAME                    TYPE                              DATA   AGE
   docker-platform-creds   kubernetes.io/dockerconfigjson    1     15d
   s3-minio-creds          Opaque                            2     13d