Overview
In the previous post, I set up Harbor container registry, Argo Events, and Argo Workflows as the foundation of the IDP. This post covers integrating those components with ArgoCD and designing Helm chart-based project templates to turn them into an Internal Developer Platform (IDP) that can deploy projects from a single YAML file.
What Is an Internal Developer Platform?
What is an Internal Developer Platform (IDP)?
An Internal Developer Platform is a system that provides developers with an abstracted self-service interface to deploy and operate applications without directly configuring infrastructure and deployment pipelines. As a core deliverable of platform engineering, it aims to improve developer experience and reduce operational burden through standardized deployment processes.
Traditional CI/CD pipelines usually had to be set up separately for each project. What I wanted instead was a template-driven internal platform that could provision most of the surrounding infrastructure from a small configuration file. The version I built in this post worked like this:
- A developer pushes code to a Git repository.
- A GitHub webhook sends an event to Argo Events’ EventSource.
- Argo Events’ Sensor filters the event and triggers an Argo Workflow.
- Argo Workflows builds the code and pushes the container image to Harbor.
- When the workflow completes, it calls the GitHub API to update the project configuration file.
- ArgoCD detects the changed configuration file and deploys the application with the new image.
Project Template Design
I designed a Helm chart-based project template that I could reuse across multiple projects. The idea was to reduce a new project to a small YAML file while keeping the surrounding CI/CD wiring shared.
Project Template Requirements
For this homelab setup, I wanted the template to provide the following features:
- Automated CI/CD Pipeline: Automatically builds and deploys when code changes in a GitHub repository.
- Declarative Resource Management: Define applications, databases, and network settings in YAML files.
- Secrets Management Integration: Safely manage passwords, API keys, and other secrets through Vault integration.
- Multi-Application Support: Manage multiple applications and databases within a single project.
Git Repository Structure
I organized the GitOps repository with the following structure:
projects-gitops/
├── .github/workflows/
│ └── update-config.yaml
├── applicationset.yaml
├── chart/
│ ├── Chart.yaml
│ └── templates/
│ ├── app/
│ │ ├── ci/
│ │ │ ├── eventbus.yaml
│ │ │ ├── eventsource.yaml
│ │ │ ├── sensor.yaml
│ │ │ └── workflow-template.yaml
│ │ ├── deployment.yaml
│ │ ├── service.yaml
│ │ └── ingressroute.yaml
│ └── db/
│ ├── statefulset.yaml
│ └── service.yaml
└── projects/
├── project-a.yaml
├── project-b.yaml
└── ...
In this structure, the chart/ directory contains the Helm chart shared by all projects, and the projects/ directory contains the configuration files for each project. Deploying a new project was as simple as adding a YAML file to the projects/ directory.
ApplicationSet Configuration
What is ApplicationSet?
ApplicationSet is an ArgoCD feature that uses templates and generators to automatically create and manage multiple Applications. It can dynamically create Applications based on file lists in Git repositories, directory structures, cluster lists, and more, enabling efficient management of large-scale multi-project environments.
The applicationset.yaml file looked like this:
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: projects-apps
namespace: argocd
spec:
goTemplate: true
goTemplateOptions: ["missingkey=error"]
generators:
- git:
repoURL: https://github.com/injunweb/projects-gitops.git
revision: HEAD
files:
- path: "projects/*.yaml"
template:
metadata:
name: '{{.path.filenameNormalized | trimSuffix ".yaml"}}'
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/injunweb/projects-gitops.git
targetRevision: HEAD
path: chart
plugin:
name: argocd-vault-plugin-helm
env:
- name: HELM_ARGS
value: >-
-f ../projects/{{.path.filename}}
--set project={{.path.filenameNormalized | trimSuffix ".yaml"}}
destination:
server: https://kubernetes.default.svc
namespace: '{{.path.filenameNormalized | trimSuffix ".yaml"}}'
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
This ApplicationSet uses a Git generator configured with the projects/*.yaml files pattern and automatically creates an ArgoCD Application for each file. The filename, minus the .yaml extension, becomes the project name and namespace. Secrets stored in Vault are then injected through the ArgoCD Vault Plugin.
Project Configuration File Structure
Each project in this setup is described with a YAML file that looks like this:
applications:
- name: api
git:
type: github
owner: myorg
repo: my-api
branch: main
hash: ~
port: 8080
domains:
- api.example.com
- name: frontend
git:
type: github
owner: myorg
repo: my-frontend
branch: main
hash: ~
port: 80
domains:
- www.example.com
- example.com
databases:
- name: mysql
type: mysql
version: "8.0"
port: 3306
size: 5Gi
- name: redis
type: redis
version: "7.0"
port: 6379
size: 1Gi
Some of the more important fields in this configuration file are:
- applications[].git.hash: The Git commit hash that the CI pipeline will build and deploy. It is initially empty and automatically updated when a build succeeds. Deployments are only created when this value exists.
- applications[].domains: A list of domains for accessing the application. A Traefik IngressRoute is created for each domain.
- databases[]: A list of databases to use in the project, supporting MySQL, PostgreSQL, Redis, and MongoDB.
CI Pipeline Templates
For the CI side, I used Argo Events and Argo Workflows together and kept the pipeline pieces as reusable Helm templates.
EventBus Template
A template that creates an independent event bus for each project:
apiVersion: argoproj.io/v1alpha1
kind: EventBus
metadata:
name: {{ $.Values.project }}-ci-eventbus
namespace: {{ $.Values.project }}
spec:
nats:
native:
replicas: 3
auth: none
antiAffinity: false
This EventBus runs 3 NATS replicas for high availability and gives each project its own event transport layer.
EventSource Template
An EventSource template that receives GitHub webhooks:
{{- range $app := .Values.applications }}
---
apiVersion: argoproj.io/v1alpha1
kind: EventSource
metadata:
name: {{ $.Values.project }}-{{ $app.name }}-github-eventsource
namespace: {{ $.Values.project }}
spec:
eventBusName: {{ $.Values.project }}-ci-eventbus
template:
serviceAccountName: {{ $.Values.project }}-ci-workflow-sa
service:
ports:
- port: 12000
targetPort: 12000
name: webhook
github:
{{ $.Values.project }}-{{ $app.name }}-github-trigger:
repositories:
- owner: {{ $app.git.owner }}
names:
- {{ $app.git.repo }}
webhook:
endpoint: /{{ $.Values.project }}-{{ $app.name }}
port: "12000"
method: POST
url: https://webhook.injunweb.com
events:
- push
apiToken:
name: {{ $.Values.project }}-github-access-secret
key: token
insecure: false
active: true
contentType: json
{{- end }}
This template creates an EventSource for each application defined in the project configuration and listens for push events from its GitHub repository. The webhook.url value is the externally accessible endpoint that GitHub sends events to.
Sensor Template
A Sensor template that filters events and triggers workflows:
{{- range $app := .Values.applications }}
---
apiVersion: argoproj.io/v1alpha1
kind: Sensor
metadata:
name: {{ $.Values.project }}-{{ $app.name }}-github-workflow-sensor
namespace: {{ $.Values.project }}
spec:
eventBusName: {{ $.Values.project }}-ci-eventbus
template:
serviceAccountName: {{ $.Values.project }}-ci-workflow-sa
dependencies:
- name: github-dep
eventSourceName: {{ $.Values.project }}-{{ $app.name }}-github-eventsource
eventName: {{ $.Values.project }}-{{ $app.name }}-github-trigger
filters:
data:
- path: body.ref
type: string
comparator: "="
value:
- "refs/heads/{{ $app.git.branch }}"
triggers:
- template:
name: workflow-trigger
k8s:
operation: create
source:
resource:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: {{ $.Values.project }}-{{ $app.name }}-build-workflow-
spec:
arguments:
parameters:
- name: git_sha
workflowTemplateRef:
name: {{ $.Values.project }}-{{ $app.name }}-build-workflow-template
parameters:
- src:
dependencyName: github-dep
dataKey: body.after
dest: spec.arguments.parameters.0.value
retryStrategy:
steps: 3
{{- end }}
This Sensor filters only push events to specific branches (e.g., main, develop) in the filters.data section, and when matching events occur, it creates a Workflow by referencing the WorkflowTemplate. The body.after value (the commit hash after the push) is passed as a workflow parameter.
WorkflowTemplate
A WorkflowTemplate that defines the build step and updates the project config afterward:
{{- range $app := .Values.applications }}
---
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: {{ $.Values.project }}-{{ $app.name }}-build-workflow-template
namespace: {{ $.Values.project }}
spec:
serviceAccountName: {{ $.Values.project }}-ci-workflow-sa
entrypoint: build
arguments:
parameters:
- name: git_sha
description: "Git commit hash"
volumes:
- name: docker-config
secret:
secretName: registry-secret
items:
- key: .dockerconfigjson
path: config.json
templates:
- name: build
dag:
tasks:
- name: build
template: build-container
arguments:
parameters:
- name: sha
value: "{{`{{workflow.parameters.git_sha}}`}}"
- name: update-config
template: update-config
dependencies: [build]
arguments:
parameters:
- name: sha
value: "{{`{{workflow.parameters.git_sha}}`}}"
- name: build-container
inputs:
parameters:
- name: sha
hostAliases:
- ip: "192.168.0.200"
hostnames:
- "harbor.injunweb.com"
container:
image: gcr.io/kaniko-project/executor:latest
args:
- "--context=git://github.com/{{ $app.git.owner }}/{{ $app.git.repo }}.git#refs/heads/{{ $app.git.branch }}#{{`{{inputs.parameters.sha}}`}}"
- "--dockerfile=Dockerfile"
- "--destination=harbor.injunweb.com/injunweb/{{ $.Values.project }}-{{ $app.name }}:{{`{{inputs.parameters.sha}}`}}"
- "--destination=harbor.injunweb.com/injunweb/{{ $.Values.project }}-{{ $app.name }}:latest"
- "--cache=true"
- "--cache-repo=harbor.injunweb.com/injunweb/cache"
env:
- name: GIT_USERNAME
valueFrom:
secretKeyRef:
name: {{ $.Values.project }}-github-access-secret
key: username
- name: GIT_PASSWORD
valueFrom:
secretKeyRef:
name: {{ $.Values.project }}-github-access-secret
key: token
volumeMounts:
- name: docker-config
mountPath: /kaniko/.docker/
- name: update-config
inputs:
parameters:
- name: sha
container:
image: curlimages/curl:latest
command: ["/bin/sh", "-c"]
args:
- |
curl -X POST https://api.github.com/repos/injunweb/projects-gitops/dispatches \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: Bearer $GITHUB_TOKEN" \
-d '{
"event_type": "config-api",
"client_payload": {
"path": "projects/{{$.Values.project}}/applications/{{$app.name}}",
"action": "apply",
"spec": {
"git": {
"hash": "'"{{`{{inputs.parameters.sha}}`}}"'"
}
}
}
}'
env:
- name: GITHUB_TOKEN
valueFrom:
secretKeyRef:
name: {{ $.Values.project }}-github-access-secret
key: token
{{- end }}
The key components of this WorkflowTemplate are:
- DAG Template: Defines two tasks,
buildandupdate-config, as a DAG with dependencies so that the configuration update only runs after a successful build. - Kaniko: A tool for building images inside containers without a Docker daemon, allowing image builds without privilege escalation. Caching is enabled to reduce build times.
- GitHub API Call: When the build succeeds, it triggers a repository_dispatch event to update the
git.hashvalue in the project configuration file.
CD Pipeline Templates
When the CI pipeline updates the project configuration file, ArgoCD picks up the change and rolls out the new image.
Deployment Template
A Deployment template for deploying applications:
{{- range $app := .Values.applications }}
{{- if $app.git.hash }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ $app.name }}-app
namespace: {{ $.Values.project }}
spec:
replicas: 1
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
selector:
matchLabels:
app: {{ $app.name }}-app
template:
metadata:
labels:
app: {{ $app.name }}-app
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- {{ $app.name }}-app
topologyKey: "kubernetes.io/hostname"
terminationGracePeriodSeconds: 120
containers:
- name: {{ $app.name }}-app
image: harbor.injunweb.com/injunweb/{{ $.Values.project }}-{{ $app.name }}:{{ $app.git.hash }}
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 10"]
ports:
- containerPort: {{ $app.port }}
readinessProbe:
tcpSocket:
port: {{ $app.port }}
initialDelaySeconds: 20
periodSeconds: 10
successThreshold: 3
envFrom:
- secretRef:
name: {{ $.Values.project }}-{{ $app.name }}-secret
optional: true
imagePullSecrets:
- name: registry-secret
{{- end }}
{{- end }}
The key point of this template is the {{- if $app.git.hash }} condition, which ensures that the Deployment is only created when the git.hash value is set. This guarantees that deployment only occurs after the CI pipeline has completed successfully.
The main features of the Deployment template are:
- Rolling Update: The
maxSurge: 1,maxUnavailable: 0settings implement zero-downtime deployment. - Pod Anti-Affinity: Distributes Pods of the same application across different nodes to improve availability.
- Graceful Shutdown: The
preStophook and 120-second termination grace period allow existing connections to complete normally.
IngressRoute Template
An IngressRoute template that provides external access to applications:
{{- range $app := .Values.applications }}
{{- if $app.git.hash }}
{{- range $domain := $app.domains }}
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: {{ $.Values.project }}-{{ $app.name }}-{{ $domain | replace "." "-" }}-route
namespace: {{ $.Values.project }}
spec:
entryPoints:
- web
- websecure
routes:
- match: Host(`{{ $domain }}`)
kind: Rule
services:
- name: {{ $app.name }}
port: {{ $app.port }}
{{- end }}
{{- end }}
{{- end }}
This template creates a Traefik IngressRoute for each domain defined in the project configuration, using the web and websecure entry points for HTTP and HTTPS traffic.
Database StatefulSet Template
A StatefulSet template for database deployment:
{{- range $db := .Values.databases }}
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ $.Values.project }}-{{ $db.name }}-db
namespace: {{ $.Values.project }}
spec:
serviceName: {{ $db.name }}
selector:
matchLabels:
app: {{ $.Values.project }}-{{ $db.name }}-db
template:
metadata:
labels:
app: {{ $.Values.project }}-{{ $db.name }}-db
spec:
containers:
- name: {{ $db.name }}
image: {{ $db.type }}:{{ $db.version }}
{{- if eq $db.type "mysql" }}
env:
- name: MYSQL_DATABASE
value: {{ $db.name }}_db
- name: MYSQL_USER
value: {{ $db.name }}_user
- name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
name: {{ $.Values.project }}-{{ $db.name }}-secret
key: password
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: {{ $.Values.project }}-{{ $db.name }}-secret
key: password
{{- else if eq $db.type "redis" }}
args: ["--requirepass", "$(REDIS_PASSWORD)"]
env:
- name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: {{ $.Values.project }}-{{ $db.name }}-secret
key: password
{{- end }}
ports:
- containerPort: {{ $db.port }}
volumeMounts:
- name: {{ $.Values.project }}-{{ $db.name }}-data
mountPath: {{- if eq $db.type "mysql" }} /var/lib/mysql
{{- else if eq $db.type "redis" }} /data
{{- else if eq $db.type "postgres" }} /var/lib/postgresql/data
{{- else if eq $db.type "mongodb" }} /data/db
{{- end }}
volumeClaimTemplates:
- metadata:
name: {{ $.Values.project }}-{{ $db.name }}-data
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: {{ $db.size }}
{{- end }}
This template supports four database types: MySQL, PostgreSQL, Redis, and MongoDB. It also sets the environment variables and volume mount paths each one needs.
GitHub Actions Configuration Update Workflow
To update the project configuration files, I used the following GitHub Actions workflow from the CI pipeline:
name: Configuration API
on:
repository_dispatch:
types: [config-api]
jobs:
handle-request:
runs-on: ubuntu-latest
concurrency:
group: config-update
cancel-in-progress: false
steps:
- uses: actions/checkout@v3
- name: Install yq
run: |
wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/local/bin/yq
chmod +x /usr/local/bin/yq
- name: Process Request
run: |
PATH_PARAMS="${{ github.event.client_payload.path }}"
ACTION="${{ github.event.client_payload.action }}"
SPEC='${{ toJson(github.event.client_payload.spec) }}'
IFS='/' read -r -a PATH_ARRAY <<< "$PATH_PARAMS"
PROJECT="${PATH_ARRAY[1]}"
SUB_RESOURCE="${PATH_ARRAY[2]}"
NAME="${PATH_ARRAY[3]}"
FILE="projects/$PROJECT.yaml"
if [ "$ACTION" = "apply" ] && [ -f "$FILE" ]; then
if [ "$SUB_RESOURCE" = "applications" ]; then
yq eval "(.applications[] | select(.name == \"$NAME\")) *= ${SPEC}" -i $FILE
elif [ "$SUB_RESOURCE" = "databases" ]; then
yq eval "(.databases[] | select(.name == \"$NAME\")) *= ${SPEC}" -i $FILE
fi
fi
- name: Commit and push changes
run: |
git config user.name "CI Bot"
git config user.email "[email protected]"
git add .
git commit -m "${{ github.event.client_payload.action }} ${{ github.event.client_payload.path }}"
git push
This workflow receives repository_dispatch events and uses yq to update project configuration files. When a CI build succeeds, it updates the git.hash field to the new commit hash. ArgoCD then detects the change and deploys the new image.
Project Creation and Usage
In practice, adding a new project to this platform looked like this.
Creating a Project Configuration File
The projects/myproject.yaml file in this setup looked like this:
applications:
- name: api
git:
type: github
owner: myorg
repo: my-api-server
branch: main
port: 8080
domains:
- api.myproject.example.com
databases:
- name: mysql
type: mysql
version: "8.0"
port: 3306
size: 2Gi
Storing Secrets in Vault
I first stored the project-specific secrets in Vault:
vault kv put injunweb/myproject-github-access username=myuser token=ghp_xxxxx
vault kv put injunweb/myproject-mysql-secret password=mysecretpassword
vault kv put injunweb/myproject-api-secret API_KEY=my-api-key
Verifying Deployment
After I committed and pushed the project configuration file, ArgoCD created the related resources automatically:
kubectl get ns myproject
kubectl get eventbus,eventsource,sensor -n myproject
kubectl get statefulset -n myproject
When code is pushed to the GitHub repository, the CI pipeline runs automatically, then builds and deploys the application.
Conclusion
This post covered how I built a small Internal Developer Platform (IDP) on top of the homelab Kubernetes cluster using Helm templates and ArgoCD ApplicationSet. The biggest payoff was not having to rebuild the same CI/CD and infrastructure plumbing every time I wanted to stand up a new project.
The next post covers installing Prometheus, Grafana, and Loki to build a monitoring setup that collects and visualizes cluster metrics and logs.