Gitea on Kubernetes on One IP
For some reason I wanted to try to run my own Git server at home. I’m always looking for ways to automate the infrastructure in my basement (VMs, Ansible, Kubernetes, etc) among other things, and having a centrally-located place for storage of configuration data would help me get one step closer to my goal.
I tried out Gitlab, but the resource requirements were super high, and I didn’t really need all the features. I just need a place to create repositories, add users/roles (with SSH keys), and display changes in a web browser without necessarily checking out the repository. And I wanted to run it on my Kubernetes cluster, for fun, profit, and generally playing around.
The main accomplishments I wanted to achieve:
- Access the Gitea web UI via SSL, with certificates via Lets Encrypt.
- Both Git (over SSH) and the web UI accessed via the same hostname/IP. No seperate hostnames for SSH and Web access.
Pre-existing configuration I already had running:
- Metallb: Being able to serve Kubernetes services from a range of IP addresses from bare metal Kubernetes.
- Lets Encrypt via Cert Manager
- ingress-nginx hooked up to cert-manager: Providing HTTP/TCP proxying for services with the ability to generate and serve SSL certificates for those services.
So how did I tie this all together into a self-hosted local Git server?
Some yaml configuration, and then a hopefully-easier-to-understand RPC diagram
Helm values.yaml for gitea
--- gitea: config: APP_NAME: "Gitea: git.internal.domain.com" service: http: type: ClusterIP port: 3000 ssh: type: ClusterIP port: 22
--- apiVersion: v1 kind: Service metadata: name: gitea-metallb-https annotations: metallb.universe.tf/allow-shared-ip: "foobarbaz1" spec: type: LoadBalancer loadBalancerIP: 192.168.1.250 ports: - port: 443 protocol: TCP targetPort: 443 selector: app: nginx-ingress app.kubernetes.io/component: controller release: nginx-ingress-release --- apiVersion: v1 kind: Service metadata: name: gitea-metallb-ssh annotations: metallb.universe.tf/allow-shared-ip: "foobarbaz1" spec: type: LoadBalancer loadBalancerIP: 192.168.1.250 ports: - port: 22 protocol: TCP targetPort: 22 selector: app: gitea
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: gitea annotations: cert-manager.io/cluster-issuer: "letsencrypt-issuer-prod" spec: rules: - host: git.local.lan http: paths: - path: / pathType: Prefix backend: service: name: gitea-http port: number: 3000 tls: - hosts: - git.local.lan secretName: gitea-tls-secret
graph TD; User-- 192.168.1.250:ssh -->service:gitea-metallb-ssh; User-- 192.168.1.250:https -->service:gitea-metallb-https; service:gitea-metallb-https --> service:nginx-ingress; service:nginx-ingress --> gitea:http; service:gitea-metallb-ssh --> gitea:ssh subgraph "kubernetes"; service:nginx-ingress subgraph "metallb"; service:gitea-metallb-https service:gitea-metallb-ssh end subgraph "gitea"; gitea:http gitea:ssh end end
Both SSH and HTTPS requests for the git service’s IP address (192.168.1.250) are sent to the physical (or in my case, VM) node which currently attracts traffic for that IP address.
This metallb node is listening on ports 22 and 443. Port 22 traffic is directed at the node whose labels match “app=gitea”, while port 443 traffic is directed at the service with the labels
app.kubernetes.io/component: controller, and
In this case, port 22 (git+ssh) traffic goes directly to the Gitea pod, while port 443 (https) traffic is sent to the nginx-ingress controller. This is where SSL termination takes place, where the TLS certificate provided by certbot and Lets Encrypt is served. It is at this point that HTTPS traffic is turned into HTTP traffic and forwarded to port 3000 on the gitea-http service.