Background

After setting up a Kubernetes cluster and a private image registry, it's time to begin practical service orchestration. This series uses the ruoyi-cloud project for hands-on deployment, following the order of MySQL, Nacos, Redis, Nginx, Gateway, Auth, and System to deploy the microservices.

Before deploying, distinguish between stateful and stateless services. MySQL, Nacos, Redis, and Nginx are treated as stateful services using StatefulSet, whereas Gateway, Auth, and System are deployed as stateless services using Deployment.

This deployment utilizes YAML files to understand how Kubernetes manages resources. Latter, platforms like KubeSphere can be used for visual deployement. However, manual YAML creation becomes labor-intensive when dealing with many services. The kompose tool from Kubernetes can convert docker-compose YAML to Kubernetes YAML.

All base images use fixed versions for production stability:

• MySQL: 8.0
• Nacos: 2.2.3
• Redis: 7.2.3
• Nginx: 1.25.3

Virtual Machine Resources

Three virtual machines are used: one master node and two worker nodes.

Hostname	IP	Description
k8s-master	192.168.44.25	Master
k8s-node1	192.168.44.26	Worker
k8s-node2	192.168.44.27	Worker

[root@k8s-master ~]# kubectl get nodes
NAME         STATUS   ROLES                  AGE   VERSION
k8s-master   Ready    control-plane,master   37h   v1.20.9
k8s-node1    Ready    <none>                 35h   v1.20.9
k8s-node2    Ready    <none>                 35h   v1.20.9

System Environment

[root@k8s-master ~]# uname -a
Linux k8s-master 3.10.0-1160.71.1.el7.x86_64 #1 SMP Tue Jun 28 15:37:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
[root@k8s-master ~]# cat /proc/version 
Linux version 3.10.0-1160.71.1.el7.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) ) #1 SMP Tue Jun 28 15:37:28 UTC 2022
[root@k8s-master ~]# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)

YAML Conversion

The ruoyi-cloud project encludes a docker-compose configuration and build scripts for dependencies in its docker directory. Upload this directory to the Kubernetes master node and use kompose to convert the docker-compose YAML to Kubernetes YAML.

curl -L https://github.com/kubernetes/kompose/releases/download/v1.26.0/kompose-linux-amd64 -o kompose
chmod +x kompose
mv ./kompose /usr/local/bin/kompose
[root@k8s-master docker]# cd /opt/docker
[root@k8s-master docker]# kompose convert

The generated YAML requires minor adjustments before application. Below is the modified YAML for MySQL.

• ruoyi-mysql-ns-sc-pv-pvc.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: ruoyi-basic
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: ruoyi-mysql-data-pv
  labels:
    pv: ruoyi-mysql-data-pv
spec:
  capacity: 
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /data/mysql/data
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-node1
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: ruoyi-mysql-log-pv
  labels:
    pv: ruoyi-mysql-log-pv
spec:
  capacity: 
    storage: 5Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /data/mysql/log
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-node1
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ruoyi-mysql-data-pvc
  namespace: ruoyi-basic
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Mi
  storageClassName: local-storage
  selector:
    matchLabels:
      pv: ruoyi-mysql-data-pv
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ruoyi-mysql-log-pvc
  namespace: ruoyi-basic
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
  storageClassName: local-storage
  selector:
    matchLabels:
      pv: ruoyi-mysql-log-pv
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: ruoyi-mysql-configmap
  namespace: ruoyi-basic
data:
  my-custom.cnf: |
    [mysqld]
    sql_mode=STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION
    socket=/var/run/mysqld/mysqld.sock

    [client]
    socket=/var/run/mysqld/mysqld.sock

Note: Local storage is used here, but NFS is recommended for production.

• ruoyi-mysql-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.26.0 (40646f47)
  labels:
    io.kompose.service: ruoyi-mysql
  name: ruoyi-mysql
  namespace: ruoyi-basic
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: ruoyi-mysql
  template:
    metadata:
      annotations:
        kompose.cmd: kompose convert
        kompose.version: 1.26.0 (40646f47)
      labels:
        io.kompose.service: ruoyi-mysql
    spec:
      containers:
        - args:
            - --innodb-buffer-pool-size=80M
            - --character-set-server=utf8mb4
            - --collation-server=utf8mb4_unicode_ci
            - --default-time-zone=+8:00
            - --lower-case-table-names=1
          env:
            - name: MYSQL_DATABASE
              value: ruoyi-cloud
            - name: MYSQL_ROOT_PASSWORD
              value: you-guess
          image: mysql:8.0
          name: ruoyi-mysql
          ports:
            - containerPort: 3306
          volumeMounts:
            - mountPath: /var/lib/mysql
              name: ruoyi-mysql-data-pvc
            - mountPath: /var/log/mysql
              name: ruoyi-mysql-log-pvc
            - mountPath: /etc/mysql/conf.d
              name: ruoyi-mysql-config
      restartPolicy: Always
      volumes:
        - name: ruoyi-mysql-data-pvc
          persistentVolumeClaim:
            claimName: ruoyi-mysql-data-pvc
        - name: ruoyi-mysql-log-pvc
          persistentVolumeClaim:
            claimName: ruoyi-mysql-log-pvc
        - name: ruoyi-mysql-config
          configMap:
            name: ruoyi-mysql-configmap

• ruoyi-mysql-service.yaml

apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.26.0 (40646f47)
  labels:
    io.kompose.service: ruoyi-mysql
  name: ruoyi-mysql
  namespace: ruoyi-basic
spec:
  ports:
    - name: "3306"
      port: 3306
      targetPort: 3306
      nodePort: 30306
  selector:
    io.kompose.service: ruoyi-mysql
  type: NodePort

Deploying MySQL

Note: Using Deployment results in randomly named pods, while StatefulSet is better for production.

# Create NameSpace, StorageClass, PV, PVC
[root@k8s-master mysql]# kubectl apply -f ruoyi-mysql-ns-sc-pv-pvc.yaml
namespace/ruoyi-basic created
persistentvolume/ruoyi-mysql-data-pv created
persistentvolume/ruoyi-mysql-log-pv created
persistentvolumeclaim/ruoyi-mysql-data-pvc created
persistentvolumeclaim/ruoyi-mysql-log-pvc created
configmap/ruoyi-mysql-configmap created

# Deploy MySQL
[root@k8s-master mysql]# kubectl apply -f ruoyi-mysql-deployment.yaml
deployment.apps/ruoyi-mysql created

# Create MySQL service
[root@k8s-master mysql]# kubectl apply -f ruoyi-mysql-service.yaml
service/ruoyi-mysql created

# Check PV info
[root@k8s-master mysql]# kubectl get pv -owide
NAME                 CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                 STORAGECLASS    REASON   AGE     VOLUMEMODE
ruoyi-mysql-data-pv   5Gi        RWX            Retain           Bound    ruoyi-basic/ruoyi-mysql-data-pvc        local-storage            5m16s   Filesystem
ruoyi-mysql-log-pv    5Gi        RWX            Retain           Bound    ruoyi-basic/ruoyi-mysql-log-pvc         local-storage            5m16s   Filesystem

# Get ConfigMap info
[root@k8s-master mysql]# kubectl get cm -n ruoyi-basic
NAME                   DATA   AGE
kube-root-ca.crt       1      17m
ruoyi-mysql-configmap   1      17m

# Edit MySQL config
[root@k8s-master ~]# kubectl edit cm ruoyi-mysql-configmap -n ruoyi-basic

# View MySQL logs
[root@k8s-master ~]# kubectl logs ruoyi-mysql-8c779d94c-b7r9n -n ruoyi-basic

# View all Pods, MySQL should be ready
[root@k8s-master mysql]# kubectl get pod -A
NAMESPACE              NAME                                         READY   STATUS    RESTARTS   AGE
ruoyi-basic             ruoyi-mysql-8c779d94c-b7r9n                   1/1     Running   0          4m3s

# Exposed port for testing
[root@k8s-master mysql]# kubectl get svc -n ruoyi-basic -o wide
NAME         TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)          AGE   SELECTOR
ruoyi-mysql   NodePort   10.96.58.67   <none>        3306:30306/TCP   4s    io.kompose.service=ruoyi-mysql

Validating MySQL Service

1. Connect via container shell: mysql -uroot -p
2. Use busybox to test internal connectivity: telnet ruoyi-mysql.ruoyi-basic.svc.cluster.local:3306
3. Connect externally using a database client on port 30306

# Enter MySQL container
kubectl exec -it pod/ruoyi-mysql-8c779d94c-b7r9n  -n ruoyi-basic -- /bin/bash

# Test with busybox
[root@k8s-master ~]# kubectl exec -it pod/busybox -- /bin/sh
/ # telnet ruoyi-mysql.ruoyi-basic.svc.cluster.local:3306
J
8.0.27,'~_xJ{C~;f<*,mqlcaching_sha2_password

Importing Data

Use SQL scripts provided by ruoyi-cloud to import data into MySQL after deployment.

Summary

Using kompose, we converted docker-compose YAML to Kubernetes YAML and applied it successfully. MySQL 8.0 is now running in the Kubernetes cluster. Next, we'll proceed with deploying Nacos v2.2.3.