Setting Up NFS Storage for MicroK8s Cluster

Overview

This guide provides step-by-step instructions to implement an NFS-based storage solution for MicroK8s clusters. The solution is designed to improve scalability and reliability, particularly in environments with multi-node setups where the AddNode functionality is enabled. By using NFS storage, PersistentVolumeClaims (PVCs) can be shared across nodes, enabling better scalability for Java services.

Prerequisites

MicroK8s Multi-Node Cluster: Ensure an existing multi-node MicroK8s cluster setup is available.
NFS Server: Deploy an NFS server in the same subnet/VNet as the MicroK8s nodes.
Network Connectivity: Confirm network connectivity between the NFS server and MicroK8s nodes.

Set Up the NFS Server

Deploy the NFS server in the same subnet or VNet as the MicroK8s nodes. Refer to the NSF Server Setup documentation provided by MicroK8s.

Install the CSI Driver for NFS

On the master node (or primary node in a multi-node setup), enable the CSI NFS driver by following the steps in the MicroK8s documentation.

Configure the Storage Class

Create a Kubernetes Storage Class that uses the nfs.csi.k8s.io CSI driver. Assuming you have configured an NFS share /srv/nfs and the address of your NFS server is 10.0.0.42, create the following file named sc-nfs.yaml
CODE
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: nfs-csi provisioner: nfs.csi.k8s.io parameters: server: 10.0.0.42 share: /srv/nfs reclaimPolicy: Delete volumeBindingMode: Immediate mountOptions: - hard - nfsvers=4.1
```
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-csi
provisioner: nfs.csi.k8s.io
parameters:
  server: 10.0.0.42
  share: /srv/nfs
reclaimPolicy: Delete
volumeBindingMode: Immediate
mountOptions:
  - hard
  - nfsvers=4.1
```
Apply it to your MicroK8s cluster:
CODE
microk8s kubectl apply -f - < sc-nfs.yaml
```
microk8s kubectl apply -f - < sc-nfs.yaml
```
Verify the StorageClass is created:
CODE
microk8s kubectl get storageclass
```
microk8s kubectl get storageclass
```

Test PVC Creation

Create a PersistentVolumeClaim (PVC) using the NFS StorageClass. See the guidance from MicroK8s here.
- Example PVC YAML:
  CODE
  apiVersion: v1 kind: PersistentVolumeClaim metadata: name: my-pvc spec: storageClassName: nfs-csi accessModes: [ReadWriteOnce] resources: requests: storage: 5Gi
```
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  storageClassName: nfs-csi
  accessModes: [ReadWriteOnce]
  resources:
    requests:
      storage: 5Gi
```
Apply the PVC YAML:
CODE
microk8s kubectl apply -f - < pvc-nfs.yaml
```
microk8s kubectl apply -f - < pvc-nfs.yaml
```
Verify that the PVC is bound:
CODE
microk8s kubectl describe pvc my-pvc
```
microk8s kubectl describe pvc my-pvc
```

Configure Worker Node Deployment

Update the utils/.guided-envs file:
CODE
export STORAGE_CLASS_NAME=nfs-csi export EXTLIB_SHARED_STORAGE_NAME=nfs-csi
```
export STORAGE_CLASS_NAME=nfs-csi
export EXTLIB_SHARED_STORAGE_NAME=nfs-csi
```
Deploy the worker node by running the installation script:
CODE
./deploy.sh install
```
./deploy.sh install
```
Ensure all pods are running:
CODE
microk8s kubectl get pods -n <workernode-namespace>
```
microk8s kubectl get pods -n <workernode-namespace>
```

Notes

Multibox Setup:
1. Run installer.sh on all nodes to install prerequisites, MicroK8s, and enable plugins.
2. Exclude cluster node hostnames and IPs from proxy settings.
3. Follow the AddNode steps; it may take a few minutes for the new node to appear in the cluster.
Onebox Setup: For single-node setups, the default hostpath-storage can continue to be used.

Data Use Governance