Setup Kubernetes NFS Subdir External Provisioner

In the previous step, we set up our NFS server using OpenMediaVault. Now we’ll install NFS subdir external Provisioner. NFS subdir external provisioner is an automatic provisioner that use your existing and already configured NFS server to support dynamic provisioning of Kubernetes Persistent Volumes via Persistent Volume Claims.

We’ll be following the steps laid out in the Official Github Repo, with a few minor changes.

Download Deployment files

We’ll start off by making the nfs-provisioner folder in the ~/k3s/ directory. Once the folder has been created, let’s go in the folder and download our deployment files.

$ mkdir ~/k3s/nfs-provisioner && cd ~/k3s/nfs-provisioner

Now let’s begin by downloading our deployment files

$ wget https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/class.yaml
$ wget https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/rbac.yaml
$ wget https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/deployment.yaml
$ wget https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/test-claim.yaml
$ wget https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/test-pod.yaml

Customization

1. deployment.yaml

   We’ll first start off by editing our deployment.yaml file.

   1. first, let’s change the PROVISIONER_NAME in the deployment.yaml file from k8s-sigs.io/nfs-subdir-external-provisioner to something simpler. I’ve gone with nfs-storage, but you can change it to anything else.

   2. Next we’ll change the NFS_SERVER to point to our nfs server’s IP. In my case, based off the Setup NFS Shared Drive Steps, my NFS server’s IP would be 192.168.122.81

   3. Afterwards, let’s edit the value of NFS_Path to the location of our shared drive on our server. Again, based off the steps in Setup NFS Shared Drive Steps, the location of my drive is /export/shared-nas.

   4. We’ll copy the values of Server_Server and NFS_path and paste them again under the volumes section for server and path respectively.

In the end, our deployment.yaml file will look something like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: nfs-storage
            - name: NFS_SERVER
              value: 192.168.122.81
            - name: NFS_PATH
              value: /export/nas-shared
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.122.81
            path: /export/nas-shared

Once we’ve changed the PROVISIONER_NAME in the deployment.yaml file, we’ll need to change the provisioner line the class.yaml file as well. “PROVISIONER_NAME in deployment.yaml and “provisioner” in class.yaml need to match

2. Now we’ll want to change the class.yaml file to change the provisioner from provisioner: k8s-sigs.io/nfs-subdir-external-provisioner to provisioner: nfs-storage

We’ll also want to add a pathPattern as well as a onDelete parameter. The pathPattern will create a directory in the NFS drive with the PVC name to save your app data

in the end, our class.yaml file will look something like this:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-client
provisioner: nfs-storage # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  pathPattern: "${.PVC.namespace}/${.PVC.annotations.nfs.io/storage-path}" # waits for nfs.io/storage-path annotation, if not specified will accept as empty string.
  onDelete: delete

Deployment

Now we’re ready to deploy our provisioner. This part is pretty simple. We will run $kubectl apply -f (FILENAME) on our rbac.yaml, deployment.yaml and class.yaml

$ kubectl apply -f rbac.yaml
$ kubectl apply -f deployment.yaml
$ kubectl apply -f class.yaml

Testing

We’re now ready to test and see if our deployment was successful. In the terminal, we’ll want to run the following command:

$ kubectl create -f test-claim.yaml -f test-pod.yaml

persistentvolumeclaim/test-claim created
pod/test-pod created

With this command, we’ll deploy a test pod that will create a file called SUCCESS

ssh to your NFS server(keep in mind, you can only ssh as root) and confirm the SUCCESS file was created

root@k3sNAS:/export/nas-shared# ls
default
root@k3sNAS:/export/nas-shared# cd default/
root@k3sNAS:/export/nas-shared/default# ls
SUCCESS

Once we’ve confirmed everything was successful, we’re okay to delete our test deployment. We’ll want to check deleting our deployment also deleted the folder/file that was created due to our test deployment.

$ kubectl delete -f test-pod.yaml -f test-claim.yaml

pod "test-pod" deleted
persistentvolumeclaim "test-claim" deleted

Now let’s confirm the file was deleted successfully:

root@k3sNAS:/export/nas-shared# ls
default
root@k3sNAS:/export/nas-shared# cd default/
root@k3sNAS:/export/nas-shared/default# ls
SUCCESS
root@k3sNAS:/export/nas-shared/default# ls
root@k3sNAS:/export/nas-shared/default# 

Perfect! You’ve now deployed your very own nfs provisioner!