Proposal: Add Helm Chart for nydus snapshotter

[changweige]

Nydus-snapshotter has no Helm Chart to be conventionally deployed in the K8s cluster yet. We can compose such a Helm Chart to pack all necessary binaries, configs and K8s manifests into a single Helm Chart package. Depoly nydus-snapshotter as Daemonset running a each node. Run nydusd by systemd-run which makes it possible that nydusd runs in the host namespace while nydus-snapshotter runs in the its container namespace.
Nydus-snapshotter container image packs：

1. nydus-snapshotter a.k.a containerd-nydus-grpc
2. nydusd
3. TOML configs

nydus-snapshotter pod's init-container installs nydusd binary to host's system path during pod startup, which means systemd can find it in the host namespace.

In such manner, even nydus-snapshotter pod is destroyed for the reason like Daemonset rolling upgrade or other maintenance purpose or unintentional operation, the container image IO is not affected.

[imeoer]

https://github.com/dragonflyoss/helm-charts/tree/main/charts/nydus-snapshotter is it enough for this?

[gane5hvarma]

@imeoer The above helm chart is not updated with latest version of nydus snapshotter. When updated to latest snapshotter version v0.15.0. it fails with the below error

Successfully pulled image "ghcr.io/containerd/nydus-snapshotter:v0.15.0" in 118ms (118ms including waiting). Image size: 64560834 bytes.
  Warning  Failed     10s               kubelet            Error: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/var/lib/containerd/io.containerd.grpc.v1.cri/containers/nydus-snapshotter/volumes/34e0211ad3175a3621a689759309c8138927f6d053df6d181dc4270e8a5d1b84" to rootfs at "/var/lib/containerd/io.containerd.snapshotter.v1.nydus": possibly malicious path detected -- refusing to operate on /run/containerd/io.containerd.runtime.v2.task/k8s.io/nydus-snapshotter/rootfs/var/lib/containerd/io.containerd.snapshotter.v1.nydus (deleted): unknown