Add an example for dataplane healing between vl3 NSEs (#11313)

* Add an example for dataplane healing between vl3 NSEs

Signed-off-by: NikitaSkrynnik <[email protected]>

* test

Signed-off-by: NikitaSkrynnik <[email protected]>

* add some descriptions

Signed-off-by: NikitaSkrynnik <[email protected]>

* cleanup

Signed-off-by: NikitaSkrynnik <[email protected]>

* use more clients in the example

Signed-off-by: NikitaSkrynnik <[email protected]>

---------

Signed-off-by: NikitaSkrynnik <[email protected]>

examples/heal/README.md

@@ -44,4 +44,5 @@ SPIRE:
 
  vl3:
  - [vl3 NSE death](./vl3-nse-death)
- - [vl3 NSCs death](./vl3-nscs-death)
+ - [vl3 NSCs death](./vl3-nscs-death)
+ - [vl3 Dataplane Interruption](./vl3-dataplane-interrupt)

examples/heal/vl3-dataplane-interrupt/README.md

@@ -0,0 +1,137 @@
+# vL3-network - Dataplane interruption
+
+This example shows that vl3 network recovers itself after dataplane interruption
+
+
+## Run
+
+Deploy clients and vl3 nses:
+```bash
+kubectl apply -k https://github.com/networkservicemesh/deployments-k8s/examples/heal/vl3-dataplane-interrupt?ref=e3050e61b33b1833638145ae01c2bb3443aa42d3
+```
+
+Wait for clients to be ready:
+```bash
+kubectl wait -n ns-vl3-dataplane-interrupt --for=condition=ready --timeout=1m pod -l app=alpine
+```
+
+Find all clients:
+```bash
+nscs=$(kubectl  get pods -l app=alpine -o go-template --template="{{range .items}}{{.metadata.name}} {{end}}" -n ns-vl3-dataplane-interrupt)
+[[ ! -z $nscs ]]
+```
+
+Check connections between clients:
+```bash
+(
+for nsc in $nscs 
+do
+    ipAddr=$(kubectl exec -n ns-vl3-dataplane-interrupt $nsc -- ifconfig nsm-1) || exit
+    ipAddr=$(echo $ipAddr | grep -Eo 'inet addr:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'| cut -c 11-)
+    for pinger in $nscs
+    do
+        echo $pinger pings $ipAddr
+        kubectl exec $pinger -n ns-vl3-dataplane-interrupt -- ping -c2 -i 0.5 $ipAddr || exit
+    done
+done
+)
+```
+
+Check connections between clients and vl3 endpoints:
+```bash
+(
+for nsc in $nscs 
+do
+    echo $nsc pings nses
+    kubectl exec -n ns-vl3-dataplane-interrupt $nsc -- ping 172.16.0.0 -c2 -i 0.5 || exit
+    kubectl exec -n ns-vl3-dataplane-interrupt $nsc -- ping 172.16.1.0 -c2 -i 0.5 || exit
+done
+)
+```
+
+Get vl3 NSEs:
+```bash
+nses=$(kubectl get pods -l app=nse-vl3-vpp -n ns-vl3-dataplane-interrupt --template '{{range .items}}{{.metadata.name}} {{end}}')
+NSE1=$(echo $nses | cut -d " " -f 1)
+NSE2=$(echo $nses | cut -d " " -f 2)
+```
+
+Disable all memif interfaces on the first vl3 NSE:
+```bash
+ifaces=$(kubectl exec -n ns-vl3-dataplane-interrupt $NSE1 -- vppctl show int | grep memif | awk '{print $1}' | tr '\n' ' ')
+for if in $ifaces
+do
+    kubectl exec -n ns-vl3-dataplane-interrupt $NSE1 -- vppctl set interface state $if down
+done
+```
+
+Check connections between clients:
+```bash
+(
+for nsc in $nscs 
+do
+    ipAddr=$(kubectl exec -n ns-vl3-dataplane-interrupt $nsc -- ifconfig nsm-1) || exit
+    ipAddr=$(echo $ipAddr | grep -Eo 'inet addr:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'| cut -c 11-)
+    for pinger in $nscs
+    do
+        echo $pinger pings $ipAddr
+        kubectl exec $pinger -n ns-vl3-dataplane-interrupt -- ping -c2 -i 0.5 $ipAddr || exit
+    done
+done
+)
+```
+
+Check connections between clients and vl3 endpoints:
+```bash
+(
+for nsc in $nscs 
+do
+    echo $nsc pings nses
+    kubectl exec -n ns-vl3-dataplane-interrupt $nsc -- ping 172.16.0.0 -c2 -i 0.5 || exit
+    kubectl exec -n ns-vl3-dataplane-interrupt $nsc -- ping 172.16.1.0 -c2 -i 0.5 || exit
+done
+)
+```
+
+Disable all memif interfaces on the second vl3 NSE:
+```bash
+ifaces=$(kubectl exec -n ns-vl3-dataplane-interrupt $NSE2 -- vppctl show int | grep memif | awk '{print $1}' | tr '\n' ' ')
+for if in $ifaces
+do
+    kubectl exec -n ns-vl3-dataplane-interrupt $NSE2 -- vppctl set interface state $if down
+done
+```
+
+Check connections between clients:
+```bash
+(
+for nsc in $nscs 
+do
+    ipAddr=$(kubectl exec -n  ns-vl3-dataplane-interrupt $nsc -- ifconfig nsm-1) || exit
+    ipAddr=$(echo $ipAddr | grep -Eo 'inet addr:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'| cut -c 11-)
+    for pinger in $nscs
+    do
+        echo $pinger pings $ipAddr
+        kubectl exec $pinger -n ns-vl3-dataplane-interrupt -- ping -c2 -i 0.5 $ipAddr || exit
+    done
+done
+)
+```
+
+Check connections between clients and vl3 endpoints:
+```bash
+(
+for nsc in $nscs 
+do
+    echo $nsc pings nses
+    kubectl exec -n ns-vl3-dataplane-interrupt $nsc -- ping 172.16.0.0 -c2 -i 0.5 || exit
+    kubectl exec -n ns-vl3-dataplane-interrupt $nsc -- ping 172.16.1.0 -c2 -i 0.5 || exit
+done
+)
+```
+
+## Cleanup
+
+```bash
+kubectl delete ns ns-vl3-dataplane-interrupt
+```

examples/heal/vl3-dataplane-interrupt/client.yaml

@@ -0,0 +1,26 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: alpine
+  labels:
+    app: alpine
+spec:
+  replicas: 2
+  selector:
+    matchLabels:
+      app: alpine
+  template:
+    metadata:
+      labels:
+        app: alpine
+      annotations:
+        networkservicemesh.io: kernel://vl3-dataplane-interrupt/nsm-1
+    spec:
+      containers:
+      - name: alpine
+        image: alpine:3.15.0
+        imagePullPolicy: IfNotPresent
+        # simple `sleep` command would work
+        # but we need `trap` to be able to delete pods quckly
+        command: ["/bin/sh", "-c", "trap : TERM INT; sleep infinity & wait"]

examples/heal/vl3-dataplane-interrupt/kustomization.yaml

@@ -0,0 +1,16 @@
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+namespace: ns-vl3-dataplane-interrupt
+
+resources:
+- ns-vl3-dataplane-interrupt.yaml
+- netsvc.yaml
+- client.yaml
+- ../../../apps/nse-vl3-vpp
+- ../../../apps/vl3-ipam
+
+
+patchesStrategicMerge:
+- nse-patch.yaml

examples/heal/vl3-dataplane-interrupt/netsvc.yaml

@@ -0,0 +1,7 @@
+---
+apiVersion: networkservicemesh.io/v1
+kind: NetworkService
+metadata:
+  name: vl3-dataplane-interrupt
+spec:
+  payload: IP

examples/heal/vl3-dataplane-interrupt/ns-vl3-dataplane-interrupt.yaml

@@ -0,0 +1,5 @@
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: ns-vl3-dataplane-interrupt

examples/heal/vl3-dataplane-interrupt/nse-patch.yaml

@@ -0,0 +1,18 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: nse-vl3-vpp
+  labels:
+    app: nse-vl3-vpp
+spec:
+  replicas: 2
+  template:
+    spec:
+      containers:
+        - name: nse
+          env:
+            - name: NSM_SERVICE_NAMES
+              value: "vl3-dataplane-interrupt"
+            - name: NSM_REGISTER_SERVICE
+              value: "false"