controller: fix frequent sidecar restarts

[sauterp]

Description

Containers in the controller pod would frequently restart

❯ kubectl -n kube-system get pods | grep exoscale-csi
exoscale-csi-controller-67c9d7ddb-2hgtv    7/7     Running   35 (89m ago)    6d19h
exoscale-csi-controller-67c9d7ddb-78744    7/7     Running   41 (5h6m ago)   6d19h
exoscale-csi-node-fvr62                    3/3     Running   0               6d19h
exoscale-csi-node-lsx6b                    3/3     Running   0               6d19h

Upon closer inspection we found that only sidecar containers would restart with the same errors at roughly the same time:

I0215 11:52:04.337536       1 leaderelection.go:281] successfully renewed lease kube-system/external-resizer-csi-exoscale-com
E0215 11:52:16.353119       1 leaderelection.go:369] Failed to update lock: etcdserver: request timed out
I0215 11:52:19.341615       1 leaderelection.go:285] failed to renew lease kube-system/external-resizer-csi-exoscale-com: timed out waiting for the condition
F0215 11:52:19.341778       1 leader_election.go:182] stopped leading

Judging by apiserver logs it looks like etcd throttled the requests because they were too frequent, which led the sidecars to restart. To fix this we double the lease durations, renew deadlines and retry periods for the leader elections of all sidecars.
We also add some missing RBAC rules.

Checklist

(For exoscale contributors)

• Changelog updated (under Unreleased block)
• Integration tests OK

Testing

The sidecars have been running for more than two days without restarting, which was the case previously.

❯ kubectl -n kube-system get pods | grep exoscale-csi
exoscale-csi-controller-6d55bf4879-4wjl7   7/7     Running   0          2d17h
exoscale-csi-controller-6d55bf4879-tvb9t   7/7     Running   0          2d17h
exoscale-csi-node-fvr62                    3/3     Running   0          10d
exoscale-csi-node-lsx6b                    3/3     Running   0          10d

I also checked the logs of all the sidecars and couldn't find anymore errors.

[shortcut-integration]

This pull request has been linked to Shortcut Story #86085: GH Issue (exoscale-csi-driver): [Bug]: Controller crashing / restarting.

[pierre-emmanuelJ]

👍