GRPC response DeadlineExceeded

[ruan875417]

What happened:

When use external-provisioner, we creat pvc, then our driver creates volume successfully, [controllerserver.go:116] CreateVolume pvc-069b89a0-271f-4d86-86bf-61608aae66d4 success. cost time:10.125894947s, it cost time more than 10s

then external-provisioner reports context deadline exceeded, it seems that timeout is 10s

howerver, --timeout param seems not work

What you expected to happen:

external-provisioner GRPC response is normal

How to reproduce it:

driver creates volume cost time more than 10s

Anything else we need to know?:

Environment:

• Driver version:
• Kubernetes version (use kubectl version):
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Others:

[jsafrane]

What external-provisioner do you use? I can see that the latest v5.1.0 uses value of --timeout parameter for CreateVolume call.

With --timeout=20s and a CSI driver that never finishes CreateVolume call, I can see the call times out after 20s:

I1217 10:29:07.625495       1 connection.go:264] "GRPC call" method="/csi.v1.Controller/CreateVolume" request="{\"accessibility_requirements\":{\"preferred\":[{\"segments\":{\"topology.hostpath.csi/node\":\"192.168.122.186\"}}],\"requisite\":[{\"segments\":{\"topology.hostpath.csi/node\":\"192.168.122.186\"}}]},\"capacity_range\":{\"required_bytes\":1073741824},\"name\":\"pvc-1f75dddf-6b0c-444d-a150-b3db8e7df7b9\",\"volume_capabilities\":[{\"AccessType\":{\"Mount\":{}},\"access_mode\":{\"mode\":7}}]}"
I1217 10:29:27.625435       1 connection.go:270] "GRPC response" response="{}" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"

And with --timeout=50s, I can see 50s delay:

I1217 10:31:50.988082       1 connection.go:264] "GRPC call" method="/csi.v1.Controller/CreateVolume" request="{\"accessibility_requirements\":{\"preferred\":[{\"segments\":{\"topology.hostpath.csi/node\":\"192.168.122.186\"}}],\"requisite\":[{\"segments\":{\"topology.hostpath.csi/node\":\"192.168.122.186\"}}]},\"capacity_range\":{\"required_bytes\":1073741824},\"name\":\"pvc-5923156e-a2bf-4f41-afc6-0ff224a74baf\",\"volume_capabilities\":[{\"AccessType\":{\"Mount\":{}},\"access_mode\":{\"mode\":7}}]}"
I1217 10:32:40.988504       1 connection.go:270] "GRPC response" response="{}" err="rpc error: code = DeadlineExceeded desc = context deadline exceeded"

If you have the latest provisioner version, I would suggest to debug on your side, as I can't reproduce the issue.

[jsafrane]

I can see the --timeout works correcly even with v5.0.1, so please triple check everything on your side.