Questions about the performance comparison

[AmbitionXiang]

Hi~ I noticed that Constellation performs even faster than insecure AKS and GKS in terms of API latency. It is amazing! How did you do that? Why is it even faster than k8s without TEEs?

[burgerdev]

Hi @AmbitionXiang,

I'm happy to see interest in Constellation's performance! The link you posted is for an older release (2.4), please also take a look at the improved benchmark tables in 2.17: https://constellation-docs.netlify.app/constellation/overview/performance/.

To answer your question: k8s API server request latency should usually not be CPU/memory-bound, so the effect of runtime encryption is often negligible. What we see in these graphs is likely a combination of:

• Increased network latency due to hosted k8s API server not being local to the cluster.
  • This should explain all the diff for read requests (GET, LIST).
• Additional work being done by the hosted API server (audit logging, tracing, synchronized replicated persistence).
  • May or may not be the reason for diff in CREATE/UPDATE/...

It's probably not a fair comparison after all, and we don't list it in the docs anymore - my takeaway from the numbers is that Constellation latency is not worse than hosted alternatives.

Cheers, Markus

[AmbitionXiang]

Hi @burgerdev , thank you for your reply！What does "API latency" mean here? Does it refer to the time when user sees the return of API instead of the time when the Pod gets started? I want to know the scheduling overhead of Constellation (the time from requesting the Pod to getting Pod started), since confidential VMs may have a large cost for network I/O.

[burgerdev]

Yes, API latency refers to the time it takes the API server to return a response to the user (cf. https://developer.mozilla.org/en-US/docs/Web/Performance/Understanding_latency#what_is_latency). Kubernetes scheduling is asynchronous, so this latency does not include the time it takes to start a pod.

From the top of my head, pod startup should roughly go through the following phases

• Pod creation
• Scheduler processing
• Kubelet processing
• Image pulling
• Container setup

Parts of these steps are bound by network latency or network throughput, but I don't think this metric is ideal for measuring CVM impact on network performance.

Did you see the network benchmark in the docs? The script for running it is available here: https://github.com/edgelesssys/k8s-bench-suite/blob/67c64c854841165b778979375444da1c02e02210/knb. We don't have comparisons with non-confidential clusters yet, but it should be straightforward to set up an Azure Kubernetes Engine with Cilium for that, using a node pool of similar VM type.

[AmbitionXiang]

Thanks for your reply again! I noticed the docs you mentioned. I do not want to "measure CVM impact on network performance". Instead, I want to measure CVM impact on pod e2e startup latency/throughput. I would appreciate it if you have such performance data. If you don't, it is fine~

[burgerdev]

Sorry, we don't have that data. If you want to measure this, the reference setup should not be a hosted Kubernetes, but a self-managed one with stacked etcd, HA config (3x etcd, 3x API server) and Cilium CNI with Wireguard enabled. kubeadm, kops, ...

[AmbitionXiang]

I got it. Thank you very much!