Pods exposed to peers in VPC

[jeffreyflynt]

I was looking at the commit here: 5507982

It runs a bash script that pulls an entry from: encryption-strict-mode-pod-cidrs from the cilium-config; however, I do not have that entry in my cilium config. In turn, I won't get the benefit of this mitigation.

In my cilium-config, I do have the entry: encryption-strict-mode-cidr: ""

Any ideas on how to fix this?

Running:
Image: v2.16.3

[burgerdev]

Hi Jeffrey,

Thanks for reporting this, I do agree that it is unexpected and that the mitigation from 5507982 will not work if that entry isn't set. Could you please share some more information about your setup, in particular:

• What cloud provider are you running on?
• Did you create this cluster at v2.16.3, or upgrade from an earlier version?
• Did you modify any of the default settings in config or Terraform?

Cheers, Markus

[jeffreyflynt]

Thanks for the response.

I am on Azure, I created the cluster I believe on v.2.14.x and upgraded. I didn't modify any settings via Terraform.

What I have tried was copy the helm values locally and when I put the value in like this for podCIDRList: "10.0.x.x/16" it creates the entry successfully; however, since the latest cilium fork isn't published here: https://github.com/edgelesssys/cilium/pkgs/container/cilium%2Fcilium 1.15.0-pre.3-edg.3, I can't take advantage of the fix.

Any ideas?

[burgerdev]

I believe we never released a Constellation version that had encryption-strict-mode-cidr set at all. Since upgrading to v1.15.0-pre.3 with v2.14.0, we had this override that replaces it with the more specific encryption-strict-mode-(pod|node)-cidrs.

The authoritative version of Cilium, including all patches and derivation of Helm values, is what we ship with the Constellation CLI. Cilium version 1.15.0-pre.3-edg.3 is currently only a fiction that still needs to be backported to https://github.com/edgelesssys/cilium/. We introduced this in the Constellation repo to quickly mitigate GHSA-g8fc-vrcg-8vjg. It is the same as -edg.2, except for the added init container, and should be installed in your cluster after upgrading to v2.16.3. What does the Chart metadata look like for you?

$ helm get metadata -n kube-system cilium
NAME: cilium
CHART: cilium
VERSION: 1.15.0-pre.3-edg.3
APP_VERSION: 1.15.0-pre.3-edg.3
NAMESPACE: kube-system
REVISION: 1
STATUS: deployed
DEPLOYED_AT: 2024-04-17T15:14:17+02:00

Reinstall

If the versions look different in your cluster, I think it would be best to force a reinstallation of the chart. If anything is unclear or looks unexpected to you during this process, don't hesitate to stop and clarify with me!

Ensure you're using the right version of the CLI:

$ constellation version
Version:	v2.16.3 (Enterprise build; see documentation for license agreement)
GitCommit:	ea5cdfb247829d0ea2b8573651132d9376bf9a30
GitTreeState:	clean
BuildDate:	2024-04-11T08:29:03
GoVersion:	go1.22.1 X:nocoverageredesign
Compiler:	bazel/gc
Platform:	linux/amd64

Run this command and verify that all versions (except maybe Cilium) are up-to-date:

$ constellation apply --skip-phases infrastructure,init,attestationconfig,certsans,image,k8s        
[...]
invalid upgrade for cilium: upgrading from v1.15.0-pre.3-edg.3 to v1.15.0-pre.3-edg.3 is not a valid upgrade: current version newer than or equal to new version
invalid upgrade for constellation-services: upgrading from v2.16.3 to v2.16.3 is not a valid upgrade: current version newer than or equal to new version
invalid upgrade for cert-manager: upgrading from v1.12.6 to v1.12.6 is not a valid upgrade: current version newer than or equal to new version
invalid upgrade for constellation-operators: upgrading from v2.16.3 to v2.16.3 is not a valid upgrade: current version newer than or equal to new version
invalid upgrade for constellation-csi: upgrading from v2.16.3 to v2.16.3 is not a valid upgrade: current version newer than or equal to new version
[...]

Force the Helm chart upgrade with the following command (you can ignore the note about Cert Manager if the versions are the same):

constellation-2.16.3 apply --skip-phases infrastructure,init,attestationconfig,certsans,image,k8s --force

You should now have encryption-strict-mode-pod-cidrs set in your Cilium config. Restart all Cilium agents with

kubectl rollout restart ds/cilium -n kube-system

Spot check that the init container is applying the mitigation correctly - output should be similar to the following, in particular the tc filter replace line at the very bottom.

$ kubectl logs ds/cilium -n kube-system -c firewall-pods
Found 4 pods, using pod/cilium-s2g75
+ pref=32
++ awk '/^default/ { print $5 }'
++ ip route
+ interface=eth0
+ tc qdisc add dev eth0 clsact
Error: Exclusivity flag on, cannot modify.
+ true
+ tc filter del dev eth0 ingress pref 32
+ handle=0
+ for cidr in ${POD_CIDRS}
+ handle=1
+ tc filter replace dev eth0 ingress pref 32 handle 1 protocol ip flower dst_ip 10.244.0.0/16 action drop

[jeffreyflynt]

@burgerdev

Thanks for the assist on this. I ran:

constellation apply --skip-phases infrastructure,init,attestationconfig,certsans,image,k8s

and it returned:

invalid upgrade for cilium: upgrading from v1.15.5 to v1.15.0-pre.3-edg.3 is not a valid upgrade: current version newer than or equal to new version

When I ran the above command with the --force flag, it then installed: v1.15.0-pre.3-edg.3 and I confirmed the initContainer: firewall-pods was created and ran successfully.

[burgerdev]

That's great! Btw, stay tuned for the Cilium upgrade to 1.15.5, it should come with the v2.17.0 release.

[jeffreyflynt]

@burgerdev

For future reference, how could one update the helm chart values for cilium? For instance, say I wanted to enable the Gateway Controller.

[burgerdev]

The method should be the same as listed under https://docs.cilium.io/en/latest/network/servicemesh/gateway-api/gateway-api/#installation (Helm tab), but instead of checking out Cilium you'll need to start with the chart from https://github.com/edgelesssys/constellation/tree/v2.16.4/internal/constellation/helm/charts/cilium.

Cilium has a lot of config knobs, and I would not be surprised if some combination we don't test would compromise confidentiality or render our patches ineffective. However, I don't see any particular problem in enabling the Gateway Controller.