Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with Installing Trivy-Operator on GKE Autopilot Cluster #2346

Open
mohammadzaidsha opened this issue Dec 4, 2024 · 1 comment
Open
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@mohammadzaidsha
Copy link

mohammadzaidsha commented Dec 4, 2024

Hi Team,

I am encountering an issue while trying to install the Trivy-Operator using the Helm chart on our GKE Autopilot cluster. Our Kubernetes cluster is created in Autopilot mode and configured using Terraform. Below is the error message I received in the log of 'trivy-operator' :

creating job: admission webhook "warden-validating.common-webhooks.networking.gke.io" denied the request: GKE Warden rejected the request because it violates one or more constraints.
Violations details: {
  "[denied by autogke-disallow-hostnamespaces]": [
    "enabling hostPID is not allowed in Autopilot."
  ],
  "[denied by autogke-no-write-mode-hostpath]": [
    "hostPath volume var-lib-etcd used in container node-collector uses path /var/lib/etcd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume var-lib-kubelet used in container node-collector uses path /var/lib/kubelet which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume var-lib-kube-scheduler used in container node-collector uses path /var/lib/kube-scheduler which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume var-lib-kube-controller-manager used in container node-collector uses path /var/lib/kube-controller-manager which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume etc-systemd used in container node-collector uses path /etc/systemd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume lib-systemd used in container node-collector uses path /lib/systemd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume etc-kubernetes used in container node-collector uses path /etc/kubernetes which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].",
    "hostPath volume etc-cni-netd used in container node-collector uses path /etc/cni/net.d which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/]."
  ],
  "[denied by autogke-node-affinity-selector-limitation]": [
    "Key 'kubernetes.io/hostname' is not allowed with node selector; Autopilot only allows labels with keys: cloud.google.com/compute-class,cloud.google.com/machine-family,cloud.google.com/gke-ephemeral-storage-local-ssd,cloud.google.com/gke-spot,cloud.google.com/gke-placement-group,topology.kubernetes.io/region,topology.kubernetes.io/zone,failure-domain.beta.kubernetes.io/region,failure-domain.beta.kubernetes.io/zone,cloud.google.com/gke-os-distribution,kubernetes.io/os,kubernetes.io/arch,cloud.google.com/private-node,sandbox.gke.io/runtime,cloud.google.com/gke-accelerator,cloud.google.com/gke-accelerator-count,iam.gke.io/gke-metadata-server-enabled,cloud.google.com/reservation-name,cloud.google.com/gke-tpu-accelerator,cloud.google.com/gke-tpu-topology,cloud.google.com/reservation-project,cloud.google.com/reservation-affinity,cloud.google.com/gke-gpu-sharing-strategy,cloud.google.com/gke-max-shared-clients-per-gpu,cloud.google.com/gke-gpu-partition-size,cloud.google.com/pods-per-node,cloud.google.com/gke-boot-disk,cloud.google.com/gke-boot-disk-size,cloud.google.com/gke-gpu-driver-version,cloud.google.com/gke-nccl-fastsocket,cloud.google.com/pods-per-node,autoscaling.gke.io/provisioning-request."
  ]
}

Requested by user: 'system:serviceaccount:scanning:trivy-operator', groups: 'system:serviceaccounts,system:serviceaccounts:scanning,system:authenticated'.",

Here is my main.tf as configure in terraform code to deploy in my existing cluster using helm chart

resource "kubernetes_namespace" "scanning" {
  metadata {
    name = "scanning"
  }
}

resource "helm_release" "trivy" {
  name             = "trivy-operator"
  repository       = "oci://ghcr.io/aquasecurity/helm-charts"
  namespace        = kubernetes_namespace.scanning.id
  chart            = "trivy-operator"
  version          = "0.21.4"
  timeout          = 1200
  create_namespace = true

  set {
    name  = "trivy.ignoreUnfixed"
    value = "true"
  }
}

Environment:

  • Trivy-Operator version 0.21.4:
  • GKE version 1.30.5-gke.1443001
@mohammadzaidsha mohammadzaidsha added the kind/bug Categorizes issue or PR as related to a bug. label Dec 4, 2024
@itaysk
Copy link

itaysk commented Dec 10, 2024

these are all required by the node-collector. it is a component that is required for some parts of the CIS benchmarks. In trivy, you can disable the node collector using the --disable-node-collector flag. In trivy-operator you can set the same flag using Helm value: nodeCollector.useNodeSelector: false

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

2 participants