Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

operator: upgrade all control plane nodes first #3444

Draft
wants to merge 15 commits into
base: burgerdev/k8s-1.31
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .github/workflows/docs-vale.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,11 @@ jobs:
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
ref: ${{ !github.event.pull_request.head.repo.fork && github.head_ref || '' }}

# Work around https://github.com/errata-ai/vale-action/issues/128.
- run: |
venv="$HOME/.local/share/venv"
python3 -m venv "$venv"
echo "$venv/bin" >> "$GITHUB_PATH"
- name: Vale
uses: errata-ai/vale-action@91ac403e8d26f5aa1b3feaa86ca63065936a85b6 # tag=reviewdog
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/e2e-test-daily.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ jobs:
fail-fast: false
max-parallel: 5
matrix:
kubernetesVersion: ["1.28"] # should be default
kubernetesVersion: ["1.30"] # This should correspond to the current default k8s minor.
attestationVariant: ["gcp-sev-es", "gcp-sev-snp", "azure-sev-snp", "azure-tdx", "aws-sev-snp"]
refStream: ["ref/main/stream/debug/?", "ref/release/stream/stable/?"]
test: ["sonobuoy quick"]
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/e2e-test-internal-lb.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ on:
required: true
kubernetesVersion:
description: "Kubernetes version to create the cluster from."
default: "1.28"
required: true
cliVersion:
description: "Version of a released CLI to download. Leave empty to build the CLI from the checked out ref."
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/e2e-test-marketplace-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ on:
required: true
kubernetesVersion:
description: "Kubernetes version to create the cluster from."
default: "1.28"
required: true
cliVersion:
description: "Version of a released CLI to download. Leave empty to build the CLI from the checked out ref."
Expand Down
20 changes: 10 additions & 10 deletions .github/workflows/e2e-test-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,53 +73,53 @@ jobs:

- test: "sonobuoy full"
attestationVariant: "gcp-sev-es"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
runner: "ubuntu-22.04"
clusterCreation: "cli"
- test: "sonobuoy full"
attestationVariant: "gcp-sev-snp"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
runner: "ubuntu-22.04"
clusterCreation: "cli"
- test: "sonobuoy full"
attestationVariant: "azure-sev-snp"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
runner: "ubuntu-22.04"
clusterCreation: "cli"
- test: "sonobuoy full"
attestationVariant: "azure-tdx"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
runner: "ubuntu-22.04"
clusterCreation: "cli"
- test: "sonobuoy full"
attestationVariant: "aws-sev-snp"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
runner: "ubuntu-22.04"
clusterCreation: "cli"

- test: "sonobuoy full"
attestationVariant: "gcp-sev-es"
kubernetes-version: "v1.28"
kubernetes-version: "v1.29"
runner: "ubuntu-22.04"
clusterCreation: "cli"
- test: "sonobuoy full"
attestationVariant: "gcp-sev-snp"
kubernetes-version: "v1.28"
kubernetes-version: "v1.29"
runner: "ubuntu-22.04"
clusterCreation: "cli"
- test: "sonobuoy full"
attestationVariant: "azure-sev-snp"
kubernetes-version: "v1.28"
kubernetes-version: "v1.29"
runner: "ubuntu-22.04"
clusterCreation: "cli"
- test: "sonobuoy full"
attestationVariant: "azure-tdx"
kubernetes-version: "v1.28"
kubernetes-version: "v1.29"
runner: "ubuntu-22.04"
clusterCreation: "cli"
- test: "sonobuoy full"
attestationVariant: "aws-sev-snp"
kubernetes-version: "v1.28"
kubernetes-version: "v1.29"
runner: "ubuntu-22.04"
clusterCreation: "cli"

Expand Down
1 change: 0 additions & 1 deletion .github/workflows/e2e-test-terraform-provider.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ on:
required: true
kubernetesVersion:
description: "Kubernetes version to create the cluster from."
default: "1.28"
required: true
releaseVersion:
description: "Version of a released provider to download. Leave empty to build the provider from the checked out ref."
Expand Down
30 changes: 15 additions & 15 deletions .github/workflows/e2e-test-weekly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,53 +89,53 @@ jobs:
- test: "sonobuoy quick"
refStream: "ref/main/stream/debug/?"
attestationVariant: "gcp-sev-es"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
clusterCreation: "cli"
- test: "sonobuoy quick"
refStream: "ref/main/stream/debug/?"
attestationVariant: "gcp-sev-snp"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
clusterCreation: "cli"
- test: "sonobuoy quick"
refStream: "ref/main/stream/debug/?"
attestationVariant: "azure-sev-snp"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
clusterCreation: "cli"
- test: "sonobuoy quick"
refStream: "ref/main/stream/debug/?"
attestationVariant: "azure-tdx"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
clusterCreation: "cli"
- test: "sonobuoy quick"
refStream: "ref/main/stream/debug/?"
attestationVariant: "aws-sev-snp"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
clusterCreation: "cli"

- test: "sonobuoy quick"
refStream: "ref/main/stream/debug/?"
attestationVariant: "gcp-sev-es"
kubernetes-version: "v1.28"
kubernetes-version: "v1.29"
clusterCreation: "cli"
- test: "sonobuoy quick"
refStream: "ref/main/stream/debug/?"
attestationVariant: "gcp-sev-snp"
kubernetes-version: "v1.28"
kubernetes-version: "v1.29"
clusterCreation: "cli"
- test: "sonobuoy quick"
refStream: "ref/main/stream/debug/?"
attestationVariant: "azure-sev-snp"
kubernetes-version: "v1.28"
kubernetes-version: "v1.29"
clusterCreation: "cli"
- test: "sonobuoy quick"
refStream: "ref/main/stream/debug/?"
attestationVariant: "azure-tdx"
kubernetes-version: "v1.28"
kubernetes-version: "v1.29"
clusterCreation: "cli"
- test: "sonobuoy quick"
refStream: "ref/main/stream/debug/?"
attestationVariant: "aws-sev-snp"
kubernetes-version: "v1.28"
kubernetes-version: "v1.29"
clusterCreation: "cli"


Expand Down Expand Up @@ -290,27 +290,27 @@ jobs:
- test: "verify"
refStream: "ref/release/stream/stable/?"
attestationVariant: "gcp-sev-es"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
clusterCreation: "cli"
- test: "verify"
refStream: "ref/release/stream/stable/?"
attestationVariant: "gcp-sev-snp"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
clusterCreation: "cli"
- test: "verify"
refStream: "ref/release/stream/stable/?"
attestationVariant: "azure-sev-snp"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
clusterCreation: "cli"
- test: "verify"
refStream: "ref/release/stream/stable/?"
attestationVariant: "azure-tdx"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
clusterCreation: "cli"
- test: "verify"
refStream: "ref/release/stream/stable/?"
attestationVariant: "aws-sev-snp"
kubernetes-version: "v1.29"
kubernetes-version: "v1.30"
clusterCreation: "cli"

runs-on: ubuntu-24.04
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/e2e-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ on:
required: true
kubernetesVersion:
description: "Kubernetes version to create the cluster from."
default: "1.29"
default: "1.30"
required: true
cliVersion:
description: "Version of a released CLI to download. Leave empty to build the CLI from the checked out ref."
Expand Down
1 change: 1 addition & 0 deletions bootstrapper/internal/kubernetes/k8sapi/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ go_library(
"@io_k8s_kubelet//config/v1beta1",
"@io_k8s_kubernetes//cmd/kubeadm/app/apis/kubeadm/v1beta3",
"@io_k8s_kubernetes//cmd/kubeadm/app/constants",
"@org_golang_x_mod//semver",
],
)

Expand Down
8 changes: 7 additions & 1 deletion bootstrapper/internal/kubernetes/k8sapi/kubeadm_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"github.com/edgelesssys/constellation/v2/bootstrapper/internal/certificate"
"github.com/edgelesssys/constellation/v2/internal/constants"
"github.com/edgelesssys/constellation/v2/internal/kubernetes"
"golang.org/x/mod/semver"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
kubeletconf "k8s.io/kubelet/config/v1beta1"
Expand All @@ -38,7 +39,7 @@ func (c *KubdeadmConfiguration) InitConfiguration(externalCloudProvider bool, cl
cloudProvider = "external"
}

return KubeadmInitYAML{
initConfig := KubeadmInitYAML{
InitConfiguration: kubeadm.InitConfiguration{
TypeMeta: metav1.TypeMeta{
APIVersion: kubeadm.SchemeGroupVersion.String(),
Expand Down Expand Up @@ -157,6 +158,11 @@ func (c *KubdeadmConfiguration) InitConfiguration(externalCloudProvider bool, cl
TLSPrivateKeyFile: certificate.KeyFilename,
},
}

if semver.Compare(clusterVersion, "v1.31.0") >= 0 {
initConfig.ClusterConfiguration.FeatureGates = map[string]bool{"ControlPlaneKubeletLocalMode": true}
}
return initConfig
}

// JoinConfiguration returns a new kubeadm join configuration.
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/architecture/versions.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ Subsequent Constellation releases drop support for the oldest (and deprecated) K
The following Kubernetes versions are currently supported:
<!--AUTO_GENERATED_BY_BAZEL-->
<!--DO_NOT_EDIT-->
* v1.28.13
* v1.29.8
* v1.30.4
* v1.31.1
2 changes: 1 addition & 1 deletion docs/docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ constellation config generate {aws|azure|gcp|openstack|qemu|stackit} [flags]
```
-a, --attestation string attestation variant to use {aws-sev-snp|aws-nitro-tpm|azure-sev-snp|azure-tdx|azure-trustedlaunch|gcp-sev-snp|gcp-sev-es|qemu-vtpm}. If not specified, the default for the cloud provider is used
-h, --help help for generate
-k, --kubernetes string Kubernetes version to use in format MAJOR.MINOR (default "v1.29")
-k, --kubernetes string Kubernetes version to use in format MAJOR.MINOR (default "v1.30")
-t, --tags strings additional tags for created resources given a list of key=value
```

Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: autoscalingstrategies.update.edgeless.systems
annotations:
controller-gen.kubebuilder.io/version: v0.9.0
controller-gen.kubebuilder.io/version: v0.16.4
name: autoscalingstrategies.update.edgeless.systems
spec:
group: update.edgeless.systems
names:
Expand All @@ -20,14 +21,19 @@ spec:
API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
Expand All @@ -48,8 +54,8 @@ spec:
deployment.
type: string
enabled:
description: Enabled defines whether cluster autoscaling should be enabled
or not.
description: Enabled defines whether cluster autoscaling should be
enabled or not.
type: boolean
required:
- deploymentName
Expand All @@ -64,7 +70,8 @@ spec:
enabled or not.
type: boolean
replicas:
description: Replicas is the number of replicas for the autoscaler deployment.
description: Replicas is the number of replicas for the autoscaler
deployment.
format: int32
type: integer
type: object
Expand All @@ -73,9 +80,3 @@ spec:
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: joiningnodes.update.edgeless.systems
annotations:
controller-gen.kubebuilder.io/version: v0.9.0
controller-gen.kubebuilder.io/version: v0.16.4
name: joiningnodes.update.edgeless.systems
spec:
group: update.edgeless.systems
names:
Expand All @@ -19,14 +20,19 @@ spec:
description: JoiningNode is the Schema for the joiningnodes API.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
Expand Down Expand Up @@ -59,9 +65,3 @@ spec:
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
Loading