Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BOP-1572] Add etcd configuration and custom flag migration info #243

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions content/docs/configuration/kubernetes/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ MKE 4 uses K0s to deploy core Kubernetes components, including:
- [kube-apiserver](../../configuration/kubernetes/kube-apiserver)
- [kube-controller-manager](../../configuration/kubernetes/kube-controller-manager)
- [kube-scheduler](../../configuration/kubernetes/kube-scheduler)
- [etcd](../../configuration/kubernetes/etcd)

You can configure all of the Kubernetes components through the MKE
configuration file.
29 changes: 29 additions & 0 deletions content/docs/configuration/kubernetes/etcd.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
title: etcd
weight: 2
---

etcd is a consistent, distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. It handles leader elections during network partitions and can tolerate machine failure, even in the leader node.

For MKE, etcd serves as the Kubernetes backing store for all cluster data, with an etcd replica deployed on each MKE manager node. This is a primary reason why Mirantis recommends that you deploy an odd number of MKE manager nodes, as etcd uses the Raft consensus algorithm and thus requires that a quorum of nodes agree on any updates to the cluster state.

You can find in-depth information about etcd in the official [documentation](https://etcd.io/docs/).

You can configure etcd through the `etcd` section of the MKE configuration file, an example of which follows:

```yaml
spec:
etcd:
storageQuota: 2GB
tlsCipherSuites: TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256
```

## Configure etcd storage quota

You can control the etcd distributed key-value storage quota using the `etcd.storageQuota` parameter in the MKE configuration file. By default, the value of the parameter is 2GB.

If you choose to increase the etcd quota, be aware that this quota has a limit and such action should be used in conjunction with other strategies, such as decreasing events TTL to ensure that the etcd database does not run out of space.

{{< callout type="warning" >}} If a manager node virtual machine runs out of disk space, or if all of its system memory is depleted, etcd can cause the MKE cluster to move into an irrecoverable state. To prevent this from happening, configure the disk space and the memory of the manager node VMs to levels that are well in excess of the set etcd storage quota. {{< /callout >}}

{{< callout type="info" >}} etcd official documentation, [How to debug large db size issue?](https://etcd.io/blog/2023/how_to_debug_large_db_size_issue/) {{< /callout >}}
40 changes: 40 additions & 0 deletions content/docs/migrate-from-MKE-3.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,46 @@ mkectl init --mke3-config </path/to/mke3-config.toml>
ensure you complete the [GPU prerequisites](../configuration/nvidia-gpu/#prerequisites) before
starting the upgrade process. {{< /callout >}}

### Kubernetes Custom Flags

MKE 3 and MKE 4 both support the application of additional flags to Kubernetes components that have the following fields in the MKE configuration file, each specified as a list of strings:
```
custom_kube_api_server_flags
custom_kube_controller_manager_flags
custom_kubelet_flags
custom_kube_scheduler_flags
custom_kube_proxy_flags
```

MKE 4 supports an `extraArgs` field for each of these components, though, which accepts a map of key-value pairs. During upgrade from MKE 3, MKE 4 converts these custom flags to the corresponding `extraArgs` field. Any flags that cannot be automatically converted are listed in the upgrade summary.

Example of custom flags conversion:

- MKE 3 configuration file:

```
[cluster_config.custom_kube_api_server_flags] = ["--enable-garbage-collector=false"]
```

- MKE 4 configuration file:

```
spec:
apiServer:
extraArgs:
enable-garbage-collector: false
```

### Kubelet Custom Flag Profiles

MKE 3 supports a map of kubelet flag profiles to specific nodes using the `custom_kubelet_flags_profiles` setting in the toml configuration file.

MKE 4 does not support kubelet flag profiles, but you can use [Kubelet custom profiles](../configuration/kubernetes/kubelet.md#kubelet-custom-profiles) to map `KubeletConfiguration` values to specific nodes. MKE 4 does supports the migration of MKE 3 kubelet flag profiles to kubelet custom profiles.

The conversion of flags to `KubeletConfiguration` values is best-effort ,and any flags that cannot be
converted are listed in the upgrade summary. Hosts with a custom flag profile label are marked for the
corresponding kubelet custom profile.

## Perform the migration

An upgrade from MKE 3 to MKE 4 consists of the following steps, all of which
Expand Down