kubeadm Configuration (v1beta4)
Overview
Package v1beta4 defines the v1beta4 version of the kubeadm configuration file format. This version improves on the v1beta3 format by fixing some minor issues and adding a few new fields.
A list of changes since v1beta3:
- TODO https://github.com/kubernetes/kubeadm/issues/2890
- Support custom environment variables in control plane components under
ClusterConfiguration
. UseAPIServer.ExtraEnvs
,ControllerManager.ExtraEnvs
,Scheduler.ExtraEnvs
,Etcd.Local.ExtraEnvs
. - The
ResetConfiguration
API type is now supported in v1beta4. Users are able to reset a node by passing a--config
file tokubeadm reset
.
Migration from old kubeadm config versions
- kubeadm v1.15.x and newer can be used to migrate from v1beta1 to v1beta2.
- kubeadm v1.22.x and newer no longer support v1beta1 and older APIs, but can be used to migrate v1beta2 to v1beta3.
- kubeadm v1.27.x and newer no longer support v1beta2 and older APIs.
- TODO: https://github.com/kubernetes/kubeadm/issues/2890 add version that can be used to convert to v1beta4
Basics
The preferred way to configure kubeadm is to pass an YAML configuration file with the `--config“ option. Some of the configuration options defined in the kubeadm config file are also available as command line flags, but only the most common/simple use case are supported with this approach.
A kubeadm config file could contain multiple configuration types separated using three dashes (---
).
kubeadm supports the following configuration types:
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
apiVersion: kubeadm.k8s.io/v1beta4
kind: JoinConfiguration
To print the defaults for "init" and "join" actions use the following commands:
kubeadm config print init-defaults kubeadm config print join-defaults
The list of configuration types that must be included in a configuration file depends by the action you are
performing (init or
join`) and by the configuration options you are going to use (defaults or advanced customization).
If some configuration types are not provided, or provided only partially, kubeadm will use default values; defaults
provided by kubeadm includes also enforcing consistency of values across components when required (e.g.
--cluster-cidr
flag on controller manager and clusterCIDR
on kube-proxy).
Users are always allowed to override default values, with the only exception of a small subset of setting with relevance for security (e.g. enforce authorization-mode Node and RBAC on api server).
If the user provides a configuration types that is not expected for the action you are performing, kubeadm will ignore those types and print a warning.
Kubeadm init configuration types
When executing kubeadm init with the `--config“ option, the following configuration types could be used: InitConfiguration, ClusterConfiguration, KubeProxyConfiguration, KubeletConfiguration, but only one between InitConfiguration and ClusterConfiguration is mandatory.
apiVersion: kubeadm.k8s.io/v1beta4 kind: InitConfiguration bootstrapTokens: ... nodeRegistration: ...
The InitConfiguration type should be used to configure runtime settings, that in case of kubeadm init are the configuration of the bootstrap token and all the setting which are specific to the node where kubeadm is executed, including:
NodeRegistration, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node ip).
LocalAPIEndpoint, that represents the endpoint of the instance of the API server to be deployed on this node; use it e.g. to customize the API server advertise address.
apiVersion: kubeadm.k8s.io/v1beta4 kind: ClusterConfiguration networking: ... etcd: ... apiServer: extraArgs: ... extraVolumes: ... ...
The ClusterConfiguration type should be used to configure cluster-wide settings, including settings for:
networking
that holds configuration for the networking topology of the cluster; use it e.g. to customize Pod subnet or services subnet.etcd
: use it e.g. to customize the local etcd or to configure the API server for using an external etcd cluster.kube-apiserver, kube-scheduler, kube-controller-manager configurations; use it to customize control-plane components by adding customized setting or overriding kubeadm default settings.
apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration ...
The KubeProxyConfiguration type should be used to change the configuration passed to kube-proxy instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.
See https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/ or https://pkg.go.dev/k8s.io/kube-proxy/config/v1alpha1#KubeProxyConfiguration for kube-proxy official documentation.
apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration ...
The KubeletConfiguration type should be used to change the configurations that will be passed to all kubelet instances deployed in the cluster. If this object is not provided or provided only partially, kubeadm applies defaults.
See https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ or https://pkg.go.dev/k8s.io/kubelet/config/v1beta1#KubeletConfiguration for kubelet official documentation.
Here is a fully populated example of a single YAML file containing multiple
configuration types to be used during a kubeadm init
run.
apiVersion: kubeadm.k8s.io/v1beta4 kind: InitConfiguration bootstrapTokens: - token: "9a08jv.c0izixklcxtmnze7" description: "kubeadm bootstrap token" ttl: "24h" - token: "783bde.3f89s0fje9f38fhf" description: "another bootstrap token" usages: - authentication - signing groups: - system:bootstrappers:kubeadm:default-node-token nodeRegistration: name: "ec2-10-100-0-1" criSocket: "unix:///var/run/containerd/containerd.sock" taints: - key: "kubeadmNode" value: "someValue" effect: "NoSchedule" kubeletExtraArgs: v: 4 ignorePreflightErrors: - IsPrivilegedUser imagePullPolicy: "IfNotPresent" localAPIEndpoint: advertiseAddress: "10.100.0.1" bindPort: 6443 certificateKey: "e6a2eb8581237ab72a4f494f30285ec12a9694d750b9785706a83bfcbbbd2204" skipPhases: - addon/kube-proxy --- apiVersion: kubeadm.k8s.io/v1beta4 kind: ClusterConfiguration etcd: # one of local or external local: imageRepository: "registry.k8s.io" imageTag: "3.2.24" dataDir: "/var/lib/etcd" extraArgs: listen-client-urls: "http://10.100.0.1:2379" serverCertSANs: - "ec2-10-100-0-1.compute-1.amazonaws.com" peerCertSANs: - "10.100.0.1" # external: # endpoints: # - "10.100.0.1:2379" # - "10.100.0.2:2379" # caFile: "/etcd/kubernetes/pki/etcd/etcd-ca.crt" # certFile: "/etcd/kubernetes/pki/etcd/etcd.crt" # keyFile: "/etcd/kubernetes/pki/etcd/etcd.key" networking: serviceSubnet: "10.96.0.0/16" podSubnet: "10.244.0.0/24" dnsDomain: "cluster.local" kubernetesVersion: "v1.21.0" controlPlaneEndpoint: "10.100.0.1:6443" apiServer: extraArgs: authorization-mode: "Node,RBAC" extraVolumes: - name: "some-volume" hostPath: "/etc/some-path" mountPath: "/etc/some-pod-path" readOnly: false pathType: File certSANs: - "10.100.1.1" - "ec2-10-100-0-1.compute-1.amazonaws.com" timeoutForControlPlane: 4m0s controllerManager: extraArgs: "node-cidr-mask-size": "20" extraVolumes: - name: "some-volume" hostPath: "/etc/some-path" mountPath: "/etc/some-pod-path" readOnly: false pathType: File scheduler: extraArgs: address: "10.100.0.1" extraVolumes: - name: "some-volume" hostPath: "/etc/some-path" mountPath: "/etc/some-pod-path" readOnly: false pathType: File certificatesDir: "/etc/kubernetes/pki" imageRepository: "registry.k8s.io" clusterName: "example-cluster" --- apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration # kubelet specific options here --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration # kube-proxy specific options here
Kubeadm join configuration types
When executing kubeadm join with the --config option, the JoinConfiguration type should be provided.
apiVersion: kubeadm.k8s.io/v1beta4 kind: JoinConfiguration ...
The JoinConfiguration type should be used to configure runtime settings, that in case of kubeadm join are the discovery method used for accessing the cluster info and all the setting which are specific to the node where kubeadm is executed, including:
nodeRegistration
, that holds fields that relate to registering the new node to the cluster; use it to customize the node name, the CRI socket to use or any other settings that should apply to this node only (e.g. the node ip).`apiEndpoint“, that represents the endpoint of the instance of the API server to be eventually deployed on this node.
Resource Types
ClusterConfiguration
ClusterConfiguration contains cluster-wide configuration for a kubeadm cluster
Field | Description |
---|---|
apiVersion string | kubeadm.k8s.io/v1beta4 |
kind string | ClusterConfiguration |
etcd Etcd | Etcd holds configuration for etcd. |
networking Networking | Networking holds configuration for the networking topology of the cluster. |
kubernetesVersion string | KubernetesVersion is the target version of the control plane. |
controlPlaneEndpoint string | ControlPlaneEndpoint sets a stable IP address or DNS name for the control plane; it can be a valid IP address or a RFC-1123 DNS subdomain, both with optional TCP port. In case the ControlPlaneEndpoint is not specified, the AdvertiseAddress + BindPort are used; in case the ControlPlaneEndpoint is specified but without a TCP port, the BindPort is used. Possible usages are: e.g. In a cluster with more than one control plane instances, this field should be assigned the address of the external load balancer in front of the control plane instances. e.g. in environments with enforced node recycling, the ControlPlaneEndpoint could be used for assigning a stable DNS to the control plane. |
apiServer APIServer | APIServer contains extra settings for the API server control plane component |
controllerManager ControlPlaneComponent | ControllerManager contains extra settings for the controller manager control plane component |
scheduler ControlPlaneComponent | Scheduler contains extra settings for the scheduler control plane component |
dns DNS | DNS defines the options for the DNS add-on installed in the cluster. |
certificatesDir string | CertificatesDir specifies where to store or look for all required certificates. |
imageRepository string | ImageRepository sets the container registry to pull images from.
If empty, |
featureGates map[string]bool | FeatureGates enabled by the user. |
clusterName string | The cluster name |
InitConfiguration
InitConfiguration contains a list of elements that is specific "kubeadm init"-only runtime information.
Field | Description |
---|---|
apiVersion string | kubeadm.k8s.io/v1beta4 |
kind string | InitConfiguration |
bootstrapTokens []invalid type | BootstrapTokens is respected at |
nodeRegistration NodeRegistrationOptions | NodeRegistration holds fields that relate to registering the new control-plane node to the cluster |
localAPIEndpoint APIEndpoint | LocalAPIEndpoint represents the endpoint of the API server instance that's deployed on this control plane node In HA setups, this differs from ClusterConfiguration.ControlPlaneEndpoint in the sense that ControlPlaneEndpoint is the global endpoint for the cluster, which then loadbalances the requests to each individual API server. This configuration object lets you customize what IP/DNS name and port the local API server advertises it's accessible on. By default, kubeadm tries to auto-detect the IP of the default interface and use that, but in case that process fails you may set the desired value here. |
certificateKey string | CertificateKey sets the key with which certificates and keys are encrypted prior to being uploaded in a secret in the cluster during the uploadcerts init phase. |
skipPhases []string | SkipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the "kubeadm init --help" command. The flag "--skip-phases" takes precedence over this field. |
patches Patches | Patches contains options related to applying patches to components deployed by kubeadm during "kubeadm init". |
JoinConfiguration
JoinConfiguration contains elements describing a particular node.
Field | Description |
---|---|
apiVersion string | kubeadm.k8s.io/v1beta4 |
kind string | JoinConfiguration |
nodeRegistration NodeRegistrationOptions | NodeRegistration holds fields that relate to registering the new control-plane node to the cluster |
caCertPath string | CACertPath is the path to the SSL certificate authority used to secure comunications between node and control-plane. Defaults to "/etc/kubernetes/pki/ca.crt". |
discovery [Required]Discovery | Discovery specifies the options for the kubelet to use during the TLS Bootstrap process |
controlPlane JoinControlPlane | ControlPlane defines the additional control plane instance to be deployed on the joining node. If nil, no additional control plane instance will be deployed. |
skipPhases []string | SkipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the "kubeadm join --help" command. The flag "--skip-phases" takes precedence over this field. |
patches Patches | Patches contains options related to applying patches to components deployed by kubeadm during "kubeadm join". |
ResetConfiguration
ResetConfiguration contains a list of fields that are specifically "kubeadm reset"-only runtime information.
Field | Description |
---|---|
apiVersion string | kubeadm.k8s.io/v1beta4 |
kind string | ResetConfiguration |
cleanupTmpDir bool | CleanupTmpDir specifies whether the "/etc/kubernetes/tmp" directory should be cleaned during the reset process. |
certificatesDir string | CertificatesDir specifies the directory where the certificates are stored. If specified, it will be cleaned during the reset process. |
criSocket string | CRISocket is used to retrieve container runtime info and used for the removal of the containers. If CRISocket is not specified by flag or config file, kubeadm will try to detect one valid CRISocket instead. |
dryRun bool | DryRun tells if the dry run mode is enabled, don't apply any change if it is and just output what would be done. |
force bool | Force flag instructs kubeadm to reset the node without prompting for confirmation. |
ignorePreflightErrors []string | IgnorePreflightErrors provides a slice of pre-flight errors to be ignored during the reset process, e.g. 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks. |
skipPhases []string | SkipPhases is a list of phases to skip during command execution. The list of phases can be obtained with the "kubeadm reset phase --help" command. |
APIEndpoint
Appears in:
APIEndpoint struct contains elements of API server instance deployed on a node.
Field | Description |
---|---|
advertiseAddress string | AdvertiseAddress sets the IP address for the API server to advertise. |
bindPort int32 | BindPort sets the secure port for the API Server to bind to. Defaults to 6443. |
APIServer
Appears in:
APIServer holds settings necessary for API server deployments in the cluster
Field | Description |
---|---|
ControlPlaneComponent [Required]ControlPlaneComponent | (Members of ControlPlaneComponent are embedded into this type.)
No description provided. |
certSANs []string | CertSANs sets extra Subject Alternative Names for the API Server signing cert. |
timeoutForControlPlane meta/v1.Duration | TimeoutForControlPlane controls the timeout that we use for API server to appear |
BootstrapTokenDiscovery
Appears in:
BootstrapTokenDiscovery is used to set the options for bootstrap token based discovery
Field | Description |
---|---|
token [Required]string | Token is a token used to validate cluster information fetched from the control-plane. |
apiServerEndpoint string | APIServerEndpoint is an IP or domain name to the API server from which info will be fetched. |
caCertHashes []string | CACertHashes specifies a set of public key pins to verify when token-based discovery is used. The root CA found during discovery must match one of these values. Specifying an empty set disables root CA pinning, which can be unsafe. Each hash is specified as ":", where the only currently supported type is "sha256". This is a hex-encoded SHA-256 hash of the Subject Public Key Info (SPKI) object in DER-encoded ASN.1. These hashes can be calculated using, for example, OpenSSL. |
unsafeSkipCAVerification bool | UnsafeSkipCAVerification allows token-based discovery without CA verification via CACertHashes. This can weaken the security of kubeadm since other nodes can impersonate the control-plane. |
ControlPlaneComponent
Appears in:
ControlPlaneComponent holds settings common to control plane component of the cluster
Field | Description |
---|---|
extraArgs map[string]string | ExtraArgs is an extra set of flags to pass to the control plane component. A key in this map is the flag name as it appears on the command line except without leading dash(es). TODO: This is temporary and ideally we would like to switch all components to use ComponentConfig + ConfigMaps. |
extraVolumes []HostPathMount | ExtraVolumes is an extra set of host volumes, mounted to the control plane component. |
extraEnvs []core/v1.EnvVar | ExtraEnvs is an extra set of environment variables to pass to the control plane component. Environment variables passed using ExtraEnvs will override any existing environment variables, or *_proxy environment variables that kubeadm adds by default. |
DNS
Appears in:
DNS defines the DNS addon that should be used in the cluster
Field | Description |
---|---|
ImageMeta [Required]ImageMeta | (Members of ImageMeta are embedded into this type.)ImageMeta allows to customize the image used for the DNS component |
Discovery
Appears in:
Discovery specifies the options for the kubelet to use during the TLS Bootstrap process
Field | Description |
---|---|
bootstrapToken BootstrapTokenDiscovery | BootstrapToken is used to set the options for bootstrap token based discovery BootstrapToken and File are mutually exclusive |
file FileDiscovery | File is used to specify a file or URL to a kubeconfig file from which to load cluster information BootstrapToken and File are mutually exclusive |
tlsBootstrapToken string | TLSBootstrapToken is a token used for TLS bootstrapping. If .BootstrapToken is set, this field is defaulted to .BootstrapToken.Token, but can be overridden. If .File is set, this field must be set in case the KubeConfigFile does not contain any other authentication information |
timeout meta/v1.Duration | Timeout modifies the discovery timeout |
Etcd
Appears in:
Etcd contains elements describing Etcd configuration.
Field | Description |
---|---|
local LocalEtcd | Local provides configuration knobs for configuring the local etcd instance Local and External are mutually exclusive |
external ExternalEtcd | External describes how to connect to an external etcd cluster Local and External are mutually exclusive |
ExternalEtcd
Appears in:
ExternalEtcd describes an external etcd cluster. Kubeadm has no knowledge of where certificate files live and they must be supplied.
Field | Description |
---|---|
endpoints [Required][]string | Endpoints of etcd members. Required for ExternalEtcd. |
caFile [Required]string | CAFile is an SSL Certificate Authority file used to secure etcd communication. Required if using a TLS connection. |
certFile [Required]string | CertFile is an SSL certification file used to secure etcd communication. Required if using a TLS connection. |
keyFile [Required]string | KeyFile is an SSL key file used to secure etcd communication. Required if using a TLS connection. |
FileDiscovery
Appears in:
FileDiscovery is used to specify a file or URL to a kubeconfig file from which to load cluster information
Field | Description |
---|---|
kubeConfigPath [Required]string | KubeConfigPath is used to specify the actual file path or URL to the kubeconfig file from which to load cluster information |
HostPathMount
Appears in:
HostPathMount contains elements describing volumes that are mounted from the host.
Field | Description |
---|---|
name [Required]string | Name of the volume inside the pod template. |
hostPath [Required]string | HostPath is the path in the host that will be mounted inside the pod. |
mountPath [Required]string | MountPath is the path inside the pod where hostPath will be mounted. |
readOnly bool | ReadOnly controls write access to the volume |
pathType core/v1.HostPathType | PathType is the type of the HostPath. |
ImageMeta
Appears in:
ImageMeta allows to customize the image used for components that are not originated from the Kubernetes/Kubernetes release process
Field | Description |
---|---|
imageRepository string | ImageRepository sets the container registry to pull images from. if not set, the ImageRepository defined in ClusterConfiguration will be used instead. |
imageTag string | ImageTag allows to specify a tag for the image. In case this value is set, kubeadm does not change automatically the version of the above components during upgrades. |
JoinControlPlane
Appears in:
JoinControlPlane contains elements describing an additional control plane instance to be deployed on the joining node.
Field | Description |
---|---|
localAPIEndpoint APIEndpoint | LocalAPIEndpoint represents the endpoint of the API server instance to be deployed on this node. |
certificateKey string | CertificateKey is the key that is used for decryption of certificates after they are downloaded from the secret upon joining a new control plane node. The corresponding encryption key is in the InitConfiguration. |
LocalEtcd
Appears in:
LocalEtcd describes that kubeadm should run an etcd cluster locally
Field | Description |
---|---|
ImageMeta [Required]ImageMeta | (Members of ImageMeta are embedded into this type.)ImageMeta allows to customize the container used for etcd |
dataDir [Required]string | DataDir is the directory etcd will place its data. Defaults to "/var/lib/etcd". |
extraArgs map[string]string | ExtraArgs are extra arguments provided to the etcd binary when run inside a static pod. A key in this map is the flag name as it appears on the command line except without leading dash(es). |
extraEnvs []core/v1.EnvVar | ExtraEnvs is an extra set of environment variables to pass to the control plane component. Environment variables passed using ExtraEnvs will override any existing environment variables, or *_proxy environment variables that kubeadm adds by default. |
serverCertSANs []string | ServerCertSANs sets extra Subject Alternative Names for the etcd server signing cert. |
peerCertSANs []string | PeerCertSANs sets extra Subject Alternative Names for the etcd peer signing cert. |
Networking
Appears in:
Networking contains elements describing cluster's networking configuration
Field | Description |
---|---|
serviceSubnet string | ServiceSubnet is the subnet used by k8s services. Defaults to "10.96.0.0/12". |
podSubnet string | PodSubnet is the subnet used by pods. |
dnsDomain string | DNSDomain is the dns domain used by k8s services. Defaults to "cluster.local". |
NodeRegistrationOptions
Appears in:
NodeRegistrationOptions holds fields that relate to registering a new control-plane or node to the cluster, either via "kubeadm init" or "kubeadm join"
Field | Description |
---|---|
name string | Name is the |
criSocket string | CRISocket is used to retrieve container runtime info. This information will be annotated to the Node API object, for later re-use |
taints [Required][]core/v1.Taint | Taints specifies the taints the Node API object should be registered with. If this field is unset, i.e. nil,
it will be defaulted with a control-plane taint for control-plane nodes. If you don't want to taint your control-plane
node, set this field to an empty slice, i.e. |
kubeletExtraArgs map[string]string | KubeletExtraArgs passes through extra arguments to the kubelet. The arguments here are passed to the kubelet command line via the environment file kubeadm writes at runtime for the kubelet to source. This overrides the generic base-level configuration in the kubelet-config ConfigMap Flags have higher priority when parsing. These values are local and specific to the node kubeadm is executing on. A key in this map is the flag name as it appears on the command line except without leading dash(es). |
ignorePreflightErrors []string | IgnorePreflightErrors provides a slice of pre-flight errors to be ignored when the current node is registered, e.g. 'IsPrivilegedUser,Swap'. Value 'all' ignores errors from all checks. |
imagePullPolicy core/v1.PullPolicy | ImagePullPolicy specifies the policy for image pulling during kubeadm "init" and "join" operations. The value of this field must be one of "Always", "IfNotPresent" or "Never". If this field is unset kubeadm will default it to "IfNotPresent", or pull the required images if not present on the host. |
Patches
Appears in:
Patches contains options related to applying patches to components deployed by kubeadm.
Field | Description |
---|---|
directory string | Directory is a path to a directory that contains files named "target[suffix][+patchtype].extension". For example, "kube-apiserver0+merge.yaml" or just "etcd.json". "target" can be one of "kube-apiserver", "kube-controller-manager", "kube-scheduler", "etcd", "kubeletconfiguration". "patchtype" can be one of "strategic" "merge" or "json" and they match the patch formats supported by kubectl. The default "patchtype" is "strategic". "extension" must be either "json" or "yaml". "suffix" is an optional string that can be used to determine which patches are applied first alpha-numerically. |
BootstrapToken
Appears in:
BootstrapToken describes one bootstrap token, stored as a Secret in the cluster
Field | Description |
---|---|
token [Required]BootstrapTokenString |
|
description string |
|
ttl meta/v1.Duration |
|
expires meta/v1.Time |
|
usages []string |
|
groups []string |
|
BootstrapTokenString
Appears in:
BootstrapTokenString is a token of the format abcdef.abcdef0123456789
that is used
for both validation of the practically of the API server from a joining node's point
of view and as an authentication method for the node in the bootstrap phase of
"kubeadm join". This token is and should be short-lived.
Field | Description |
---|---|
- [Required]string | No description provided. |
- [Required]string | No description provided. |
This page is automatically generated.
If you plan to report an issue with this page, mention that the page is auto-generated in your issue description. The fix may need to happen elsewhere in the Kubernetes project.