Under certain circumstances, specially when using an insecure external
etcd cluster (no certificates), or when using external certificates (
no CA key), some keys inside the kubeadm-certs secret data can contain
the key with an empty value on the map.
When downloading certs just ignore those that are blank and inform the
user about it.
* Upload certificates: generate certificates and check that the kubeadm-certs
secret is populated with the expected contents.
* Download certificates: given we have a kubeadm-certs secret, ensure that
keys and certificates are copied on the expected target, and that depending
on the secret nature they have the expected permissions.
The fuzzer test for NodeRegistrationOptions is defaulting the CRISocket member.
This is a left over from the times this member was statically defaulted in the
config. Now that the member is dynamically defaulted, setting CRISocket to foo
can cause issues to get undetected by the fuzzer test with this member.
The resolution is to stop setting CRISocket in the fuzzer test.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
The standalone execution of upload-certs phase does not print
the key that that user should use for the newly uploaded encrypted
secret. Print this key in the upload-certs phase in both
standalone mode or if executed in the standard init workflow.
Make it possible to omit the printing if the user passes
--skip-certificate-key-print.
Also:
- Uppercase string in Printf call in copycerts.go
- Don't use V(1) for the "Skipping phase" message in uploadcerts.go
instead always print a message that the user case use
--experimental-upload-certs. This solves a problem if the user tried
the standalone phase but didn't pass --experimental-upload-certs.
CommonConfiguration is an interface, that aims at abstracting common
configuration of InitConfiguration and JoinConfiguration. It was never widely
used and never got too much functionality attached to it. With recent
developments in the kubeadm config field, it got even more out of touch with
reality and where things are going.
Thus, the only viable solution is to remove it completely.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Add ResetClusterStatusForNode() that clears a certain
control-plane node's APIEndpoint from the ClusterStatus
key in the kubeadm ConfigMap on "kubeadm reset".
Hardcoding a temp path of /tmp/... is not portable and can potentially cause
other issues (such as flakyness) too.
Use TempFile instead.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Certain join phases have flags that are redundant:
- getControlPlanePreparePhaseFlags():
- amend flags (use switch / case)
- add example for the parent command
- rename internal function and sub-phase "manifests"
to "control-plane"
- getControlPlaneJoinPhaseFlags():
- amend flags
- remove MacroCommandLongDescription
- move most unrelated to phases output to klog.V(1)
- rename some prefixes for consistency - e.g.
[kubelet] -> [kubelet-start]
- control-plane-prepare: print details for each generated CP
component manifest.
- uppercase the info text for all "[reset].." lines
- modify the text for one line in reset
Add an extra flag isSecondaryControlPlane to RunInitNodeChecks
which can be used to indicate that the node we are checking is
a secondary control-plane. In such a case we skip some tests
that are already covered by RunJoinNodeChecks and
RunOptionalJoinNodeChecks.
This implements the certificate download for the join process. If certificates
have been uploaded during init (or explicitly on any master node) and the secret
is still present in the cluster, the join process will try to download the secret
data, decrypting it with the provided key in the new `--certificate-key` flag.
The custom sub-set interfaces in init/join phases as a pattern
have isolation benefits - e.g. when a consumer imports these
but we don't want them to be able to call all methods from
the original object that satisfies a complete interface.
On the other hand these sub-set interfaces under phases/init
and phases/join are private.
Expose a couple of new common interfaces:
- InitData from phases/init/data.go
- JoinData from phases/join/data.go
Use these interfaces in init/join phases accordingly instead
of the sub-set interfaces.
Use compile-time type assertion to verify that these
interfaces can be satisfied by init.go's initData and
join.go's joinData.
Add NO-OP objects called testInitData and joinInitData
that can be used for unit testing if embedded.
This fixes the following error:
"error execution phase control-plane-join/etcd:
control-plane-join phase invoked with an invalid data struct"
The problem here is that joinData cannot be type-asserted
to the interface type under controlplanejoin.go (controlPlaneJoinData)
because joinData lacks KubeConfigPath.
Given we use KubeConfigPath in more than one place for join
it makes sense to define define the method and make it return:
kubeadmconstants.GetAdminKubeConfigPath()
This package contains public/private key utilities copied directly from
client-go/util/cert. All imports were updated.
Future PRs will actually refactor the libraries.
Updates #71004
- Rename FindExistingKubeConfig to GetKubeConfigPath
- Cobra supports a DefValue option which can be used
to differentiate between the cases where the user set a flag
and when a flag was unset, while still adding a default value.
Use this in options/generic.go for the kubeconfig flag.
- Remove the GetKubeConfigPath() logic from `reset` and `upgrade`
as these are node level kubeadm commands.
- Default kubeconfig values to "" everywhere where GetKubeConfigPath
is used. This allows to search for existing kubeconfig locations.
The structure `applyFlags` is meant to keep a user's
input from command line and as such should be immutable.
Use either a variable or the validated `InitConfig.KubernetesVersion`
field instead.
Currently kubeadm supports a couple of configuration versions - v1alpha3 and
v1beta1. The former is deprecated, but still supported.
To discourage users from using it and to speedup conversion to newer versions,
we disable the loading of deprecated configurations by all kubeadm
sub-commands, but "kubeadm config migrate".
v1alpha3 is still present and supported at source level, but cannot be used
directly with kubeadm and some of its internal APIs.
The added benefit to this is, that users won't need to lookup for an old
kubeadm binary after upgrade, just because they were stuck with a deprecated
config version for too long.
To achieve this, the following was done:
- ValidateSupportedVersion now has an allowDeprecated boolean parameter, that
controls if the function should return an error upon detecting deprecated
config version. Currently the only deprecated version is v1alpha3.
- ValidateSupportedVersion is made package private, because it's not used
outside of the package anyway.
- BytesToInitConfiguration and LoadJoinConfigurationFromFile are modified to
disallow loading of deprecated kubeadm config versions. An error message,
that points users to kubeadm config migrate is returned.
- MigrateOldConfig is still allowed to load deprecated kubeadm config versions.
- A bunch of tests were fixed to not expect success if v1alpha3 config is
supplied.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Moved all flag code from `staging/src/k8s.io/apiserver/pkg/util/[flag|globalflag]` to `component-base/cli/[flag|globalflag]` except for the term function because of unwanted dependencies.
Currently we maintain the state of the mode of interactiveness
by updating flags.nonInteractiveMode even if the flag hasn't been
set by the user.
Since the computation of the mode is done only once it's easier
and less error prone to calculate the mode in the function
sessionIsInteractive() without mutating any flags.
If /etc/kubeadm/amdin.conf doesn't exist kubeadm reset fails
with the error:
failed to load admin kubeconfig: open /root/.kube/config: no such file or directory
Fixed by properly checking if file exists before using it.
MigrateOldConfigFromFile is a function, whose purpose is to migrate one config
into another. It is working OK for now, but it has some issues:
- It is incredibly inefficient. It can reload and re-parse a single config file
for up to 3 times.
- Because of the reloads, it has to take a file containing the configuration
(not a byte slice as most of the rest config functions). However, it returns
the migrated config in a byte slice (rather asymmetric from the input
method).
- Due to the above points it's difficult to implement a proper interface for
deprecated kubeadm config versions.
To fix the issues of MigrateOldConfigFromFile, the following is done:
- Re-implement the function by removing the calls to file loading package
public APIs and replacing them with newly extracted package private APIs that
do the job with pre-provided input data in the form of
map[GroupVersionKind][]byte.
- Take a byte slice of the input configuration as an argument. This makes the
function input symmetric to its output. Also, it's now renamed to
MigrateOldConfig to represent the change from config file path as an input
to byte slice.
- As a bonus (actually forgotten from a previous change) BytesToInternalConfig
is renamed to the more descriptive BytesToInitConfiguration.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Remove the deprecated `scheduler.alpha.kubernetes.io/critical-pod` pod annotation and use
the `priorityClassName` first class attribute instead, setting all master components to
`system-cluster-critical`.
Add test files that exclude the field in question
under KubeletConfiguration -> evictionHard for non-Linux.
Add runtime abstraction for the test files in initconfiguration_tests.go
The current code logs an error and full blown backtrace if we fail to remove
the containers upon reset. This creates unneeded, huge and rather scary log
message. Fix that by leaving just the error message.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Found using script:
https://gist.github.com/dims/384dea60754042f61d79233603034038
Just run using:
`find . -name .import-restrictions | xargs python ~/junk/sanitize-import-boss.py`
The removed entries are either packages that got moved/renamed/deleted
but are still not cleaned up from .import-restrictions files.
Change-Id: I92c400f74e6f012cc75539311ed4de280e25e918
Currently ConfigFileAndDefaultsToInternalConfig and
FetchConfigFromFileOrCluster are used to default and load InitConfiguration
from file or cluster. These two APIs do a couple of completely separate things
depending on how they were invoked. In the case of
ConfigFileAndDefaultsToInternalConfig, an InitConfiguration could be either
defaulted with external override parameters, or loaded from file.
With FetchConfigFromFileOrCluster an InitConfiguration is either loaded from
file or from the config map in the cluster.
The two share both some functionality, but not enough code. They are also quite
difficult to use and sometimes even error prone.
To solve the issues, the following steps were taken:
- Introduce DefaultedInitConfiguration which returns defaulted version agnostic
InitConfiguration. The function takes InitConfiguration for overriding the
defaults.
- Introduce LoadInitConfigurationFromFile, which loads, converts, validates and
defaults an InitConfiguration from file.
- Introduce FetchInitConfigurationFromCluster that fetches InitConfiguration
from the config map.
- Reduce, when possible, the usage of ConfigFileAndDefaultsToInternalConfig by
replacing it with DefaultedInitConfiguration or LoadInitConfigurationFromFile
invocations.
- Replace all usages of FetchConfigFromFileOrCluster with calls to
LoadInitConfigurationFromFile or FetchInitConfigurationFromCluster.
- Delete FetchConfigFromFileOrCluster as it's no longer used.
- Rename ConfigFileAndDefaultsToInternalConfig to
LoadOrDefaultInitConfiguration in order to better describe what the function
is actually doing.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
systemd is the recommended driver as per the setup of running
the kubelet using systemd as the init system. Add a preflight
check that throws a warning if this isn't the case.
Currently JoinConfigFileAndDefaultsToInternalConfig is doing a couple of
different things depending on its parameters. It:
- loads a versioned JoinConfiguration from an YAML file.
- returns defaulted JoinConfiguration allowing for some overrides.
In order to make code more manageable, the following steps are taken:
- Introduce LoadJoinConfigurationFromFile, which loads a versioned
JoinConfiguration from an YAML file, defaults it (both dynamically and
statically), converts it to internal JoinConfiguration and validates it.
- Introduce DefaultedJoinConfiguration, which returns defaulted (both
dynamically and statically) and verified internal JoinConfiguration.
The possibility of overwriting defaults via versioned JoinConfiguration is
retained.
- Re-implement JoinConfigFileAndDefaultsToInternalConfig to use
LoadJoinConfigurationFromFile and DefaultedJoinConfiguration.
- Replace some calls to JoinConfigFileAndDefaultsToInternalConfig with calls to
either LoadJoinConfigurationFromFile or DefaultedJoinConfiguration where
appropriate.
- Rename JoinConfigFileAndDefaultsToInternalConfig to the more appropriate name
LoadOrDefaultJoinConfiguration.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
DetectUnsupportedVersion is somewhat uncomfortable, complex and inefficient
function to use. It takes an entire YAML document as bytes, splits it up to
byte slices of the different YAML sub-documents and group-version-kinds and
searches through those to detect an unsupported kubeadm config. If such config
is detected, the function returns an error, if it is not (i.e. the normal
function operation) everything done so far is discarded.
This could have been acceptable, if not the fact, that in all cases that this
function is called, the YAML document bytes are split up and an iteration on
GVK map is performed yet again. Hence, we don't need DetectUnsupportedVersion
in its current form as it's inefficient, complex and takes only YAML document
bytes.
This change replaces DetectUnsupportedVersion with ValidateSupportedVersion,
which takes a GroupVersion argument and checks if it is on the list of
unsupported config versions. In that case an error is returned.
ValidateSupportedVersion relies on the caller to read and split the YAML
document and then iterate on its GVK map checking if the particular
GroupVersion is supported or not.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
* Return `nil` instead of a pointer to an empty struct when possible,
before the pointer was introduced the empty struct was required.
* Explicitly accept only one argument maximum for `kubeadm join` as in
`kubeadm join <master>`.
* Accept no arguments for `kubeadm init`.
* Make phases leafs accept arguments, whereas intermediate phases
automatically gets set that they accept no arguments.
For historical reasons InitConfiguration is used almost everywhere in kubeadm
as a carrier of various configuration components such as ClusterConfiguration,
local API server endpoint, node registration settings, etc.
Since v1alpha2, InitConfiguration is meant to be used solely as a way to supply
the kubeadm init configuration from a config file. Its usage outside of this
context is caused by technical dept, it's clunky and requires hacks to fetch a
working InitConfiguration from the cluster (as it's not stored in the config
map in its entirety).
This change is a small step towards removing all unnecessary usages of
InitConfiguration. It reduces its usage by replacing it in some places with
some of the following:
- ClusterConfiguration only.
- APIEndpoint (as local API server endpoint).
- NodeRegistrationOptions only.
- Some combinations of the above types, or if single fields from them are used,
only those field.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Fix a couple of problems related to data used by the phases runners
of `init` and `join`.
1) make `newInitData()` and `newJoinData()` return pointers.
Methods of the data objects returned by these functions should
be able to modify fields in the data objects - e.g.
`func (d initData) Client()`. This allows us to store a state and
not execute the same logic multiple times - e.g. obtaining a client.
A side effect of this change is that the `new...` functions must return
pointers, so that casting the data object in a phase, from
`workflow.RunData` to a locally defined interface, works.
2) Make it possible to pass arguments from a parent command
to a sub-phase with regards to data initialization.
In output of "kubeadm init", there are two strings for meaning of
bootstrap token which are "bootstraptoken" and "bootstrap-token".
This changes `[bootstraptoken]` to `[bootstrap-token]`.
In order to allow for a smoother UX with CRIs different than Docker, we have to
make the --cri-socket command line flag optional when just one CRI is
installed.
This change does that by doing the following:
- Introduce a new runtime function (DetectCRISocket) that will attempt to
detect a CRI socket, or return an appropriate error.
- Default to using the above function if --cri-socket is not specified and
CRISocket in NodeRegistrationOptions is empty.
- Stop static defaulting to DefaultCRISocket. And rename it to
DefaultDockerCRISocket. Its use is now narrowed to "Docker or not"
distinguishment and tests.
- Introduce AddCRISocketFlag function that adds --cri-socket flag to a flagSet.
Use that in all commands, that support --cri-socket.
- Remove the deprecated --cri-socket-path flag from kubeadm config images pull
and deprecate --cri-socket in kubeadm upgrade apply.
Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com>
Kubernetes Prow Robot
PingWang
aaa
shinytang6
fabriziopandini
Marques Johansson
Jordan Liggitt
Rafael Fernández López
Ed Bartosh
RA489
Amarnath
ymlong
Rostislav M. Georgiev
d-kuro
Lubomir I. Ivanov
qingsenLi
Marek Counts
Timothy St. Clair
Yago Nobre
pytimer
peibingit
Dmitry Rozhkov
Himanshu Pandey
vanduc95
cqy0532
zhouxiaoning
Xiang Dai
Martynas Pumputis
Andrew Lytvynov
Sandeep Rajan
SataQiu
Ted Yu
Ramya Shenoy
Davanum Srinivas
Ojas M
dmaiocchi
ksubrmnn
Roy Lenferink
Ashish Ranjan
xichengliudui
Daniel Lipovetsky
Andrew Kim
mourya007
Matt Kelly
Akihito INOH