kubernetes
/
minikube
mirror of https://github.com/kubernetes/minikube.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'master' of github.com:kubernetes/minikube into localization-poc

pull/4464/head
Sharif Elgamal 2019-05-30 15:26:20 -07:00
parent fbc11c327a 94e9c002a6
commit 483f9fb19e
No known key found for this signature in database
GPG Key ID: 23CC0225BD9FD702
112 changed files with 2532 additions and 828 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
CHANGELOG.md
View File

@ -1,5 +1,55 @@
# Minikube Release Notes
## Version 1.1.0 - 2019-05-21
* Allow macOS to resolve service FQDNs during 'minikube tunnel' [#3464](https://github.com/kubernetes/minikube/pull/3464)
* Expose ‘—pod-network-cidr’ argument in minikube [#3892](https://github.com/kubernetes/minikube/pull/3892)
* Upgrade default Kubernetes release to v1.14.2 [#4279](https://github.com/kubernetes/minikube/pull/4279)
* Update to Podman 1.3 & CRIO v1.14.1 [#4299](https://github.com/kubernetes/minikube/pull/4299)
* Upgrade Docker, from 18.06.3-ce to 18.09.5 [#4204](https://github.com/kubernetes/minikube/pull/4204)
* Upgrade Docker, from 18.09.5 to 18.09.6 [#4296](https://github.com/kubernetes/minikube/pull/4296)
* Add Go modules support [#4241](https://github.com/kubernetes/minikube/pull/4241)
* Add more solutions messages [#4257](https://github.com/kubernetes/minikube/pull/4257)
* Add new kubectl command [#4193](https://github.com/kubernetes/minikube/pull/4193)
* Add solution text for common kvm2 and VirtualBox problems [#4198](https://github.com/kubernetes/minikube/pull/4198)
* Adding support for s390x [#4091](https://github.com/kubernetes/minikube/pull/4091)
* Allow minikube to function with misconfigured NO_PROXY value [#4229](https://github.com/kubernetes/minikube/pull/4229)
* Disable SystemVerification preflight on Kubernetes releases <1.13 [#4306](https://github.com/kubernetes/minikube/pull/4306)
* Don't attempt to pull docker images on relaunch [#4129](https://github.com/kubernetes/minikube/pull/4129)
* Fix location of Kubernetes binaries in cache directory [#4244](https://github.com/kubernetes/minikube/pull/4244)
* Fix registry addon ReplicationController template [#4220](https://github.com/kubernetes/minikube/pull/4220)
* Make default output of 'minikube start' consume fewer lines in the terminal [#4197](https://github.com/kubernetes/minikube/pull/4197)
* Make handling of stale mount pid files more robust [#4191](https://github.com/kubernetes/minikube/pull/4191)
* Make sure to start Docker, before getting version [#4307](https://github.com/kubernetes/minikube/pull/4307)
* Restart kube-proxy using kubeadm & add bootstrapper.WaitCluster [#4276](https://github.com/kubernetes/minikube/pull/4276)
* Return host IP when using vmware as vm driver. [#4255](https://github.com/kubernetes/minikube/pull/4255)
* Select an accessible image repository for some users [#3937](https://github.com/kubernetes/minikube/pull/3937)
* Set apiserver oom_adj to -10 to avoid OOMing before other pods [#4282](https://github.com/kubernetes/minikube/pull/4282)
* Standardize ASCII prefix for info, warning, and error messages [#4162](https://github.com/kubernetes/minikube/pull/4162)
* Unset the current-context after minikube stop [#4177](https://github.com/kubernetes/minikube/pull/4177)
* Validate kvm network exists [#4308](https://github.com/kubernetes/minikube/pull/4308)
* storageclass no longer beta #4148 [#4153](https://github.com/kubernetes/minikube/pull/4153)
Thank you to the contributors whose work made v1.1 into something we could all be proud of:
- Anders F Björklund
- Chris Eason
- Deepika Pandhi
- Himanshu Pandey
- Jan Janik
- Marcos Diez
- Maximilian Hess
- Medya Gh
- Sharif Elgamal
- Thomas Strömberg
- Tiago Ilieve
- Tobias Bradtke
- Zhongcheng Lao
- Zoran Regvart
- josedonizetti
- kerami
- salamani
## Version 1.0.1 - 2019-04-29
* update-context is confusing with profiles [#4049](https://github.com/kubernetes/minikube/pull/4049)

7
MAINTAINERS
View File

@ -1,8 +1,5 @@
Dan Lorenc <dlorenc@google.com> (@dlorenc)
Lucas Käldström <lucas.kaldstrom@hotmail.co.uk> (@luxas)
Jimmi Dyson <jimmidyson@gmail.com> (@jimmidyson)
Balint Pato <balintp@google.com> (@balopat)
Priya Wadhwa <priyawadhwa@google.com> (@priyawadhwa)
Thomas Stromberg <tstromberg@google.com> (@tstromberg)
Anders F Björklund <anders.f.bjorklund@gmail.com> (@afbjorklund)
Sharif Elgamal <selgamal@google.com (@sharifelgamal)
Sharif Elgamal <selgamal@google.com> (@sharifelgamal)
Medya Ghazizadeh <medya@google.com> (@medyagh)

27
Makefile
View File

@ -14,10 +14,10 @@
# Bump these on release - and please check ISO_VERSION for correctness.
VERSION_MAJOR ?= 1
VERSION_MINOR ?= 0
VERSION_BUILD ?= 1
VERSION_MINOR ?= 1
VERSION_BUILD ?= 0
# Default to .0 for higher cache hit rates, as build increments typically don't require new ISO versions
ISO_VERSION ?= v$(VERSION_MAJOR).$(VERSION_MINOR).1
ISO_VERSION ?= v$(VERSION_MAJOR).$(VERSION_MINOR).0
VERSION ?= v$(VERSION_MAJOR).$(VERSION_MINOR).$(VERSION_BUILD)
DEB_VERSION ?= $(VERSION_MAJOR).$(VERSION_MINOR).$(VERSION_BUILD)
@ -73,7 +73,9 @@ MINIKUBE_MARKDOWN_FILES := README.md docs CONTRIBUTING.md CHANGELOG.md
MINIKUBE_BUILD_TAGS := container_image_ostree_stub containers_image_openpgp
MINIKUBE_INTEGRATION_BUILD_TAGS := integration $(MINIKUBE_BUILD_TAGS)
SOURCE_DIRS = cmd pkg test
CMD_SOURCE_DIRS = cmd pkg
SOURCE_DIRS = $(SOURCE_DIRS) test
SOURCE_PACKAGES = ./cmd/... ./pkg/... ./test/...
# $(call DOCKER, image, command)
@ -103,7 +105,7 @@ out/minikube$(IS_EXE): out/minikube-$(GOOS)-$(GOARCH)$(IS_EXE)
out/minikube-windows-amd64.exe: out/minikube-windows-amd64
cp out/minikube-windows-amd64 out/minikube-windows-amd64.exe
out/minikube-%: pkg/minikube/assets/assets.go
out/minikube-%: pkg/minikube/assets/assets.go $(shell find $(CMD_SOURCE_DIRS) -type f -name "*.go")
ifeq ($(MINIKUBE_BUILD_IN_DOCKER),y)
$(call DOCKER,$(BUILD_IMAGE),/usr/bin/make $@)
else
@ -221,13 +223,16 @@ vet:
# Once v1.16.1+ is released, replace with
# curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh \
# | bash -s -- -b out/linters v1.16.0
# | bash -s -- -b out/linters v1.16.1
out/linters/golangci-lint:
mkdir -p out/linters \
&& cd out/linters \
&& test -f go.mod || go mod init linters \
&& go get -u github.com/golangci/golangci-lint/cmd/golangci-lint@692dacb773b703162c091c2d8c59f9cd2d6801db >/dev/null
mkdir -p out/linters
cd out/linters
test -f go.mod || go mod init linters
go get -u github.com/golangci/golangci-lint/cmd/golangci-lint@692dacb773b703162c091c2d8c59f9cd2d6801db 2>&1 \
| grep -v "go: finding"
test -x $(GOPATH)/bin/golangci-lint \
|| curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh \
| sh -s -- -b $(GOPATH)/bin v1.16.0
cp -f $(GOPATH)/bin/golangci-lint out/linters/golangci-lint
.PHONY: lint

12
OWNERS
View File

@ -1,16 +1,18 @@
# See the OWNERS docs at https://go.k8s.io/owners
reviewers:
- dlorenc
- balopat
- tstromberg
- afbjorklund
- sharifelgamal
- RA489
approvers:
- balopat
- tstromberg
- afbjorklund
- sharifelgamal
- RA489
emeritus_approvers:
- dlorenc
- luxas
- jimmidyson
- balopat
- tstromberg
- afbjorklund
- sharifelgamal

1
README.md
View File

@ -21,6 +21,7 @@ minikube's [primary goals](https://github.com/kubernetes/minikube/blob/master/do
## News
* 2019-05-21 - v1.1.0 released! [[download](https://github.com/kubernetes/minikube/releases/tag/v1.1.0)] [[release notes](https://github.com/kubernetes/minikube/blob/master/CHANGELOG.md#version-110---2019-05-21)]
* 2019-04-29 - v1.0.1 released! [[download](https://github.com/kubernetes/minikube/releases/tag/v1.0.1)] [[release notes](https://github.com/kubernetes/minikube/blob/master/CHANGELOG.md#version-101---2019-04-29)]
* 2019-03-27 - v1.0.0 released! [[download](https://github.com/kubernetes/minikube/releases/tag/v1.0.0)] [[release notes](https://github.com/kubernetes/minikube/blob/master/CHANGELOG.md#version-1000---2019-03-27)]

2
cmd/minikube/cmd/config/open.go
View File

@ -76,7 +76,7 @@ minikube addons list`, addonName)
exit.WithError("IsEnabled failed", err)
}
if !ok {
console.ErrStyle("conflict", `addon '%s' is currently not enabled.
console.ErrStyle(console.Conflict, `addon '%s' is currently not enabled.
To enable this addon run:
minikube addons enable %s`, addonName, addonName)
os.Exit(exit.Unavailable)

19
cmd/minikube/cmd/config/profile.go
View File

@ -17,7 +17,10 @@ limitations under the License.
package config
import (
"os"
"github.com/spf13/cobra"
"github.com/spf13/viper"
pkgConfig "k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/minikube/console"
"k8s.io/minikube/pkg/minikube/exit"
@ -25,12 +28,18 @@ import (
// ProfileCmd represents the profile command
var ProfileCmd = &cobra.Command{
Use: "profile MINIKUBE_PROFILE_NAME. You can return to the default minikube profile by running `minikube profile default`",
Short: "Profile sets the current minikube profile",
Long: "profile sets the current minikube profile. This is used to run and manage multiple minikube instance. You can return to the default minikube profile by running `minikube profile default`",
Use: "profile [MINIKUBE_PROFILE_NAME]. You can return to the default minikube profile by running `minikube profile default`",
Short: "Profile gets or sets the current minikube profile",
Long: "profile sets the current minikube profile, or gets the current profile if no arguments are provided. This is used to run and manage multiple minikube instance. You can return to the default minikube profile by running `minikube profile default`",
Run: func(cmd *cobra.Command, args []string) {
if len(args) != 1 {
exit.Usage("usage: minikube profile MINIKUBE_PROFILE_NAME")
if len(args) == 0 {
profile := viper.GetString(pkgConfig.MachineProfile)
console.OutLn("%s", profile)
os.Exit(0)
}
if len(args) > 1 {
exit.Usage("usage: minikube profile [MINIKUBE_PROFILE_NAME]")
}
profile := args[0]

32
cmd/minikube/cmd/config/unset_test.go Normal file
View File

@ -0,0 +1,32 @@
/*
Copyright 2019 The Kubernetes Authors All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package config
import "testing"
func TestUnsetConfig(t *testing.T) {
propName := "cpus"
propValue := "1"
err := Set(propName, propValue)
if err != nil {
t.Errorf("Failed to set the property %q", propName)
}
err = unset(propName)
if err != nil {
t.Errorf("Failed to unset property %q", err)
}
}

2
cmd/minikube/cmd/config/validations.go
View File

@ -55,7 +55,7 @@ func IsValidDriver(string, driver string) error {
// RequiresRestartMsg returns the "requires restart" message
func RequiresRestartMsg(string, string) error {
console.OutStyle("warning", "These changes will take effect upon a minikube delete and then a minikube start")
console.OutStyle(console.WarningType, "These changes will take effect upon a minikube delete and then a minikube start")
return nil
}

25
cmd/minikube/cmd/dashboard.go
View File

@ -21,6 +21,7 @@ import (
"fmt"
"io"
"net/http"
"os"
"os/exec"
"regexp"
"time"
@ -32,9 +33,11 @@ import (
configcmd "k8s.io/minikube/cmd/minikube/cmd/config"
"k8s.io/minikube/pkg/minikube/cluster"
"k8s.io/minikube/pkg/minikube/config"
pkg_config "k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/minikube/console"
"k8s.io/minikube/pkg/minikube/exit"
"k8s.io/minikube/pkg/minikube/machine"
"k8s.io/minikube/pkg/minikube/proxy"
"k8s.io/minikube/pkg/minikube/service"
"k8s.io/minikube/pkg/util"
)
@ -52,11 +55,19 @@ var dashboardCmd = &cobra.Command{
Short: "Access the kubernetes dashboard running within the minikube cluster",
Long: `Access the kubernetes dashboard running within the minikube cluster`,
Run: func(cmd *cobra.Command, args []string) {
cc, err := pkg_config.Load()
if err != nil && !os.IsNotExist(err) {
console.ErrLn("Error loading profile config: %v", err)
}
err = proxy.ExcludeIP(cc.KubernetesConfig.NodeIP) // to be used for http get calls
if err != nil {
glog.Errorf("Error excluding IP from proxy: %s", err)
}
kubectl, err := exec.LookPath("kubectl")
if err != nil {
exit.WithCode(exit.NoInput, "kubectl not found in PATH, but is required for the dashboard. Installation guide: https://kubernetes.io/docs/tasks/tools/install-kubectl/")
}
api, err := machine.NewAPIClient()
defer func() {
err := api.Close()
@ -71,7 +82,7 @@ var dashboardCmd = &cobra.Command{
cluster.EnsureMinikubeRunningOrExit(api, 1)
// Send status messages to stderr for folks re-using this output.
console.ErrStyle("enabling", "Enabling dashboard ...")
console.ErrStyle(console.Enabling, "Enabling dashboard ...")
// Enable the dashboard add-on
err = configcmd.Set("dashboard", "true")
if err != nil {
@ -80,19 +91,19 @@ var dashboardCmd = &cobra.Command{
ns := "kube-system"
svc := "kubernetes-dashboard"
console.ErrStyle("verifying", "Verifying dashboard health ...")
console.ErrStyle(console.Verifying, "Verifying dashboard health ...")
if err = util.RetryAfter(180, func() error { return service.CheckService(ns, svc) }, 1*time.Second); err != nil {
exit.WithCode(exit.Unavailable, "%s:%s is not running: %v", ns, svc, err)
}
console.ErrStyle("launch", "Launching proxy ...")
console.ErrStyle(console.Launch, "Launching proxy ...")
p, hostPort, err := kubectlProxy(kubectl)
if err != nil {
exit.WithError("kubectl proxy", err)
}
url := dashboardURL(hostPort, ns, svc)
console.ErrStyle("verifying", "Verifying proxy health ...")
console.ErrStyle(console.Verifying, "Verifying proxy health ...")
if err = util.RetryAfter(60, func() error { return checkURL(url) }, 1*time.Second); err != nil {
exit.WithCode(exit.Unavailable, "%s is not responding properly: %v", url, err)
}
@ -100,7 +111,7 @@ var dashboardCmd = &cobra.Command{
if dashboardURLMode {
console.OutLn(url)
} else {
console.ErrStyle("celebrate", "Opening %s in your default browser...", url)
console.ErrStyle(console.Celebrate, "Opening %s in your default browser...", url)
if err = browser.OpenURL(url); err != nil {
console.Failure("failed to open browser: %v", err)
}
@ -117,7 +128,9 @@ var dashboardCmd = &cobra.Command{
func kubectlProxy(path string) (*exec.Cmd, string, error) {
// port=0 picks a random system port
// config.GetMachineName() respects the -p (profile) flag
cmd := exec.Command(path, "--context", config.GetMachineName(), "proxy", "--port=0")
stdoutPipe, err := cmd.StdoutPipe()
if err != nil {
return nil, "", errors.Wrap(err, "cmd stdout")

8
cmd/minikube/cmd/delete.go
View File

@ -59,7 +59,7 @@ associated files.`,
if err == nil && cc.MachineConfig.VMDriver == "none" {
kc := cc.KubernetesConfig
bsName := viper.GetString(cmdcfg.Bootstrapper)
console.OutStyle("resetting", "Uninstalling Kubernetes %s using %s ...", kc.KubernetesVersion, bsName)
console.OutStyle(console.Resetting, "Uninstalling Kubernetes %s using %s ...", kc.KubernetesVersion, bsName)
clusterBootstrapper, err := GetClusterBootstrapper(api, viper.GetString(cmdcfg.Bootstrapper))
if err != nil {
console.ErrLn("Unable to get bootstrapper: %v", err)
@ -73,7 +73,7 @@ associated files.`,
if err = cluster.DeleteHost(api); err != nil {
switch err := errors.Cause(err).(type) {
case mcnerror.ErrHostDoesNotExist:
console.OutStyle("meh", "%q cluster does not exist", profile)
console.OutStyle(console.Meh, "%q cluster does not exist", profile)
default:
exit.WithError("Failed to delete cluster", err)
}
@ -85,12 +85,12 @@ associated files.`,
if err := os.Remove(constants.GetProfileFile(viper.GetString(pkg_config.MachineProfile))); err != nil {
if os.IsNotExist(err) {
console.OutStyle("meh", "%q profile does not exist", profile)
console.OutStyle(console.Meh, "%q profile does not exist", profile)
os.Exit(0)
}
exit.WithError("Failed to remove profile", err)
}
console.OutStyle("crushed", "The %q cluster has been deleted.", profile)
console.OutStyle(console.Crushed, "The %q cluster has been deleted.", profile)
},
}

31
cmd/minikube/cmd/env.go
View File

@ -35,13 +35,12 @@ import (
"github.com/spf13/cobra"
"k8s.io/minikube/pkg/minikube/cluster"
"k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/minikube/constants"
"k8s.io/minikube/pkg/minikube/exit"
"k8s.io/minikube/pkg/minikube/machine"
)
const (
envTmpl = `{{ .Prefix }}DOCKER_TLS_VERIFY{{ .Delimiter }}{{ .DockerTLSVerify }}{{ .Suffix }}{{ .Prefix }}DOCKER_HOST{{ .Delimiter }}{{ .DockerHost }}{{ .Suffix }}{{ .Prefix }}DOCKER_CERT_PATH{{ .Delimiter }}{{ .DockerCertPath }}{{ .Suffix }}{{ .Prefix }}DOCKER_API_VERSION{{ .Delimiter }}{{ .DockerAPIVersion }}{{ .Suffix }}{{ if .NoProxyVar }}{{ .Prefix }}{{ .NoProxyVar }}{{ .Delimiter }}{{ .NoProxyValue }}{{ .Suffix }}{{end}}{{ .UsageHint }}`
envTmpl = `{{ .Prefix }}DOCKER_TLS_VERIFY{{ .Delimiter }}{{ .DockerTLSVerify }}{{ .Suffix }}{{ .Prefix }}DOCKER_HOST{{ .Delimiter }}{{ .DockerHost }}{{ .Suffix }}{{ .Prefix }}DOCKER_CERT_PATH{{ .Delimiter }}{{ .DockerCertPath }}{{ .Suffix }}{{ if .NoProxyVar }}{{ .Prefix }}{{ .NoProxyVar }}{{ .Delimiter }}{{ .NoProxyValue }}{{ .Suffix }}{{end}}{{ .UsageHint }}`
fishSetPfx = "set -gx "
fishSetSfx = "\";\n"
@ -108,16 +107,15 @@ REM @FOR /f "tokens=*" %i IN ('minikube docker-env') DO @%i
// ShellConfig represents the shell config
type ShellConfig struct {
Prefix string
Delimiter string
Suffix string
DockerCertPath string
DockerHost string
DockerTLSVerify string
DockerAPIVersion string
UsageHint string
NoProxyVar string
NoProxyValue string
Prefix string
Delimiter string
Suffix string
DockerCertPath string
DockerHost string
DockerTLSVerify string
UsageHint string
NoProxyVar string
NoProxyValue string
}
var (
@ -165,11 +163,10 @@ func shellCfgSet(api libmachine.API) (*ShellConfig, error) {
}
shellCfg := &ShellConfig{
DockerCertPath: envMap["DOCKER_CERT_PATH"],
DockerHost: envMap["DOCKER_HOST"],
DockerTLSVerify: envMap["DOCKER_TLS_VERIFY"],
DockerAPIVersion: constants.DockerAPIVersion,
UsageHint: generateUsageHint(userShell),
DockerCertPath: envMap["DOCKER_CERT_PATH"],
DockerHost: envMap["DOCKER_HOST"],
DockerTLSVerify: envMap["DOCKER_TLS_VERIFY"],
UsageHint: generateUsageHint(userShell),
}
if noProxy {

110
cmd/minikube/cmd/env_test.go
View File

@ -58,14 +58,13 @@ var defaultAPI = &tests.MockAPI{
// Most of the shell cfg isn't configurable
func newShellCfg(shell, prefix, suffix, delim string) *ShellConfig {
return &ShellConfig{
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
DockerAPIVersion: constants.DockerAPIVersion,
UsageHint: generateUsageHint(shell),
Prefix: prefix,
Suffix: suffix,
Delimiter: delim,
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
UsageHint: generateUsageHint(shell),
Prefix: prefix,
Suffix: suffix,
Delimiter: delim,
}
}
@ -141,16 +140,15 @@ func TestShellCfgSet(t *testing.T) {
noProxyValue: "",
noProxyFlag: true,
expectedShellCfg: &ShellConfig{
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
DockerAPIVersion: constants.DockerAPIVersion,
UsageHint: usageHintMap["bash"],
Prefix: bashSetPfx,
Suffix: bashSetSfx,
Delimiter: bashSetDelim,
NoProxyVar: "NO_PROXY",
NoProxyValue: "127.0.0.1",
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
UsageHint: usageHintMap["bash"],
Prefix: bashSetPfx,
Suffix: bashSetSfx,
Delimiter: bashSetDelim,
NoProxyVar: "NO_PROXY",
NoProxyValue: "127.0.0.1",
},
},
{
@ -161,16 +159,15 @@ func TestShellCfgSet(t *testing.T) {
noProxyValue: "",
noProxyFlag: true,
expectedShellCfg: &ShellConfig{
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
DockerAPIVersion: constants.DockerAPIVersion,
UsageHint: usageHintMap["bash"],
Prefix: bashSetPfx,
Suffix: bashSetSfx,
Delimiter: bashSetDelim,
NoProxyVar: "no_proxy",
NoProxyValue: "127.0.0.1",
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
UsageHint: usageHintMap["bash"],
Prefix: bashSetPfx,
Suffix: bashSetSfx,
Delimiter: bashSetDelim,
NoProxyVar: "no_proxy",
NoProxyValue: "127.0.0.1",
},
},
{
@ -181,16 +178,15 @@ func TestShellCfgSet(t *testing.T) {
noProxyValue: "127.0.0.1",
noProxyFlag: true,
expectedShellCfg: &ShellConfig{
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
DockerAPIVersion: constants.DockerAPIVersion,
UsageHint: usageHintMap["bash"],
Prefix: bashSetPfx,
Suffix: bashSetSfx,
Delimiter: bashSetDelim,
NoProxyVar: "no_proxy",
NoProxyValue: "127.0.0.1",
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
UsageHint: usageHintMap["bash"],
Prefix: bashSetPfx,
Suffix: bashSetSfx,
Delimiter: bashSetDelim,
NoProxyVar: "no_proxy",
NoProxyValue: "127.0.0.1",
},
},
{
@ -201,16 +197,15 @@ func TestShellCfgSet(t *testing.T) {
noProxyValue: "0.0.0.0",
noProxyFlag: true,
expectedShellCfg: &ShellConfig{
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
DockerAPIVersion: constants.DockerAPIVersion,
UsageHint: usageHintMap["bash"],
Prefix: bashSetPfx,
Suffix: bashSetSfx,
Delimiter: bashSetDelim,
NoProxyVar: "no_proxy",
NoProxyValue: "0.0.0.0,127.0.0.1",
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
UsageHint: usageHintMap["bash"],
Prefix: bashSetPfx,
Suffix: bashSetSfx,
Delimiter: bashSetDelim,
NoProxyVar: "no_proxy",
NoProxyValue: "0.0.0.0,127.0.0.1",
},
},
{
@ -221,16 +216,15 @@ func TestShellCfgSet(t *testing.T) {
noProxyValue: "0.0.0.0,127.0.0.1",
noProxyFlag: true,
expectedShellCfg: &ShellConfig{
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
DockerAPIVersion: constants.DockerAPIVersion,
UsageHint: usageHintMap["bash"],
Prefix: bashSetPfx,
Suffix: bashSetSfx,
Delimiter: bashSetDelim,
NoProxyVar: "no_proxy",
NoProxyValue: "0.0.0.0,127.0.0.1",
DockerCertPath: constants.MakeMiniPath("certs"),
DockerTLSVerify: "1",
DockerHost: "tcp://127.0.0.1:2376",
UsageHint: usageHintMap["bash"],
Prefix: bashSetPfx,
Suffix: bashSetSfx,
Delimiter: bashSetDelim,
NoProxyVar: "no_proxy",
NoProxyValue: "0.0.0.0,127.0.0.1",
},
},
}

1
cmd/minikube/cmd/kubectl.go
View File

@ -67,6 +67,7 @@ var kubectlCmd = &cobra.Command{
glog.Infof("Running %s %v", path, args)
c := exec.Command(path, args...)
c.Stdin = os.Stdin
c.Stdout = os.Stdout
c.Stderr = os.Stderr
if err := c.Run(); err != nil {

32
cmd/minikube/cmd/mount.go
View File

@ -142,29 +142,29 @@ var mountCmd = &cobra.Command{
cfg.Options[parts[0]] = parts[1]
}
console.OutStyle("mounting", "Mounting host path %s into VM as %s ...", hostPath, vmPath)
console.OutStyle("mount-options", "Mount options:")
console.OutStyle("option", "Type: %s", cfg.Type)
console.OutStyle("option", "UID: %s", cfg.UID)
console.OutStyle("option", "GID: %s", cfg.GID)
console.OutStyle("option", "Version: %s", cfg.Version)
console.OutStyle("option", "MSize: %d", cfg.MSize)
console.OutStyle("option", "Mode: %o (%s)", cfg.Mode, cfg.Mode)
console.OutStyle("option", "Options: %s", cfg.Options)
console.OutStyle(console.Mounting, "Mounting host path %s into VM as %s ...", hostPath, vmPath)
console.OutStyle(console.MountOptions, "Mount options:")
console.OutStyle(console.Option, "Type: %s", cfg.Type)
console.OutStyle(console.Option, "UID: %s", cfg.UID)
console.OutStyle(console.Option, "GID: %s", cfg.GID)
console.OutStyle(console.Option, "Version: %s", cfg.Version)
console.OutStyle(console.Option, "MSize: %d", cfg.MSize)
console.OutStyle(console.Option, "Mode: %o (%s)", cfg.Mode, cfg.Mode)
console.OutStyle(console.Option, "Options: %s", cfg.Options)
// An escape valve to allow future hackers to try NFS, VirtFS, or other FS types.
if !supportedFilesystems[cfg.Type] {
console.OutLn("")
console.OutStyle("warning", "%s is not yet a supported filesystem. We will try anyways!", cfg.Type)
console.OutStyle(console.WarningType, "%s is not yet a supported filesystem. We will try anyways!", cfg.Type)
}
var wg sync.WaitGroup
if cfg.Type == nineP {
wg.Add(1)
go func() {
console.OutStyle("fileserver", "Userspace file server: ")
console.OutStyle(console.Fileserver, "Userspace file server: ")
ufs.StartServer(net.JoinHostPort(ip.String(), strconv.Itoa(port)), debugVal, hostPath)
console.OutStyle("stopped", "Userspace file server is shutdown")
console.OutStyle(console.Stopped, "Userspace file server is shutdown")
wg.Done()
}()
}
@ -180,10 +180,10 @@ var mountCmd = &cobra.Command{
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
go func() {
for sig := range c {
console.OutStyle("unmount", "Unmounting %s ...", vmPath)
console.OutStyle(console.Unmount, "Unmounting %s ...", vmPath)
err := cluster.Unmount(runner, vmPath)
if err != nil {
console.ErrStyle("failure", "Failed unmount: %v", err)
console.ErrStyle(console.FailureType, "Failed unmount: %v", err)
}
exit.WithCode(exit.Interrupted, "Exiting due to %s signal", sig)
}
@ -193,9 +193,9 @@ var mountCmd = &cobra.Command{
if err != nil {
exit.WithError("mount failed", err)
}
console.OutStyle("success", "Successfully mounted %s to %s", hostPath, vmPath)
console.OutStyle(console.SuccessType, "Successfully mounted %s to %s", hostPath, vmPath)
console.OutLn("")
console.OutStyle("notice", "NOTE: This process must stay alive for the mount to be accessible ...")
console.OutStyle(console.Notice, "NOTE: This process must stay alive for the mount to be accessible ...")
wg.Wait()
},
}

2
cmd/minikube/cmd/service_list.go
View File

@ -45,7 +45,7 @@ var serviceListCmd = &cobra.Command{
serviceURLs, err := service.GetServiceURLs(api, serviceListNamespace, serviceURLTemplate)
if err != nil {
console.Fatal("Failed to get service URL: %v", err)
console.ErrStyle("notice", "Check that minikube is running and that you have specified the correct namespace (-n flag) if required.")
console.ErrStyle(console.Notice, "Check that minikube is running and that you have specified the correct namespace (-n flag) if required.")
os.Exit(exit.Unavailable)
}

130
cmd/minikube/cmd/start.go
View File

@ -32,7 +32,6 @@ import (
"github.com/blang/semver"
"github.com/docker/machine/libmachine"
"github.com/docker/machine/libmachine/host"
"github.com/docker/machine/libmachine/state"
"github.com/golang/glog"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
@ -43,6 +42,7 @@ import (
cmdcfg "k8s.io/minikube/cmd/minikube/cmd/config"
cmdutil "k8s.io/minikube/cmd/util"
"k8s.io/minikube/pkg/minikube/bootstrapper"
"k8s.io/minikube/pkg/minikube/bootstrapper/kubeadm"
"k8s.io/minikube/pkg/minikube/cluster"
cfg "k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/minikube/console"
@ -51,6 +51,7 @@ import (
"k8s.io/minikube/pkg/minikube/exit"
"k8s.io/minikube/pkg/minikube/logs"
"k8s.io/minikube/pkg/minikube/machine"
"k8s.io/minikube/pkg/minikube/proxy"
pkgutil "k8s.io/minikube/pkg/util"
"k8s.io/minikube/pkg/version"
)
@ -102,9 +103,6 @@ var (
apiServerNames []string
apiServerIPs []net.IP
extraOptions pkgutil.ExtraOptionSlice
// proxyVars are variables we plumb through to the underlying container runtime
proxyVars = []string{"HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY"}
)
func init() {
@ -146,7 +144,8 @@ func init() {
startCmd.Flags().Var(&extraOptions, "extra-config",
`A set of key=value pairs that describe configuration that may be passed to different components.
The key should be '.' separated, and the first part before the dot is the component to apply the configuration to.
Valid components are: kubelet, kubeadm, apiserver, controller-manager, etcd, proxy, scheduler.`)
Valid components are: kubelet, kubeadm, apiserver, controller-manager, etcd, proxy, scheduler
Valid kubeadm parameters: `+fmt.Sprintf("%s, %s", strings.Join(kubeadm.KubeadmExtraArgsWhitelist[kubeadm.KubeadmCmdParam], ", "), strings.Join(kubeadm.KubeadmExtraArgsWhitelist[kubeadm.KubeadmConfigParam], ",")))
startCmd.Flags().String(uuid, "", "Provide VM UUID to restore MAC address (only supported with Hyperkit driver).")
startCmd.Flags().String(vpnkitSock, "", "Location of the VPNKit socket used for networking. If empty, disables Hyperkit VPNKitSock, if 'auto' uses Docker for Mac VPNKit connection, otherwise uses the specified VSock.")
startCmd.Flags().StringSlice(vsockPorts, []string{}, "List of guest VSock ports that should be exposed as sockets on the host (Only supported on with hyperkit now).")
@ -170,7 +169,7 @@ assumes you have already installed one of the VM drivers: virtualbox/parallels/v
// runStart handles the executes the flow of "minikube start"
func runStart(cmd *cobra.Command, args []string) {
console.OutStyle("happy", "minikube %s on %s (%s)", version.GetVersion(), runtime.GOOS, runtime.GOARCH)
console.OutStyle(console.Happy, "minikube %s on %s (%s)", version.GetVersion(), runtime.GOOS, runtime.GOARCH)
validateConfig()
oldConfig, err := cfg.Load()
@ -217,13 +216,18 @@ func runStart(cmd *cobra.Command, args []string) {
if err := CacheImagesInConfigFile(); err != nil {
exit.WithError("Failed to cache images", err)
}
console.OutStyle("check", "Download complete!")
console.OutStyle(console.Check, "Download complete!")
return
}
host, preexisting := startHost(m, config.MachineConfig)
ip := validateNetwork(host)
// Makes minikube node ip to bypass http(s) proxy. since it is local traffic.
err = proxy.ExcludeIP(ip)
if err != nil {
console.ErrStyle(console.FailureType, "Failed to set NO_PROXY Env. please Use `export NO_PROXY=$NO_PROXY,%s`.", ip)
}
// Save IP to configuration file for subsequent use
config.KubernetesConfig.NodeIP = ip
if err := saveConfig(config); err != nil {
@ -234,7 +238,15 @@ func runStart(cmd *cobra.Command, args []string) {
exit.WithError("Failed to get command runner", err)
}
cr := configureRuntimes(runner, k8sVersion)
cr := configureRuntimes(runner)
version, _ := cr.Version()
console.OutStyle(cr.Style(), "Configuring environment for Kubernetes %s on %s %s", k8sVersion, cr.Name(), version)
for _, v := range dockerOpt {
console.OutStyle(console.Option, "opt %s", v)
}
for _, v := range dockerEnv {
console.OutStyle(console.Option, "env %s", v)
}
// prepareHostEnvironment uses the downloaded images, so we need to wait for background task completion.
waitCacheImages(&cacheGroup)
@ -244,32 +256,32 @@ func runStart(cmd *cobra.Command, args []string) {
// The kube config must be update must come before bootstrapping, otherwise health checks may use a stale IP
kubeconfig := updateKubeConfig(host, &config)
bootstrapCluster(bs, cr, runner, config.KubernetesConfig, preexisting, isUpgrade)
apiserverPort := config.KubernetesConfig.NodePort
validateCluster(bs, cr, runner, ip, apiserverPort)
configureMounts()
if err = LoadCachedImagesInConfigFile(); err != nil {
console.Failure("Unable to load cached images from config file.")
}
if config.MachineConfig.VMDriver == constants.DriverNone {
console.OutStyle("starting-none", "Configuring local host environment ...")
console.OutStyle(console.StartingNone, "Configuring local host environment ...")
prepareNone()
}
if err := bs.WaitCluster(config.KubernetesConfig); err != nil {
exit.WithError("Wait failed", err)
}
showKubectlConnectInfo(kubeconfig)
}
func showKubectlConnectInfo(kubeconfig *pkgutil.KubeConfigSetup) {
if kubeconfig.KeepContext {
console.OutStyle("kubectl", "To connect to this cluster, use: kubectl --context=%s", kubeconfig.ClusterName)
console.OutStyle(console.Kubectl, "To connect to this cluster, use: kubectl --context=%s", kubeconfig.ClusterName)
} else {
console.OutStyle("ready", "Done! kubectl is now configured to use %q", cfg.GetMachineName())
console.OutStyle(console.Ready, "Done! kubectl is now configured to use %q", cfg.GetMachineName())
}
_, err := exec.LookPath("kubectl")
if err != nil {
console.OutStyle("tip", "For best results, install kubectl: https://kubernetes.io/docs/tasks/tools/install-kubectl/")
console.OutStyle(console.Tip, "For best results, install kubectl: https://kubernetes.io/docs/tasks/tools/install-kubectl/")
}
}
@ -337,6 +349,14 @@ func validateConfig() {
if viper.GetBool(hidden) && viper.GetString(vmDriver) != "kvm2" {
exit.Usage("Sorry, the --hidden feature is currently only supported with --vm-driver=kvm2")
}
// check that kubeadm extra args contain only whitelisted parameters
for param := range extraOptions.AsMap().Get(kubeadm.Kubeadm) {
if !pkgutil.ContainsString(kubeadm.KubeadmExtraArgsWhitelist[kubeadm.KubeadmCmdParam], param) &&
!pkgutil.ContainsString(kubeadm.KubeadmExtraArgsWhitelist[kubeadm.KubeadmConfigParam], param) {
exit.Usage("Sorry, the kubeadm.%s parameter is currently not supported by --extra-config", param)
}
}
}
// doCacheBinaries caches Kubernetes binaries in the foreground
@ -385,7 +405,7 @@ func generateConfig(cmd *cobra.Command, k8sVersion string) (cfg.Config, error) {
// Feed Docker our host proxy environment by default, so that it can pull images
if _, ok := r.(*cruntime.Docker); ok {
if !cmd.Flags().Changed("docker-env") {
for _, k := range proxyVars {
for _, k := range proxy.EnvVars {
if v := os.Getenv(k); v != "" {
dockerEnv = append(dockerEnv, fmt.Sprintf("%s=%s", k, v))
}
@ -396,7 +416,7 @@ func generateConfig(cmd *cobra.Command, k8sVersion string) (cfg.Config, error) {
repository := viper.GetString(imageRepository)
mirrorCountry := strings.ToLower(viper.GetString(imageMirrorCountry))
if strings.ToLower(repository) == "auto" || mirrorCountry != "" {
console.OutStyle("connectivity", "checking main repository and mirrors for images")
console.OutStyle(console.Connectivity, "checking main repository and mirrors for images")
found, autoSelectedRepository, err := selectImageRepository(mirrorCountry, k8sVersion)
if err != nil {
exit.WithError("Failed to check main repository and mirrors for images for images", err)
@ -414,7 +434,7 @@ func generateConfig(cmd *cobra.Command, k8sVersion string) (cfg.Config, error) {
}
if repository != "" {
console.OutStyle("success", "using image repository %s", repository)
console.OutStyle(console.SuccessType, "using image repository %s", repository)
}
cfg := cfg.Config{
@ -472,7 +492,7 @@ func prepareNone() {
console.OutLn("")
console.Warning("The 'none' driver provides limited isolation and may reduce system security and reliability.")
console.Warning("For more information, see:")
console.OutStyle("url", "https://github.com/kubernetes/minikube/blob/master/docs/vmdriver-none.md")
console.OutStyle(console.URL, "https://github.com/kubernetes/minikube/blob/master/docs/vmdriver-none.md")
console.OutLn("")
}
@ -483,11 +503,11 @@ func prepareNone() {
console.Warning("need to relocate them. For example, to overwrite your own settings:")
console.OutLn("")
console.OutStyle("command", "sudo mv %s/.kube %s/.minikube $HOME", home, home)
console.OutStyle("command", "sudo chown -R $USER $HOME/.kube $HOME/.minikube")
console.OutStyle(console.Command, "sudo mv %s/.kube %s/.minikube $HOME", home, home)
console.OutStyle(console.Command, "sudo chown -R $USER $HOME/.kube $HOME/.minikube")
console.OutLn("")
console.OutStyle("tip", "This can also be done automatically by setting the env var CHANGE_MINIKUBE_NONE_USER=true")
console.OutStyle(console.Tip, "This can also be done automatically by setting the env var CHANGE_MINIKUBE_NONE_USER=true")
}
if err := pkgutil.MaybeChownDirRecursiveToMinikubeUser(constants.GetMinipath()); err != nil {
@ -524,13 +544,19 @@ func validateNetwork(h *host.Host) string {
}
optSeen := false
for _, k := range proxyVars {
warnedOnce := false
for _, k := range proxy.EnvVars {
if v := os.Getenv(k); v != "" {
if !optSeen {
console.OutStyle("internet", "Found network options:")
console.OutStyle(console.Internet, "Found network options:")
optSeen = true
}
console.OutStyle("option", "%s=%s", k, v)
console.OutStyle(console.Option, "%s=%s", k, v)
ipExcluded := proxy.IsIPExcluded(ip) // Skip warning if minikube ip is already in NO_PROXY
if (k == "HTTP_PROXY" || k == "HTTPS_PROXY") && !ipExcluded && !warnedOnce {
console.Warning("You appear to be using a proxy, but your NO_PROXY environment does not include the minikube IP (%s). Please see https://github.com/kubernetes/minikube/blob/master/docs/http_proxy.md for more details", ip)
warnedOnce = true
}
}
}
@ -561,11 +587,11 @@ func validateKubernetesVersions(old *cfg.Config) (string, bool) {
if nvs.LT(ovs) {
nv = version.VersionPrefix + ovs.String()
console.ErrStyle("conflict", "Kubernetes downgrade is not supported, will continue to use %v", nv)
console.ErrStyle(console.Conflict, "Kubernetes downgrade is not supported, will continue to use %v", nv)
return nv, isUpgrade
}
if nvs.GT(ovs) {
console.OutStyle("thumbs-up", "minikube will upgrade the local cluster from Kubernetes %s to %s", ovs, nvs)
console.OutStyle(console.ThumbsUp, "minikube will upgrade the local cluster from Kubernetes %s to %s", ovs, nvs)
isUpgrade = true
}
return nv, isUpgrade
@ -578,7 +604,7 @@ func prepareHostEnvironment(api libmachine.API, kc cfg.KubernetesConfig) bootstr
exit.WithError("Failed to get bootstrapper", err)
}
for _, eo := range extraOptions {
console.OutStyle("option", "%s.%s=%s", eo.Component, eo.Key, eo.Value)
console.OutStyle(console.Option, "%s.%s=%s", eo.Component, eo.Key, eo.Value)
}
// Loads cached images, generates config files, download binaries
if err := bs.UpdateCluster(kc); err != nil {
@ -619,20 +645,12 @@ func updateKubeConfig(h *host.Host, c *cfg.Config) *pkgutil.KubeConfigSetup {
}
// configureRuntimes does what needs to happen to get a runtime going.
func configureRuntimes(runner cruntime.CommandRunner, k8sVersion string) cruntime.Manager {
func configureRuntimes(runner cruntime.CommandRunner) cruntime.Manager {
config := cruntime.Config{Type: viper.GetString(containerRuntime), Runner: runner}
cr, err := cruntime.New(config)
if err != nil {
exit.WithError(fmt.Sprintf("Failed runtime for %+v", config), err)
}
version, _ := cr.Version()
console.OutStyle(cr.Name(), "Configuring environment for Kubernetes %s on %s %s", k8sVersion, cr.Name(), version)
for _, v := range dockerOpt {
console.OutStyle("option", "opt %s", v)
}
for _, v := range dockerEnv {
console.OutStyle("option", "env %s", v)
}
err = cr.Enable()
if err != nil {
@ -648,61 +666,33 @@ func bootstrapCluster(bs bootstrapper.Bootstrapper, r cruntime.Manager, runner b
bsName := viper.GetString(cmdcfg.Bootstrapper)
if isUpgrade || !preexisting {
console.OutStyle("pulling", "Pulling images ...")
console.OutStyle(console.Pulling, "Pulling images ...")
if err := bs.PullImages(kc); err != nil {
console.OutStyle("failure", "Unable to pull images, which may be OK: %v", err)
console.OutStyle(console.FailureType, "Unable to pull images, which may be OK: %v", err)
}
}
if preexisting {
console.OutStyle("restarting", "Relaunching Kubernetes %s using %s ... ", kc.KubernetesVersion, bsName)
console.OutStyle(console.Restarting, "Relaunching Kubernetes %s using %s ... ", kc.KubernetesVersion, bsName)
if err := bs.RestartCluster(kc); err != nil {
exit.WithLogEntries("Error restarting cluster", err, logs.FindProblems(r, bs, runner))
}
return
}
console.OutStyle("launch", "Launching Kubernetes ... ")
console.OutStyle(console.Launch, "Launching Kubernetes ... ")
if err := bs.StartCluster(kc); err != nil {
exit.WithLogEntries("Error starting cluster", err, logs.FindProblems(r, bs, runner))
}
}
// validateCluster validates that the cluster is well-configured and healthy
func validateCluster(bs bootstrapper.Bootstrapper, r cruntime.Manager, runner bootstrapper.CommandRunner, ip string, apiserverPort int) {
k8sStat := func() (err error) {
st, err := bs.GetKubeletStatus()
if err != nil || st != state.Running.String() {
return &pkgutil.RetriableError{Err: fmt.Errorf("kubelet unhealthy: %v: %s", err, st)}
}
return nil
}
err := pkgutil.RetryAfter(20, k8sStat, 3*time.Second)
if err != nil {
exit.WithLogEntries("kubelet checks failed", err, logs.FindProblems(r, bs, runner))
}
aStat := func() (err error) {
st, err := bs.GetAPIServerStatus(net.ParseIP(ip), apiserverPort)
if err != nil || st != state.Running.String() {
return &pkgutil.RetriableError{Err: fmt.Errorf("apiserver status=%s err=%v", st, err)}
}
return nil
}
err = pkgutil.RetryAfter(30, aStat, 10*time.Second)
if err != nil {
exit.WithLogEntries("apiserver checks failed", err, logs.FindProblems(r, bs, runner))
}
console.OutLn("")
}
// configureMounts configures any requested filesystem mounts
func configureMounts() {
if !viper.GetBool(createMount) {
return
}
console.OutStyle("mounting", "Creating mount %s ...", viper.GetString(mountString))
console.OutStyle(console.Mounting, "Creating mount %s ...", viper.GetString(mountString))
path := os.Args[0]
mountDebugVal := 0
if glog.V(8) {

6
cmd/minikube/cmd/stop.go
View File

@ -53,7 +53,7 @@ itself, leaving all files intact. The cluster can be started again with the "sta
err = cluster.StopHost(api)
switch err := errors.Cause(err).(type) {
case mcnerror.ErrHostDoesNotExist:
console.OutStyle("meh", "%q VM does not exist, nothing to stop", profile)
console.OutStyle(console.Meh, "%q VM does not exist, nothing to stop", profile)
nonexistent = true
return nil
default:
@ -64,11 +64,11 @@ itself, leaving all files intact. The cluster can be started again with the "sta
exit.WithError("Unable to stop VM", err)
}
if !nonexistent {
console.OutStyle("stopped", "%q stopped.", profile)
console.OutStyle(console.Stopped, "%q stopped.", profile)
}
if err := cmdUtil.KillMountProcess(); err != nil {
console.OutStyle("warning", "Unable to kill mount process: %s", err)
console.OutStyle(console.WarningType, "Unable to kill mount process: %s", err)
}
machineName := pkg_config.GetMachineName()

4
cmd/minikube/cmd/update-context.go
View File

@ -49,9 +49,9 @@ var updateContextCmd = &cobra.Command{
exit.WithError("update config", err)
}
if updated {
console.OutStyle("celebrate", "%s IP has been updated to point at %s", machineName, ip)
console.OutStyle(console.Celebrate, "%s IP has been updated to point at %s", machineName, ip)
} else {
console.OutStyle("meh", "%s IP was already correctly configured for %s", machineName, ip)
console.OutStyle(console.Meh, "%s IP was already correctly configured for %s", machineName, ip)
}
},

28
deploy/addons/registry/registry-proxy.yaml.tmpl Normal file
View File

@ -0,0 +1,28 @@
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
labels:
kubernetes.io/minikube-addons: registry
addonmanager.kubernetes.io/mode: Reconcile
name: registry-proxy
namespace: kube-system
spec:
template:
metadata:
labels:
kubernetes.io/minikube-addons: registry
addonmanager.kubernetes.io/mode: Reconcile
spec:
containers:
- image: gcr.io/google_containers/kube-registry-proxy:0.4
imagePullPolicy: IfNotPresent
name: registry-proxy
ports:
- name: registry
containerPort: 80
hostPort: 5000
env:
- name: REGISTRY_HOST
value: registry.kube-system.svc.cluster.local
- name: REGISTRY_PORT
value: "80"

2
deploy/iso/minikube-iso/board/coreos/minikube/linux_defconfig
View File

@ -274,6 +274,8 @@ CONFIG_NET_CLS_BPF=m
CONFIG_NET_EMATCH=y
CONFIG_NET_CLS_ACT=y
CONFIG_NET_ACT_BPF=m
CONFIG_NET_ACT_MIRRED=m
CONFIG_NET_CLS_U32=m
CONFIG_OPENVSWITCH=m
CONFIG_VSOCKETS=m
CONFIG_VIRTIO_VSOCKETS=m

0
deploy/iso/minikube-iso/board/coreos/minikube/rootfs-overlay/etc/cni/net.d/.keep Normal file
View File

2
deploy/iso/minikube-iso/board/coreos/minikube/rootfs-overlay/etc/ssh/sshd_config
View File

@ -84,7 +84,7 @@ AuthorizedKeysFile .ssh/authorized_keys
#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
GatewayPorts yes
#X11Forwarding no
#X11DisplayOffset 10
#X11UseLocalhost yes

1
deploy/iso/minikube-iso/package/containerd-bin/containerd-bin.hash
View File

@ -1 +1,2 @@
sha256 8565a655345f3db2f7b78b77a2cd3103895229aa44cd0e4c710ba3559e344b5a v1.2.0.tar.gz
sha256 b92819bde71de947329814a3c649b8adb106cc03be16aae217b94297f4b843a1 v1.2.5.tar.gz

4
deploy/iso/minikube-iso/package/containerd-bin/containerd-bin.mk
View File

@ -3,8 +3,8 @@
# containerd
#
################################################################################
CONTAINERD_BIN_VERSION = v1.2.0
CONTAINERD_BIN_COMMIT = c4446665cb9c30056f4998ed953e6d4ff22c7c39
CONTAINERD_BIN_VERSION = v1.2.5
CONTAINERD_BIN_COMMIT = bb71b10fd8f58240ca47fbb579b9d1028eea7c84
CONTAINERD_BIN_SITE = https://github.com/containerd/containerd/archive
CONTAINERD_BIN_SOURCE = $(CONTAINERD_BIN_VERSION).tar.gz
CONTAINERD_BIN_DEPENDENCIES = host-go libgpgme

2
deploy/iso/minikube-iso/package/crio-bin/crio-bin.hash
View File

@ -3,3 +3,5 @@ sha256 9f79cee99e272c9cfc561ae31235d84d4da59fd5c8b3d3ab6623bf9a92d90c5a v1.10.0.
sha256 09e53fd550f4f10108879131ee6b8ef1c367ce71a73dcf6350c4cc898751d8c1 v1.11.8.tar.gz
sha256 92588998dbb79002c38f65f84602b5659f0d0ef1cd36b1a568a2e40269b66816 v1.13.0.tar.gz
sha256 48e7cf64a757d62a3edf214e1b93b74d99f090ca924f956ede2494a260eab2db v1.13.1.tar.gz
sha256 7435c4745017f06c260973b049440d924efe65b0df008d14175dfb8f5e23b599 v1.14.0.tar.gz
sha256 1f6f72b1f89d4286b2d5b54a48f4d5ed4c0c01065d484635dcb343a706feb743 v1.14.1.tar.gz

5
deploy/iso/minikube-iso/package/crio-bin/crio-bin.mk
View File

@ -4,7 +4,8 @@
#
################################################################################
CRIO_BIN_VERSION = v1.13.1
CRIO_BIN_VERSION = v1.14.1
CRIO_BIN_COMMIT = b7644f67e6383cc862b3e37fb74fba334b0b2721
CRIO_BIN_SITE = https://github.com/kubernetes-sigs/cri-o/archive
CRIO_BIN_SOURCE = $(CRIO_BIN_VERSION).tar.gz
CRIO_BIN_DEPENDENCIES = host-go libgpgme
@ -28,7 +29,7 @@ endef
define CRIO_BIN_BUILD_CMDS
mkdir -p $(@D)/bin
$(CRIO_BIN_ENV) $(MAKE) $(TARGET_CONFIGURE_OPTS) -C $(@D) PREFIX=/usr binaries
$(CRIO_BIN_ENV) $(MAKE) $(TARGET_CONFIGURE_OPTS) -C $(@D) GIT_COMMIT=$(CRIO_BIN_COMMIT) PREFIX=/usr binaries
endef
define CRIO_BIN_INSTALL_TARGET_CMDS

310
deploy/iso/minikube-iso/package/crio-bin/crio.conf
View File

@ -1,161 +1,269 @@
# The "crio" table contains all of the server options.
# The CRI-O configuration file specifies all of the available configuration
# options and command-line flags for the crio(8) OCI Kubernetes Container Runtime
# daemon, but in a TOML format that can be more easily modified and versioned.
#
# Please refer to crio.conf(5) for details of all configuration options.
# CRI-O reads its storage defaults from the containers-storage.conf(5) file
# located at /etc/containers/storage.conf. Modify this storage configuration if
# you want to change the system's defaults. If you want to modify storage just
# for CRI-O, you can change the storage configuration options here.
[crio]
# root is a path to the "root directory". CRIO stores all of its data,
# including container images, in this directory.
# Path to the "root directory". CRI-O stores all of its data, including
# containers images, in this directory.
root = "/var/lib/containers/storage"
# run is a path to the "run directory". CRIO stores all of its state
# in this directory.
# Path to the "run directory". CRI-O stores all of its state in this directory.
runroot = "/var/run/containers/storage"
# storage_driver select which storage driver is used to manage storage
# of images and containers.
# Storage driver used to manage the storage of images and containers. Please
# refer to containers-storage.conf(5) to see all available storage drivers.
storage_driver = "overlay"
# storage_option is used to pass an option to the storage driver.
storage_option = [
]
# List to pass options to the storage driver. Please refer to
# containers-storage.conf(5) to see all available storage options.
#storage_option = [
#]
# The "crio.api" table contains settings for the kubelet/gRPC
# interface (which is also used by crioctl).
[crio.api]
# listen is the path to the AF_LOCAL socket on which crio will listen.
listen = "/var/run/crio/crio.sock"
# stream_address is the IP address on which the stream server will listen
stream_address = ""
# stream_port is the port on which the stream server will listen
stream_port = "10010"
# file_locking is whether file-based locking will be used instead of
# in-memory locking
# If set to false, in-memory locking will be used instead of file-based locking.
file_locking = true
# The "crio.runtime" table contains settings pertaining to the OCI
# runtime used and options for how to set up and manage the OCI runtime.
# Path to the lock file.
file_locking_path = "/run/crio.lock"
# The crio.api table contains settings for the kubelet/gRPC interface.
[crio.api]
# Path to AF_LOCAL socket on which CRI-O will listen.
listen = "/var/run/crio/crio.sock"
# IP address on which the stream server will listen.
stream_address = "127.0.0.1"
# The port on which the stream server will listen.
stream_port = "0"
# Enable encrypted TLS transport of the stream server.
stream_enable_tls = false
# Path to the x509 certificate file used to serve the encrypted stream. This
# file can change, and CRI-O will automatically pick up the changes within 5
# minutes.
stream_tls_cert = ""
# Path to the key file used to serve the encrypted stream. This file can
# change, and CRI-O will automatically pick up the changes within 5 minutes.
stream_tls_key = ""
# Path to the x509 CA(s) file used to verify and authenticate client
# communication with the encrypted stream. This file can change, and CRI-O will
# automatically pick up the changes within 5 minutes.
stream_tls_ca = ""
# Maximum grpc send message size in bytes. If not set or <=0, then CRI-O will default to 16 * 1024 * 1024.
grpc_max_send_msg_size = 16777216
# Maximum grpc receive message size. If not set or <= 0, then CRI-O will default to 16 * 1024 * 1024.
grpc_max_recv_msg_size = 16777216
# The crio.runtime table contains settings pertaining to the OCI runtime used
# and options for how to set up and manage the OCI runtime.
[crio.runtime]
# runtime is the OCI compatible runtime used for trusted container workloads.
# This is a mandatory setting as this runtime will be the default one
# and will also be used for untrusted container workloads if
# runtime_untrusted_workload is not set.
runtime = "/usr/bin/runc"
# A list of ulimits to be set in containers by default, specified as
# "<ulimit name>=<soft limit>:<hard limit>", for example:
# "nofile=1024:2048"
# If nothing is set here, settings will be inherited from the CRI-O daemon
#default_ulimits = [
#]
# runtime_untrusted_workload is the OCI compatible runtime used for untrusted
# container workloads. This is an optional setting, except if
# default_container_trust is set to "untrusted".
runtime_untrusted_workload = ""
# default_runtime is the _name_ of the OCI runtime to be used as the default.
# The name is matched against the runtimes map below.
default_runtime = "runc"
# default_workload_trust is the default level of trust crio puts in container
# workloads. It can either be "trusted" or "untrusted", and the default
# is "trusted".
# Containers can be run through different container runtimes, depending on
# the trust hints we receive from kubelet:
# - If kubelet tags a container workload as untrusted, crio will try first to
# run it through the untrusted container workload runtime. If it is not set,
# crio will use the trusted runtime.
# - If kubelet does not provide any information about the container workload trust
# level, the selected runtime will depend on the default_container_trust setting.
# If it is set to "untrusted", then all containers except for the host privileged
# ones, will be run by the runtime_untrusted_workload runtime. Host privileged
# containers are by definition trusted and will always use the trusted container
# runtime. If default_container_trust is set to "trusted", crio will use the trusted
# container runtime for all containers.
default_workload_trust = "trusted"
# no_pivot instructs the runtime to not use pivot_root, but instead use MS_MOVE
# If true, the runtime will not use pivot_root, but instead use MS_MOVE.
no_pivot = true
# conmon is the path to conmon binary, used for managing the runtime.
# Path to the conmon binary, used for monitoring the OCI runtime.
conmon = "/usr/libexec/crio/conmon"
# conmon_env is the environment variable list for conmon process,
# used for passing necessary environment variable to conmon or runtime.
# Environment variable list for the conmon process, used for passing necessary
# environment variables to conmon or the runtime.
conmon_env = [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
]
# selinux indicates whether or not SELinux will be used for pod
# separation on the host. If you enable this flag, SELinux must be running
# on the host.
# If true, SELinux will be used for pod separation on the host.
selinux = false
# seccomp_profile is the seccomp json profile path which is used as the
# default for the runtime.
# Path to the seccomp.json profile which is used as the default seccomp profile
# for the runtime.
seccomp_profile = "/etc/crio/seccomp.json"
# apparmor_profile is the apparmor profile name which is used as the
# default for the runtime.
# Used to change the name of the default AppArmor profile of CRI-O. The default
# profile name is "crio-default-" followed by the version string of CRI-O.
apparmor_profile = "crio-default"
# cgroup_manager is the cgroup management implementation to be used
# for the runtime.
# Cgroup management implementation used for the runtime.
cgroup_manager = "cgroupfs"
# hooks_dir_path is the oci hooks directory for automatically executed hooks
hooks_dir_path = "/usr/share/containers/oci/hooks.d"
# List of default capabilities for containers. If it is empty or commented out,
# only the capabilities defined in the containers json file by the user/kube
# will be added.
default_capabilities = [
"CHOWN",
"DAC_OVERRIDE",
"FSETID",
"FOWNER",
"NET_RAW",
"SETGID",
"SETUID",
"SETPCAP",
"NET_BIND_SERVICE",
"SYS_CHROOT",
"KILL",
]
# default_mounts is the mounts list to be mounted for the container when created
# List of default sysctls. If it is empty or commented out, only the sysctls
# defined in the container json file by the user/kube will be added.
default_sysctls = [
]
# List of additional devices. specified as
# "<device-on-host>:<device-on-container>:<permissions>", for example: "--device=/dev/sdc:/dev/xvdc:rwm".
#If it is empty or commented out, only the devices
# defined in the container json file by the user/kube will be added.
additional_devices = [
]
# Path to OCI hooks directories for automatically executed hooks.
hooks_dir = [
]
# List of default mounts for each container. **Deprecated:** this option will
# be removed in future versions in favor of default_mounts_file.
default_mounts = [
]
# pids_limit is the number of processes allowed in a container
# Path to the file specifying the defaults mounts for each container. The
# format of the config is /SRC:/DST, one mount per line. Notice that CRI-O reads
# its default mounts from the following two files:
#
# 1) /etc/containers/mounts.conf (i.e., default_mounts_file): This is the
# override file, where users can either add in their own default mounts, or
# override the default mounts shipped with the package.
#
# 2) /usr/share/containers/mounts.conf: This is the default file read for
# mounts. If you want CRI-O to read from a different, specific mounts file,
# you can change the default_mounts_file. Note, if this is done, CRI-O will
# only add mounts it finds in this file.
#
#default_mounts_file = ""
# Maximum number of processes allowed in a container.
pids_limit = 1024
# enable using a shared PID namespace for containers in a pod
enable_shared_pid_namespace = false
# log_size_max is the max limit for the container log size in bytes.
# Negative values indicate that no limit is imposed.
# Maximum sized allowed for the container log file. Negative numbers indicate
# that no size limit is imposed. If it is positive, it must be >= 8192 to
# match/exceed conmon's read buffer. The file is truncated and re-opened so the
# limit is never exceeded.
log_size_max = -1
# The "crio.image" table contains settings pertaining to the
# management of OCI images.
# Whether container output should be logged to journald in addition to the kuberentes log file
log_to_journald = false
# Path to directory in which container exit files are written to by conmon.
container_exits_dir = "/var/run/crio/exits"
# Path to directory for container attach sockets.
container_attach_socket_dir = "/var/run/crio"
# If set to true, all containers will run in read-only mode.
read_only = false
# Changes the verbosity of the logs based on the level it is set to. Options
# are fatal, panic, error, warn, info, and debug.
log_level = "error"
# The UID mappings for the user namespace of each container. A range is
# specified in the form containerUID:HostUID:Size. Multiple ranges must be
# separated by comma.
uid_mappings = ""
# The GID mappings for the user namespace of each container. A range is
# specified in the form containerGID:HostGID:Size. Multiple ranges must be
# separated by comma.
gid_mappings = ""
# The minimal amount of time in seconds to wait before issuing a timeout
# regarding the proper termination of the container.
ctr_stop_timeout = 0
# The "crio.runtime.runtimes" table defines a list of OCI compatible runtimes.
# The runtime to use is picked based on the runtime_handler provided by the CRI.
# If no runtime_handler is provided, the runtime will be picked based on the level
# of trust of the workload.
[crio.runtime.runtimes.runc]
runtime_path = "/usr/bin/runc"
runtime_type = "oci"
# The crio.image table contains settings pertaining to the management of OCI images.
#
# CRI-O reads its configured registries defaults from the system wide
# containers-registries.conf(5) located in /etc/containers/registries.conf. If
# you want to modify just CRI-O, you can change the registries configuration in
# this file. Otherwise, leave insecure_registries and registries commented out to
# use the system's defaults from /etc/containers/registries.conf.
[crio.image]
# default_transport is the prefix we try prepending to an image name if the
# image name as we receive it can't be parsed as a valid source reference
# Default transport for pulling images from a remote container storage.
default_transport = "docker://"
# pause_image is the image which we use to instantiate infra containers.
# The image used to instantiate infra containers.
pause_image = "k8s.gcr.io/pause:3.1"
# pause_command is the command to run in a pause_image to have a container just
# sit there. If the image contains the necessary information, this value need
# not be specified.
# If not empty, the path to a docker/config.json-like file containing credentials
# necessary for pulling the image specified by pause_image above.
pause_image_auth_file = ""
# The command to run to have a container stay in the paused state.
pause_command = "/pause"
# signature_policy is the name of the file which decides what sort of policy we
# use when deciding whether or not to trust an image that we've pulled.
# Outside of testing situations, it is strongly advised that this be left
# unspecified so that the default system-wide policy will be used.
# Path to the file which decides what sort of policy we use when deciding
# whether or not to trust an image that we've pulled. It is not recommended that
# this option be used, as the default behavior of using the system-wide default
# policy (i.e., /etc/containers/policy.json) is most often preferred. Please
# refer to containers-policy.json(5) for more details.
signature_policy = ""
# image_volumes controls how image volumes are handled.
# The valid values are mkdir and ignore.
# Controls how image volumes are handled. The valid values are mkdir, bind and
# ignore; the latter will ignore volumes entirely.
image_volumes = "mkdir"
# insecure_registries is used to skip TLS verification when pulling images.
insecure_registries = [
]
# registries is used to specify a comma separated list of registries to be used
# when pulling an unqualified image (e.g. fedora:rawhide).
# List of registries to be used when pulling an unqualified image (e.g.,
# "alpine:latest"). By default, registries is set to "docker.io" for
# compatibility reasons. Depending on your workload and usecase you may add more
# registries (e.g., "quay.io", "registry.fedoraproject.org",
# "registry.opensuse.org", etc.).
registries = [
"docker.io"
]
# The "crio.network" table contains settings pertaining to the
# management of CNI plugins.
# The crio.network table containers settings pertaining to the management of
# CNI plugins.
[crio.network]
# network_dir is where CNI network configuration
# files are stored.
# Path to the directory where CNI configuration files are located.
network_dir = "/etc/cni/net.d/"
# plugin_dir is where CNI plugin binaries are stored.
plugin_dir = "/opt/cni/bin/"
# Paths to directories where CNI plugin binaries are located.
plugin_dir = [
"/opt/cni/bin/",
]

268
deploy/iso/minikube-iso/package/crio-bin/crio.conf.default Normal file
View File

@ -0,0 +1,268 @@
# The CRI-O configuration file specifies all of the available configuration
# options and command-line flags for the crio(8) OCI Kubernetes Container Runtime
# daemon, but in a TOML format that can be more easily modified and versioned.
#
# Please refer to crio.conf(5) for details of all configuration options.
# CRI-O reads its storage defaults from the containers-storage.conf(5) file
# located at /etc/containers/storage.conf. Modify this storage configuration if
# you want to change the system's defaults. If you want to modify storage just
# for CRI-O, you can change the storage configuration options here.
[crio]
# Path to the "root directory". CRI-O stores all of its data, including
# containers images, in this directory.
#root = "/var/lib/containers/storage"
# Path to the "run directory". CRI-O stores all of its state in this directory.
#runroot = "/var/run/containers/storage"
# Storage driver used to manage the storage of images and containers. Please
# refer to containers-storage.conf(5) to see all available storage drivers.
#storage_driver = "overlay"
# List to pass options to the storage driver. Please refer to
# containers-storage.conf(5) to see all available storage options.
#storage_option = [
#]
# If set to false, in-memory locking will be used instead of file-based locking.
file_locking = true
# Path to the lock file.
file_locking_path = "/run/crio.lock"
# The crio.api table contains settings for the kubelet/gRPC interface.
[crio.api]
# Path to AF_LOCAL socket on which CRI-O will listen.
listen = "/var/run/crio/crio.sock"
# IP address on which the stream server will listen.
stream_address = "127.0.0.1"
# The port on which the stream server will listen.
stream_port = "0"
# Enable encrypted TLS transport of the stream server.
stream_enable_tls = false
# Path to the x509 certificate file used to serve the encrypted stream. This
# file can change, and CRI-O will automatically pick up the changes within 5
# minutes.
stream_tls_cert = ""
# Path to the key file used to serve the encrypted stream. This file can
# change, and CRI-O will automatically pick up the changes within 5 minutes.
stream_tls_key = ""
# Path to the x509 CA(s) file used to verify and authenticate client
# communication with the encrypted stream. This file can change, and CRI-O will
# automatically pick up the changes within 5 minutes.
stream_tls_ca = ""
# Maximum grpc send message size in bytes. If not set or <=0, then CRI-O will default to 16 * 1024 * 1024.
grpc_max_send_msg_size = 16777216
# Maximum grpc receive message size. If not set or <= 0, then CRI-O will default to 16 * 1024 * 1024.
grpc_max_recv_msg_size = 16777216
# The crio.runtime table contains settings pertaining to the OCI runtime used
# and options for how to set up and manage the OCI runtime.
[crio.runtime]
# A list of ulimits to be set in containers by default, specified as
# "<ulimit name>=<soft limit>:<hard limit>", for example:
# "nofile=1024:2048"
# If nothing is set here, settings will be inherited from the CRI-O daemon
#default_ulimits = [
#]
# default_runtime is the _name_ of the OCI runtime to be used as the default.
# The name is matched against the runtimes map below.
default_runtime = "runc"
# If true, the runtime will not use pivot_root, but instead use MS_MOVE.
no_pivot = false
# Path to the conmon binary, used for monitoring the OCI runtime.
conmon = "/usr/local/libexec/crio/conmon"
# Environment variable list for the conmon process, used for passing necessary
# environment variables to conmon or the runtime.
conmon_env = [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
]
# If true, SELinux will be used for pod separation on the host.
selinux = false
# Path to the seccomp.json profile which is used as the default seccomp profile
# for the runtime.
seccomp_profile = "/etc/crio/seccomp.json"
# Used to change the name of the default AppArmor profile of CRI-O. The default
# profile name is "crio-default-" followed by the version string of CRI-O.
apparmor_profile = "crio-default"
# Cgroup management implementation used for the runtime.
cgroup_manager = "cgroupfs"
# List of default capabilities for containers. If it is empty or commented out,
# only the capabilities defined in the containers json file by the user/kube
# will be added.
default_capabilities = [
"CHOWN",
"DAC_OVERRIDE",
"FSETID",
"FOWNER",
"NET_RAW",
"SETGID",
"SETUID",
"SETPCAP",
"NET_BIND_SERVICE",
"SYS_CHROOT",
"KILL",
]
# List of default sysctls. If it is empty or commented out, only the sysctls
# defined in the container json file by the user/kube will be added.
default_sysctls = [
]
# List of additional devices. specified as
# "<device-on-host>:<device-on-container>:<permissions>", for example: "--device=/dev/sdc:/dev/xvdc:rwm".
#If it is empty or commented out, only the devices
# defined in the container json file by the user/kube will be added.
additional_devices = [
]
# Path to OCI hooks directories for automatically executed hooks.
hooks_dir = [
]
# List of default mounts for each container. **Deprecated:** this option will
# be removed in future versions in favor of default_mounts_file.
default_mounts = [
]
# Path to the file specifying the defaults mounts for each container. The
# format of the config is /SRC:/DST, one mount per line. Notice that CRI-O reads
# its default mounts from the following two files:
#
# 1) /etc/containers/mounts.conf (i.e., default_mounts_file): This is the
# override file, where users can either add in their own default mounts, or
# override the default mounts shipped with the package.
#
# 2) /usr/share/containers/mounts.conf: This is the default file read for
# mounts. If you want CRI-O to read from a different, specific mounts file,
# you can change the default_mounts_file. Note, if this is done, CRI-O will
# only add mounts it finds in this file.
#
#default_mounts_file = ""
# Maximum number of processes allowed in a container.
pids_limit = 1024
# Maximum sized allowed for the container log file. Negative numbers indicate
# that no size limit is imposed. If it is positive, it must be >= 8192 to
# match/exceed conmon's read buffer. The file is truncated and re-opened so the
# limit is never exceeded.
log_size_max = -1
# Whether container output should be logged to journald in addition to the kuberentes log file
log_to_journald = false
# Path to directory in which container exit files are written to by conmon.
container_exits_dir = "/var/run/crio/exits"
# Path to directory for container attach sockets.
container_attach_socket_dir = "/var/run/crio"
# If set to true, all containers will run in read-only mode.
read_only = false
# Changes the verbosity of the logs based on the level it is set to. Options
# are fatal, panic, error, warn, info, and debug.
log_level = "error"
# The UID mappings for the user namespace of each container. A range is
# specified in the form containerUID:HostUID:Size. Multiple ranges must be
# separated by comma.
uid_mappings = ""
# The GID mappings for the user namespace of each container. A range is
# specified in the form containerGID:HostGID:Size. Multiple ranges must be
# separated by comma.
gid_mappings = ""
# The minimal amount of time in seconds to wait before issuing a timeout
# regarding the proper termination of the container.
ctr_stop_timeout = 0
# The "crio.runtime.runtimes" table defines a list of OCI compatible runtimes.
# The runtime to use is picked based on the runtime_handler provided by the CRI.
# If no runtime_handler is provided, the runtime will be picked based on the level
# of trust of the workload.
[crio.runtime.runtimes.runc]
runtime_path = "/usr/bin/runc"
runtime_type = "oci"
# The crio.image table contains settings pertaining to the management of OCI images.
#
# CRI-O reads its configured registries defaults from the system wide
# containers-registries.conf(5) located in /etc/containers/registries.conf. If
# you want to modify just CRI-O, you can change the registries configuration in
# this file. Otherwise, leave insecure_registries and registries commented out to
# use the system's defaults from /etc/containers/registries.conf.
[crio.image]
# Default transport for pulling images from a remote container storage.
default_transport = "docker://"
# The image used to instantiate infra containers.
pause_image = "k8s.gcr.io/pause:3.1"
# If not empty, the path to a docker/config.json-like file containing credentials
# necessary for pulling the image specified by pause_image above.
pause_image_auth_file = ""
# The command to run to have a container stay in the paused state.
pause_command = "/pause"
# Path to the file which decides what sort of policy we use when deciding
# whether or not to trust an image that we've pulled. It is not recommended that
# this option be used, as the default behavior of using the system-wide default
# policy (i.e., /etc/containers/policy.json) is most often preferred. Please
# refer to containers-policy.json(5) for more details.
signature_policy = ""
# Controls how image volumes are handled. The valid values are mkdir, bind and
# ignore; the latter will ignore volumes entirely.
image_volumes = "mkdir"
# List of registries to be used when pulling an unqualified image (e.g.,
# "alpine:latest"). By default, registries is set to "docker.io" for
# compatibility reasons. Depending on your workload and usecase you may add more
# registries (e.g., "quay.io", "registry.fedoraproject.org",
# "registry.opensuse.org", etc.).
#registries = [
# ]
# The crio.network table containers settings pertaining to the management of
# CNI plugins.
[crio.network]
# Path to the directory where CNI configuration files are located.
network_dir = "/etc/cni/net.d/"
# Paths to directories where CNI plugin binaries are located.
plugin_dir = [
"/opt/cni/bin/",
]

2
deploy/iso/minikube-iso/package/docker-bin/docker-bin.hash
View File

@ -8,3 +8,5 @@ sha256 1270dce1bd7e1838d62ae21d2505d87f16efc1d9074645571daaefdfd0c14054 docker-
sha256 83be159cf0657df9e1a1a4a127d181725a982714a983b2bdcc0621244df93687 docker-18.06.1-ce.tgz
sha256 a979d9a952fae474886c7588da692ee00684cb2421d2c633c7ed415948cf0b10 docker-18.06.2-ce.tgz
sha256 346f9394393ee8db5f8bd1e229ee9d90e5b36931bdd754308b2ae68884dd6822 docker-18.06.3-ce.tgz
sha256 99ca9395e9c7ffbf75537de71aa828761f492491d02bc6e29db2920fa582c6c5 docker-18.09.5.tgz
sha256 1f3f6774117765279fce64ee7f76abbb5f260264548cf80631d68fb2d795bb09 docker-18.09.6.tgz

19
deploy/iso/minikube-iso/package/docker-bin/docker-bin.mk
View File

@ -4,7 +4,7 @@
#
################################################################################
DOCKER_BIN_VERSION = 18.06.3-ce
DOCKER_BIN_VERSION = 18.09.6
DOCKER_BIN_SITE = https://download.docker.com/linux/static/stable/x86_64
DOCKER_BIN_SOURCE = docker-$(DOCKER_BIN_VERSION).tgz
@ -17,24 +17,11 @@ define DOCKER_BIN_INSTALL_TARGET_CMDS
$(@D)/docker \
$(TARGET_DIR)/bin/docker
$(INSTALL) -D -m 0755 \
$(@D)/docker-containerd-shim \
$(TARGET_DIR)/bin/docker-containerd-shim
# TODO(tstromberg): Remove once we can upgrade to docker 18.09, which can call standard containerd.
$(INSTALL) -D -m 0755 \
$(@D)/docker-containerd \
$(TARGET_DIR)/bin/docker-containerd
# As of 2019-05, we use upstream containerd so that we may update it independently of docker.
# As of 2019-01, we use upstream runc so that we may update it independently of docker.
# TODO(tstromberg): Remove once we can upgrade to docker 18.09, which can call standard runc.
$(INSTALL) -D -m 0755 \
$(@D)/docker-runc \
$(TARGET_DIR)/bin/docker-runc.orig
$(INSTALL) -D -m 0755 \
$(@D)/docker-containerd-ctr \
$(TARGET_DIR)/bin/docker-containerd-ctr
# As of 2019-05, we use upstream ctr so that we may update it independently of docker.
$(INSTALL) -D -m 0755 \
$(@D)/dockerd \

3
deploy/iso/minikube-iso/package/podman/buildah.profile Normal file
View File

@ -0,0 +1,3 @@
# BUILDAH_NOPIVOT=true disables pivot_root in Buildah, using MS_MOVE instead.
# (Buildah is used by Podman for building container images using a Dockerfile)
export BUILDAH_NOPIVOT=true

3
deploy/iso/minikube-iso/package/podman/podman.hash
View File

@ -3,3 +3,6 @@ sha256 004d2695cf3de1802b3a1fc70bb03115e33ffe943477c49558b682313cb4fca2 v0.6.1.t
sha256 0b478f4918b984480b2f24ac5df1f45e31bca798dd8e2364bb56b2810b2ccff6 v0.7.4.tar.gz
sha256 065d2ef4cbbbcb87a7be5e1b3adb36d932819bafae7ccc26ad80f653b4328692 v0.9.3.tar.gz
sha256 f7a462563dd587208eff3c3c0689bc4d01071a8f7933bec2a13126be123f63a8 v1.0.0.tar.gz
sha256 b4d6843e13c0f2f1557ab20b4364bb42ea9102442ee3f1be806c06128aa687d8 v1.2.0.tar.gz
sha256 548a82b62ff183ca9e68f164779d6ded94fce07261242ceb96faa267fcecf56b v1.3.0.tar.gz
sha256 7ccbfa80900cd438468986911038da4c38382ef507a2fa717217faefb738fbc4 v1.3.1.tar.gz

5
deploy/iso/minikube-iso/package/podman/podman.mk
View File

@ -1,5 +1,5 @@
PODMAN_VERSION = v1.0.0
PODMAN_COMMIT = 82e80110c3f2d8728745c47e340f3bee4d408846
PODMAN_VERSION = v1.3.1
PODMAN_COMMIT = 7210727e205c333af9a2d0ed0bb66adcf92a6369
PODMAN_SITE = https://github.com/containers/libpod/archive
PODMAN_SOURCE = $(PODMAN_VERSION).tar.gz
PODMAN_LICENSE = Apache-2.0
@ -29,6 +29,7 @@ endef
define PODMAN_INSTALL_TARGET_CMDS
$(INSTALL) -Dm755 $(@D)/bin/podman $(TARGET_DIR)/usr/bin/podman
$(INSTALL) -Dm644 $(BR2_EXTERNAL_MINIKUBE_PATH)/package/podman/buildah.profile $(TARGET_DIR)/etc/profile.d/podman.sh
endef
$(eval $(generic-package))

1
deploy/iso/minikube-iso/package/runc-master/runc-master.hash
View File

@ -3,3 +3,4 @@ sha256 fc780966c4d70c275a87930e93cda4210e63a490eabfb0fa5f2fe70be6dcdc58 0fdc908b
sha256 9318fa9de6e3b2c89760f08d73bf718c97c93d683611716e024d2f3283c96d90 c1e454b2a1bfb0f0ebd9e621a1433f98f9a8d4b0.tar.gz
sha256 a960decadf6bd5d3cee1ca7b94455d37cc921c964061428bd9f3dd17a13c8bb3 6635b4f0c6af3810594d2770f662f34ddc15b40d.tar.gz
sha256 ad41ae930059fef18de1926cd78e00474c89290248fecdcc0e431c8aefee1deb 0a012df867a2d525f62a146d8ebdf2e6ab8a5ad5.tar.gz
sha256 e52c5d7365b2b9048f977bac8f06bf626dccb4d816d0947ec8523f543272f4ff 2b18fe1d885ee5083ef9f0838fee39b62d653e30.tar.gz

4
deploy/iso/minikube-iso/package/runc-master/runc-master.mk
View File

@ -4,8 +4,8 @@
#
################################################################################
# HEAD as of 2019-02-13
RUNC_MASTER_VERSION = 0a012df867a2d525f62a146d8ebdf2e6ab8a5ad5
# HEAD as of 2019-03-07
RUNC_MASTER_VERSION = 2b18fe1d885ee5083ef9f0838fee39b62d653e30
RUNC_MASTER_SITE = https://github.com/opencontainers/runc/archive
RUNC_MASTER_SOURCE = $(RUNC_MASTER_VERSION).tar.gz
RUNC_MASTER_LICENSE = Apache-2.0

8
deploy/minikube/releases.json
View File

@ -1,4 +1,12 @@
[
{
"name": "v1.1.0",
"checksums": {
"darwin": "3f635089d93ba7bd193b8f015bfa9434b810256daa8ced03816c7f0df7b06a96",
"linux": "43ce88a10c7b57f5e4f83e5a1485e13e4ad537bfb20f7d57364f66450b872634",
"windows": "2a8e95fbe4ec9b32c603767f16b731b5f09ddd04c3e43f6bbe7fffd22afbf811"
}
},
{
"name": "v1.0.1",
"checksums": {

4
docs/README.md
View File

@ -26,12 +26,16 @@
* **Reusing the Docker Daemon** ([reusing_the_docker_daemon.md](reusing_the_docker_daemon.md)): How to point your docker CLI to the docker daemon running inside minikube
* **Building images within the VM** ([building_images_within_the_vm.md](building_images_within_the_vm.md)): How to build a container image within the minikube VM
#### Storage
* **Persistent Volumes** ([persistent_volumes.md](persistent_volumes.md)): Persistent Volumes in Minikube and persisted locations in the VM
* **Host Folder Mounting** ([host_folder_mount.md](host_folder_mount.md)): How to mount your files from your host into the minikube VM
* **Syncing files into the VM** ([syncing-files.md](syncing-files.md)): How to sync files from your host into the minikube VM
#### Networking
* **HTTP Proxy** ([http_proxy.md](http_proxy.md)): Instruction on how to run minikube behind a HTTP Proxy

31
docs/building_images_within_the_vm.md Normal file
View File

@ -0,0 +1,31 @@
# Building images within the VM
When using a single VM of Kubernetes it's really handy to build inside the VM; as this means you don't have to build on your host machine and push the image into a docker registry - you can just build inside the same machine as minikube which speeds up local experiments.
## Docker (containerd)
For Docker, you can either set up your host docker client to communicate by [reusing the docker daemon](reusing_the_docker_daemon.md).
Or you can use `minikube ssh` to connect to the virtual machine, and run the `docker build` there:
```shell
docker build
```
For more information on the `docker build` command, read the [Docker documentation](https://docs.docker.com/engine/reference/commandline/build/) (docker.com).
## Podman (cri-o)
For Podman, there is no daemon running. The processes are started by the user, monitored by `conmon`.
So you need to use `minikube ssh`, and you will also make sure to run the command as the root user:
```shell
sudo -E podman build
```
For more information on the `podman build` command, read the [Podman documentation](https://github.com/containers/libpod/blob/master/docs/podman-build.1.md) (podman.io).
## Build context
For the build context you can use any directory on the virtual machine, or any address on the network.

8
docs/contributors/releasing_minikube.md
View File

@ -54,12 +54,8 @@ Merge the output into CHANGELOG.md. See [PR#3175](https://github.com/kubernetes/
## Tag the Release
NOTE: Confirm that all release-related PR's have been submitted before doing this step.
Do this in a direct clone of the upstream kubernetes/minikube repository (not your fork!):
```shell
hack/tag_release.sh <new version number>
sh hack/tag_release.sh 1.<minor>.<patch>
```
## Build the Release
@ -75,7 +71,7 @@ This step uses the git tag to publish new binaries to GCS and create a github re
## Check the release logs
Once the release completes, click "Console Output" to look or anything unusual. This is typically where you will see the brew automation fail, for instance.
After job completion, click "Console Output" to verify that the release completed without errors. This is typically where one will see brew automation fail, for instance.
## Check releases.json

4
docs/contributors/roadmap.md
View File

@ -8,7 +8,7 @@ Please send a PR to suggest any improvements to it.
- [ ] Creation of a user-centric minikube website for installation & documentation
- [ ] Localized output to 5+ written languages
- [ ] Make minikube usable in environments with challenging connectivity requirements
- [x] Make minikube usable in environments with challenging connectivity requirements
- [ ] Support lightweight deployment methods for environments where VM's are impractical
- [x] Add offline support
@ -44,7 +44,7 @@ Please send a PR to suggest any improvements to it.
## (#8) Stable and easy to debug
- [ ] Pre-flight error checks for common connectivity and configuration errors
- [x] Pre-flight error checks for common connectivity and configuration errors
- [ ] Improve the `minikube status` command so that it can diagnose common issues
- [ ] Mark all features not covered by continuous integration as `experimental`
- [x] Stabilize and improve profiles support (AKA multi-cluster)

101
docs/drivers.md
View File

@ -1,4 +1,4 @@
# Driver plugin installation
# VM Driver plugin installation
Minikube uses Docker Machine to manage the Kubernetes VM so it benefits from the
driver plugin architecture that Docker Machine uses to provide a consistent way to
@ -16,54 +16,25 @@ the host PATH:
## KVM2 driver
To install the KVM2 driver, first install and configure the prereqs:
To install the KVM2 driver, first install and configure the prerequisites, namely libvirt 1.3.1 or higher, and qemu-kvm:
* Debian or Ubuntu 18.x:
* Debian or Ubuntu 18.x: `sudo apt install libvirt-clients libvirt-daemon-system qemu-kvm`
* Ubuntu 16.x or older: `sudo apt install libvirt-bin libvirt-daemon-system qemu-kvm`
* Fedora/CentOS/RHEL: `sudo yum install libvirt-daemon-kvm qemu-kvm`
```shell
sudo apt install libvirt-clients libvirt-daemon-system qemu-kvm
```
Check your installed virsh version:
* Ubuntu 16.x or older:
`virsh --version`
```shell
sudo apt install libvirt-bin libvirt-daemon-system qemu-kvm
```
If your version of virsh is newer than 1.3.1 (January 2016), you may download our pre-built driver:
* Fedora/CentOS/RHEL:
```shell
sudo yum install libvirt-daemon-kvm qemu-kvm
```
Enable,start, and verify the `libvirtd` service has started.
```shell
sudo systemctl enable libvirtd.service
sudo systemctl start libvirtd.service
sudo systemctl status libvirtd.service
```
Then you will need to add yourself to `libvirt` group (older distributions may use `libvirtd` instead)
```shell
sudo usermod -a -G libvirt $(whoami)
```
Then to join the group with your current user session:
```shell
newgrp libvirt
```
Now install the driver:
```shell
curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-kvm2 \
&& sudo install docker-machine-driver-kvm2 /usr/local/bin/
```
NOTE: Ubuntu users on a release older than 18.04, or anyone experiencing [#3206: Error creating new host: dial tcp: missing address.](https://github.com/kubernetes/minikube/issues/3206) you will need to build your own driver until [#3689](https://github.com/kubernetes/minikube/issues/3689) is resolved. Building this binary will require [Go v1.12](https://golang.org/dl/) or newer to be installed.
If your version of virsh is older than 1.3.1 (Januarry 2016), you may build your own driver binary if you have go 1.12+ installed.
```shell
$ sudo apt install libvirt-dev
@ -73,22 +44,54 @@ $ make out/docker-machine-driver-kvm2
$ sudo install out/docker-machine-driver-kvm2 /usr/local/bin
```
To finish the kvm installation, start and verify the `libvirtd` service
```shell
sudo systemctl enable libvirtd.service
sudo systemctl start libvirtd.service
sudo systemctl status libvirtd.service
```
Add your user to `libvirt` group (older distributions may use `libvirtd` instead)
```shell
sudo usermod -a -G libvirt $(whoami)
```
Join the `libvirt` group with your current shell session:
```shell
newgrp libvirt
```
To use the kvm2 driver:
```shell
minikube start --vm-driver kvm2
```
or, to use kvm2 as a default driver:
or, to use kvm2 as a default driver for `minikube start`:
```shell
minikube config set vm-driver kvm2
```
and run minikube as usual:
### Troubleshoot
If minikube can't start, check if the kvm default network exists.
```shell
minikube start
virsh net-list
Name State Autostart Persistent
----------------------------------------------------------
default active yes yes
```
In case the default network doesn't exist you can define it.
```shell
curl https://raw.githubusercontent.com/libvirt/libvirt/master/src/network/default.xml > kvm-default.xml
virsh net-define kvm-default.xml
```
## Hyperkit driver
@ -185,3 +188,19 @@ and run minikube as usual:
```shell
minikube start
```
# Troubleshooting
minikube is currently unable to display the error message received back from the VM driver. Users can however reveal the error by passing `--alsologtostderr -v=8` to `minikube start`. For instance:
```shell
minikube start --vm-driver=kvm2 --alsologtostderr -v=8
```
Output:
```
Found binary path at /usr/local/bin/docker-machine-driver-kvm2
Launching plugin server for driver kvm2
Error starting plugin binary: fork/exec /usr/local/bin/docker-machine-driver-kvm2: exec format error
```

3
docs/env_vars.md
View File

@ -19,8 +19,7 @@ Some features can only be accessed by environment variables, here is a list of t
* **MINIKUBE_REMINDERWAITPERIODINHOURS** - (int) sets the number of hours to check for an update notification
* **MINIKUBE_WANTKUBECTLDOWNLOADMSG** - (bool) sets whether minikube should tell a user that `kubectl` cannot be found on there path
* **MINIKUBE_WANTNONEDRIVERWARNING** - (bool) sets whether minikube should warn a user about running the 'none' driver
* **CHANGE_MINIKUBE_NONE_USER** - (bool) automatically change ownership of ~/.minikube to the value of $SUDO_USER
* **MINIKUBE_ENABLE_PROFILING** - (int, `1` enables it) enables trace profiling to be generated for minikube

37
docs/networking.md
View File

@ -1,7 +1,18 @@
# Networking
The minikube VM is exposed to the host system via a host-only IP address, that can be obtained with the `minikube ip` command.
Any services of type `NodePort` can be accessed over that IP address, on the NodePort.
## Firewalls, VPN's, and proxies
minikube may require access from the host to the following IP ranges: 192.168.99.0/24, 192.168.39.0/24, and 10.96.0.0/12. These networks can be changed in minikube using `--host-only-cidr` and `--service-cluster-ip-range`.
* To use minikube with a proxy, see [Using HTTP/HTTPS proxies](http_proxy.md).
* If you are using minikube with a VPN, you may need to configure the VPN to allow local routing for traffic to the afforementioned IP ranges.
* If you are using minikube with a local firewall, you will need to allow access from the host to the afforementioned IP ranges on TCP ports 22 and 8443. You will also need to add access from these IP's to TCP ports 443 and 53 externally to pull images.
## Access to NodePort services
The minikube VM is exposed to the host system via a host-only IP address, that can be obtained with the `minikube ip` command. Any services of type `NodePort` can be accessed over that IP address, on the NodePort.
To determine the NodePort for your service, you can use a `kubectl` command like this (note that `nodePort` begins with lowercase `n` in JSON output):
@ -11,7 +22,17 @@ We also have a shortcut for fetching the minikube IP and a service's `NodePort`:
`minikube service --url $SERVICE`
## LoadBalancer emulation (`minikube tunnel`)
### Increasing the NodePort range
By default, minikube only exposes ports 30000-32767. If this is not enough, you can configure the apiserver to allow all ports using:
`minikube start --extra-config=apiserver.service-node-port-range=1-65535`
This flag also accepts a comma separated list of ports and port ranges.
## Access to LoadBalancer services using `minikube tunnel`
Services of type `LoadBalancer` can be exposed via the `minikube tunnel` command.
@ -50,13 +71,11 @@ To cleanup orphaned routes, run:
minikube tunnel --cleanup
````
## (Advanced) Running tunnel as root to avoid entering password multiple times
## Tunnel: Avoid entering password multiple times
`minikube tunnel` runs as a separate daemon, creates a network route on the host to the service CIDR of the cluster using the cluster's IP address as a gateway.
Adding a route requires root privileges for the user, and thus there are differences in how to run `minikube tunnel` depending on the OS.
`minikube tunnel` runs as a separate daemon, creates a network route on the host to the service CIDR of the cluster using the cluster's IP address as a gateway. Adding a route requires root privileges for the user, and thus there are differences in how to run `minikube tunnel` depending on the OS.
Recommended way to use on Linux with KVM2 driver and MacOSX with Hyperkit driver:
If you want to avoid entering the root password, consider setting NOPASSWD for "ip" and "route" commands:
`sudo -E minikube tunnel`
https://superuser.com/questions/1328452/sudoers-nopasswd-for-single-executable-but-allowing-others
Using VirtualBox on Windows, Mac and Linux _both_ `minikube start` and `minikube tunnel` needs to be started from the same Administrator user session otherwise [VBoxManage can't recognize the created VM](https://forums.virtualbox.org/viewtopic.php?f=6&t=81551).

16
docs/syncing-files.md Normal file
View File

@ -0,0 +1,16 @@
# minikube: Syncing files into the VM
## Syncing files during start up
As soon as a VM is created, minikube will populate the root filesystem with any files stored in $MINIKUBE_HOME (~/.minikube/files).
For example, running the following commands will result in `/etc/OMG` being added with the contents of `hello` into the minikube VM:
```
mkdir -p ~/.minikube/files/etc
echo hello > ~/.minikube/files/etc/OMG
minikube start
```
This method of file synchronization can be useful for adding configuration files for apiserver, or adding HTTPS certificates.

2
docs/vmdriver-none.md
View File

@ -12,7 +12,7 @@ The `none` driver supports releases of Debian, Ubuntu, and Fedora that are less
## Example: basic usage
`sudo minikube start`
`sudo minikube start --vm-driver=none`
NOTE: The none driver requires minikube to be run as root, until [#3760](https://github.com/kubernetes/minikube/issues/3760) can be addressed.

3
go.mod
View File

@ -14,6 +14,8 @@ require (
github.com/docker/docker v1.13.1 // indirect
github.com/docker/go-units v0.0.0-20170127094116-9e638d38cf69
github.com/docker/machine v0.16.1
github.com/elazarl/goproxy v0.0.0-20190421051319-9d40249d3c2f
github.com/elazarl/goproxy/ext v0.0.0-20190421051319-9d40249d3c2f // indirect
github.com/fatih/color v1.7.0 // indirect
github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680 // indirect
github.com/gogo/protobuf v0.0.0-20170330071051-c0656edd0d9e // indirect
@ -62,6 +64,7 @@ require (
github.com/pelletier/go-buffruneio v0.1.0 // indirect
github.com/pelletier/go-toml v0.0.0-20160822122712-0049ab3dc4c4 // indirect
github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2
github.com/pkg/browser v0.0.0-20160118053552-9302be274faa
github.com/pkg/errors v0.8.0
github.com/pkg/profile v0.0.0-20161223203901-3a8809bd8a80

12
go.sum
View File

@ -17,6 +17,8 @@ github.com/cloudfoundry/jibber_jabber v0.0.0-20151120183258-bcc4c8345a21 h1:tuij
github.com/cloudfoundry/jibber_jabber v0.0.0-20151120183258-bcc4c8345a21/go.mod h1:po7NpZ/QiTKzBKyrsEAxwnTamCoh8uDk/egRpQ7siIc=
github.com/cpuguy83/go-md2man v1.0.4 h1:OwjhDpK9YGCcI5CDf8HcdfsXqr6znFyAJfuZ27ixJsc=
github.com/cpuguy83/go-md2man v1.0.4/go.mod h1:N6JayAiVKtlHSnuTCeuLSQVs75hb8q+dYQLjr7cDsKY=
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@ -26,6 +28,10 @@ github.com/docker/go-units v0.0.0-20170127094116-9e638d38cf69 h1:N4WAsrRIb+4U1yI
github.com/docker/go-units v0.0.0-20170127094116-9e638d38cf69/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/machine v0.16.1 h1:zrgroZounGVkxLmBqMyc1uT2GgapXVjIWHCfBf0udrA=
github.com/docker/machine v0.16.1/go.mod h1:I8mPNDeK1uH+JTcUU7X0ZW8KiYz0jyAgNaeSJ1rCfDI=
github.com/elazarl/goproxy v0.0.0-20190421051319-9d40249d3c2f h1:8GDPb0tCY8LQ+OJ3dbHb5sA6YZWXFORQYZx5sdsTlMs=
github.com/elazarl/goproxy v0.0.0-20190421051319-9d40249d3c2f/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc=
github.com/elazarl/goproxy/ext v0.0.0-20190421051319-9d40249d3c2f h1:AUj1VoZUfhPhOPHULCQQDnGhRelpFWHMLhQVWDsS0v4=
github.com/elazarl/goproxy/ext v0.0.0-20190421051319-9d40249d3c2f/go.mod h1:gNh8nYJoAm43RfaxurUnxr+N1PwuFV3ZMl/efxlIlY8=
github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
@ -136,6 +142,8 @@ github.com/pelletier/go-toml v0.0.0-20160822122712-0049ab3dc4c4 h1:tMVXZ04h5CqgT
github.com/pelletier/go-toml v0.0.0-20160822122712-0049ab3dc4c4/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/peterbourgon/diskv v2.0.1+incompatible h1:UBdAOUP5p4RWqPBg048CAvpKN+vxiaj6gdUUzhl4XmI=
github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=
github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2 h1:JhzVVoYvbOACxoUmOs6V/G4D5nPVUW73rKvXxP4XUJc=
github.com/phayes/freeport v0.0.0-20180830031419-95f893ade6f2/go.mod h1:iIss55rKnNBTvrwdmkUpLnDpZoAHvWaiq5+iMmen4AE=
github.com/pkg/browser v0.0.0-20160118053552-9302be274faa h1:od00Tr1U7+cLVtc+RNFmR53spHUF98Ziu33S8UIQnt0=
github.com/pkg/browser v0.0.0-20160118053552-9302be274faa/go.mod h1:4OwLy04Bl9Ef3GJJCoec+30X3LQs/0/m4HFRt/2LUSA=
github.com/pkg/errors v0.8.0 h1:WdK/asTD0HN+q6hsWO3/vpuAkAr+tw6aNJNDFFf0+qw=
@ -148,6 +156,7 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/r2d4/external-storage v0.0.0-20171222174501-8c0e8605dc7b h1:+wokSDzl6kjfWhVQsBhFOC2t4TYfdLfRXfWorEg3KUE=
github.com/r2d4/external-storage v0.0.0-20171222174501-8c0e8605dc7b/go.mod h1:/UlUhYuWiiitqIPbAxyU96i/wDlBS8sRHX2lRN+ffgs=
github.com/rogpeppe/go-charset v0.0.0-20180617210344-2471d30d28b4/go.mod h1:qgYeAmZ5ZIpBWTGllZSQnw97Dj+woV0toclVaRGI8pc=
github.com/russross/blackfriday v0.0.0-20151117072312-300106c228d5 h1:+6eORf9Bt4C3Wjt91epyu6wvLW+P6+AEODb6uKgO+4g=
github.com/russross/blackfriday v0.0.0-20151117072312-300106c228d5/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
github.com/samalba/dockerclient v0.0.0-20160414174713-91d7393ff859 h1:XRl74t6xHKI5EVIjDI5nPlHRq0bHED9/TjQuD8/UMkE=
@ -169,10 +178,13 @@ github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnIn
github.com/spf13/viper v1.0.0 h1:RUA/ghS2i64rlnn4ydTfblY8Og8QzcPtCcHvgMn+w/I=
github.com/spf13/viper v1.0.0/go.mod h1:A8kyI5cUJhb8N+3pkfONlcEcZbueH6nhAm0Fq7SrnBM=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/xeipuuv/gojsonpointer v0.0.0-20151027082146-e0fe6f683076 h1:KM4T3G70MiR+JtqplcYkNVoNz7pDwYaBxWBXQK804So=
github.com/xeipuuv/gojsonpointer v0.0.0-20151027082146-e0fe6f683076/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=

23
installers/windows/minikube.nsi
View File

@ -43,6 +43,8 @@ OutFile "minikube-installer.exe"
!define MUI_WELCOMEFINISHPAGE_BITMAP "logo.bmp"
!define MUI_UNWELCOMEFINISHPAGE_BITMAP "logo.bmp"
!define MUI_HEADERIMAGE_BITMAP "logo.bmp"
!define MUI_FINISHPAGE_NOAUTOCLOSE
!define MUI_UNFINISHPAGE_NOAUTOCLOSE
!insertmacro MUI_PAGE_WELCOME
!insertmacro MUI_PAGE_LICENSE "LICENSE.txt"
@ -124,7 +126,7 @@ Section "Install"
# Files added here should be removed by the uninstaller (see section "uninstall")
File "minikube.exe"
File "logo.ico"
File "update_path.bat"
File "update_path.ps1"
# Add any other files for the install directory (license files, app data, etc) here
# Uninstaller - See function un.onInit and section "uninstall" for configuration
@ -147,17 +149,20 @@ Section "Install"
WriteRegStr HKLM "${UNINSTALLDIR}" "DisplayVersion" "$\"${VERSIONMAJOR}.${VERSIONMINOR}.${VERSIONBUILD}$\""
WriteRegDWORD HKLM "${UNINSTALLDIR}" "VersionMajor" ${VERSIONMAJOR}
WriteRegDWORD HKLM "${UNINSTALLDIR}" "VersionMinor" ${VERSIONMINOR}
# There is no option for modifying or repairing the install
# There is no option for modifying or repairing the install
WriteRegDWORD HKLM "${UNINSTALLDIR}" "NoModify" 1
WriteRegDWORD HKLM "${UNINSTALLDIR}" "NoRepair" 1
# Set the INSTALLSIZE constant (!defined at the top of this script) so Add/Remove Programs can accurately report the size
# Set the INSTALLSIZE constant (!defined at the top of this script) so Add/Remove Programs can accurately report the size
WriteRegDWORD HKLM "${UNINSTALLDIR}" "EstimatedSize" ${INSTALLSIZE}
# Add installed executable to PATH
# Cannot uset EnvVarUpdate since the path can be too long
# this is explicitly warned in the documentation page
# http://nsis.sourceforge.net/Environmental_Variables:_append,_prepend,_and_remove_entries
nsExec::Exec '"$INSTDIR\update_path.bat" add $INSTDIR'
# Cannot uset EnvVarUpdate since the path can be too long
# this is explicitly warned in the documentation page
# http://nsis.sourceforge.net/Environmental_Variables:_append,_prepend,_and_remove_entries
nsExec::ExecToLog 'powershell -ExecutionPolicy Bypass -WindowStyle Hidden -File "$INSTDIR\update_path.ps1" -Add -Path "$INSTDIR"'
SectionEnd
Section "Uninstall"
@ -168,12 +173,12 @@ Section "Uninstall"
RmDir /REBOOTOK "$SMPROGRAMS\${COMPANYNAME}"
# Remove uninstalled executable from PATH
nsExec::Exec '"$INSTDIR\update_path.bat" remove $INSTDIR' ; appends to the system path
nsExec::ExecToLog 'powershell -ExecutionPolicy Bypass -WindowStyle Hidden -File "$INSTDIR\update_path.ps1" -Remove -Path "$INSTDIR"' ; appends to the system path
# Remove files
Delete /REBOOTOK $INSTDIR\minikube.exe
Delete /REBOOTOK $INSTDIR\logo.ico
Delete /REBOOTOK $INSTDIR\update_path.bat
Delete /REBOOTOK $INSTDIR\update_path.ps1
# Always delete uninstaller as the last action
Delete /REBOOTOK $INSTDIR\uninstall.exe

42
installers/windows/update_path.bat
View File

@ -1,42 +0,0 @@
@echo off
setlocal EnableExtensions EnableDelayedExpansion
if [%1] == [] (
goto :usage
)
if [%2] == [] (
goto :usage
)
set TARGET_PATH=%2 %3 %4 %5 %6 %7 %8 %9
:: Remove trailing spaces
for /f "tokens=* delims= " %%a in ("!TARGET_PATH!") do set "TARGET_PATH=%%a"
for /l %%a in (1,1,100) do if "!TARGET_PATH:~-1!"==" " set "TARGET_PATH=!TARGET_PATH:~0,-1!"
:: Remove trailing ; if any
if "%PATH:~-1%"==";" (
set PATH=!PATH:~0,-1!
)
if "%1" == "add" (
set "PATH=!PATH!;%TARGET_PATH%"
goto :update
)
if "%1" == "remove" (
set "PATH=!PATH:;%TARGET_PATH%=!"
goto :update
)
:usage
echo Script to add or remove to path environment variable
echo Usage:
echo %0 [add^|remove] path
exit /b 1
:update
call Setx PATH "!PATH!" /m

70
installers/windows/update_path.ps1 Normal file
View File

@ -0,0 +1,70 @@
<#
.DESCRIPTION
This script is used to add/remove the installation path of Minikube in the PATH Environment variable as part of installation/uninstallation of Minikube.
The script assumes that the PATH exists before running.
.PARAMETER Add
This is a Switch parameter which tells the script to ADD the path supplied to the System's PATH Environment variable.
.PARAMETER Remove
This is a Switch parameter which tells the script to REMOVE the path supplied from the System's PATH Environment variable.
.PARAMETER Path
This parameter accepts a string which needs to be added/removed from the System's PATH Environment Variable.
#>
param(
[cmdletbinding()]
# This parameter dictates if the path needs to be added
[Parameter(Mandatory=$false,ParameterSetName="EnvironmentVariableAddOperation")]
[switch]
$Add,
# This parameter dictates if the path needs to be removed
[Parameter(Mandatory=$false,ParameterSetName="EnvironmentVariableRemoveOperation")]
[switch]
$Remove,
# This parameter tells us the path inside the $PATH Environment Variable for which the operation needs to be performed
[Parameter(Mandatory=$true)]
[string]
$Path
)
$currentSystemPath = [Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine)
try {
if ($Add) {
Write-Output "Path needs to be added."
Write-Output "Checking if the given path already exists or not"
if ($currentSystemPath -match [Regex]::Escape($Path)) {
Write-Output "The provided path already exists in the system. Exiting now."
} else {
Write-Output "The given path was not found. Adding it now."
if ($currentSystemPath.EndsWith(";")) {
$newSystemPath = $currentSystemPath + $Path.Trim() + ";"
} else {
$newSystemPath = $currentSystemPath + ";" + $Path.Trim() + ";"
}
[Environment]::SetEnvironmentVariable("Path", $newSystemPath, [EnvironmentVariableTarget]::Machine)
Write-Output "Path has been added successfully."
}
} else {
Write-Output "Path needs to be added."
Write-Output "Checking if the given path already exists or not"
if ($currentSystemPath -match [Regex]::Escape($Path)) {
Write-Output "The provided path exists in the system. Removing now."
$newSystemPath = $currentSystemPath.Replace(($Path.Trim() + ";"), "")
[Environment]::SetEnvironmentVariable("Path", $newSystemPath, [EnvironmentVariableTarget]::Machine)
} else {
Write-Output "The given path was not found. Exiting now."
}
}
}
catch {
Write-Output "[Error]:: There was an error while execution. Please see the details below. Ensure that the script is running with administrator privileges."
Write-Output $_
}

2
pkg/drivers/kvm/gpu.go
View File

@ -174,7 +174,7 @@ func isIsolated(device string) bool {
iommuGroupPath := filepath.Join(sysFsPCIDevicesPath, device, "iommu_group", "devices")
otherDevices, err := ioutil.ReadDir(iommuGroupPath)
if err != nil {
log.Infof("Error reading %q: %v", iommuGroupPath)
log.Infof("Error reading %q: %v", iommuGroupPath, err)
return false
}

55
pkg/drivers/kvm/kvm.go
View File

@ -442,17 +442,54 @@ func (d *Driver) Remove() error {
log.Debug("Checking if the domain needs to be deleted")
dom, err := conn.LookupDomainByName(d.MachineName)
if err != nil {
log.Warn("Domain %s does not exist, nothing to clean up...", d.MachineName)
log.Warnf("Domain %s does not exist, nothing to clean up...", d.MachineName)
return nil
}
if dom != nil {
log.Infof("Domain %s exists, removing...", d.MachineName)
if err := dom.Destroy(); err != nil {
return err
}
if err := dom.Undefine(); err != nil {
return err
}
log.Infof("Domain %s exists, removing...", d.MachineName)
if err := d.destroyRunningDomain(dom); err != nil {
return errors.Wrap(err, "destroying running domain")
}
if err := d.undefineDomain(conn, dom); err != nil {
return errors.Wrap(err, "undefine domain")
}
return nil
}
func (d *Driver) destroyRunningDomain(dom *libvirt.Domain) error {
state, reason, err := dom.GetState()
if err != nil {
return errors.Wrap(err, "getting domain state")
}
if state == libvirt.DOMAIN_SHUTOFF && reason == int(libvirt.DOMAIN_SHUTOFF_DESTROYED) {
log.Warnf("Domain %s already destroyed, skipping...", d.MachineName)
return nil
}
return dom.Destroy()
}
func (d *Driver) undefineDomain(conn *libvirt.Connect, dom *libvirt.Domain) error {
definedDomains, err := conn.ListDefinedDomains()
if err != nil {
return errors.Wrap(err, "list defined domains")
}
var found bool
for _, domain := range definedDomains {
if domain == d.MachineName {
found = true
break
}
}
if !found {
log.Warnf("Domain %s not defined, skipping undefine...", d.MachineName)
return nil
}
return dom.Undefine()
}

41
pkg/drivers/kvm/network.go
View File

@ -89,10 +89,9 @@ func (d *Driver) ensureNetwork() error {
// network: default
// Start the default network
// It is assumed that the libvirt/kvm installation has already created this network
log.Infof("Ensuring network %s is active", defaultNetworkName)
if err := setupNetwork(conn, defaultNetworkName); err != nil {
log.Infof("Ensuring network %s is active", d.Network)
if err := setupNetwork(conn, d.Network); err != nil {
return err
}
@ -117,9 +116,11 @@ func (d *Driver) createNetwork() error {
// network: default
// It is assumed that the libvirt/kvm installation has already created this network
if _, err := conn.LookupNetworkByName(d.Network); err != nil {
return errors.Wrapf(err, "network %s doesn't exist", d.Network)
}
// network: private
// Only create the private network if it does not already exist
if _, err := conn.LookupNetworkByName(d.PrivateNetwork); err != nil {
// create the XML for the private network from our networkTmpl
@ -172,8 +173,12 @@ func (d *Driver) deleteNetwork() error {
log.Debugf("Checking if network %s exists...", d.PrivateNetwork)
network, err := conn.LookupNetworkByName(d.PrivateNetwork)
if err != nil {
// TODO: decide if we really wanna throw an error?
return errors.Wrap(err, "network %s does not exist")
if libvirtErr, ok := err.(libvirt.Error); ok && libvirtErr.Code == libvirt.ERR_NO_NETWORK {
log.Warnf("Network %s does not exist. Skipping deletion", d.PrivateNetwork)
return nil
}
return errors.Wrapf(err, "failed looking for network %s", d.PrivateNetwork)
}
log.Debugf("Network %s exists", d.PrivateNetwork)
@ -287,24 +292,34 @@ func (d *Driver) lookupIPFromStatusFile(conn *libvirt.Connect) (string, error) {
if err != nil {
return "", errors.Wrap(err, "reading status file")
}
return parseStatusAndReturnIP(d.PrivateMAC, statuses)
}
func parseStatusAndReturnIP(privateMAC string, statuses []byte) (string, error) {
type StatusEntry struct {
IPAddress string `json:"ip-address"`
MacAddress string `json:"mac-address"`
}
var statusEntries []StatusEntry
// If the status file is empty, parsing will fail, ignore this error.
_ = json.Unmarshal(statuses, &statusEntries)
// empty file return blank
if len(statuses) == 0 {
return "", nil
}
err := json.Unmarshal(statuses, &statusEntries)
if err != nil {
return "", errors.Wrap(err, "reading status file")
}
ipAddress := ""
for _, status := range statusEntries {
if status.MacAddress == d.PrivateMAC {
ipAddress = status.IPAddress
if status.MacAddress == privateMAC {
return status.IPAddress, nil
}
}
return ipAddress, nil
return "", nil
}
func (d *Driver) lookupIPFromLeasesFile() (string, error) {

97
pkg/drivers/kvm/network_test.go Normal file
View File

@ -0,0 +1,97 @@
// +build linux
/*
Copyright 2016 The Kubernetes Authors All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package kvm
import (
"testing"
)
var (
emptyFile = []byte(``)
fileWithInvalidJSON = []byte(`{`)
fileWithNoStatus = []byte(`[
]`)
fileWithStatus = []byte(`[
{
"ip-address": "1.2.3.5",
"mac-address": "a4:b5:c6:d7:e8:f9",
"hostname": "host2",
"client-id": "01:44:59:e7:fd:f4:d6",
"expiry-time": 1558638717
},
{
"ip-address": "1.2.3.4",
"mac-address": "a1:b2:c3:d4:e5:f6",
"hostname": "host1",
"client-id": "01:ec:97:de:a2:86:81",
"expiry-time": 1558639092
}
]`)
)
func Test_parseStatusAndReturnIp(t *testing.T) {
type args struct {
mac string
statuses []byte
}
tests := []struct {
name string
args args
want string
wantErr bool
}{
{
"emptyFile",
args{"a1:b2:c3:d4:e5:f6", emptyFile},
"",
false,
},
{
"fileWithStatus",
args{"a1:b2:c3:d4:e5:f6", fileWithStatus},
"1.2.3.4",
false,
},
{
"fileWithNoStatus",
args{"a4:b5:c6:d7:e8:f9", fileWithNoStatus},
"",
false,
},
{
"fileWithInvalidJSON",
args{"a4:b5:c6:d7:e8:f9", fileWithInvalidJSON},
"",
true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := parseStatusAndReturnIP(tt.args.mac, tt.args.statuses)
if (err != nil) != tt.wantErr {
t.Errorf("parseStatusAndReturnIP() error = %v, wantErr %v", err, tt.wantErr)
return
}
if got != tt.want {
t.Errorf("parseStatusAndReturnIP() = %v, want %v", got, tt.want)
}
})
}
}

1
pkg/minikube/bootstrapper/bootstrapper.go
View File

@ -39,6 +39,7 @@ type Bootstrapper interface {
UpdateCluster(config.KubernetesConfig) error
RestartCluster(config.KubernetesConfig) error
DeleteCluster(config.KubernetesConfig) error
WaitCluster(config.KubernetesConfig) error
// LogCommands returns a map of log type to a command which will display that log.
LogCommands(LogOptions) map[string]string
SetupCerts(cfg config.KubernetesConfig) error

227
pkg/minikube/bootstrapper/kubeadm/kubeadm.go
View File

@ -33,6 +33,7 @@ import (
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/util/wait"
"k8s.io/minikube/pkg/minikube/assets"
"k8s.io/minikube/pkg/minikube/bootstrapper"
"k8s.io/minikube/pkg/minikube/config"
@ -43,22 +44,33 @@ import (
"k8s.io/minikube/pkg/util"
)
// SkipPreflights are preflight checks we always skip.
var SkipPreflights = []string{
// We use --ignore-preflight-errors=DirAvailable since we have our own custom addons
// that we also stick in /etc/kubernetes/manifests
"DirAvailable--etc-kubernetes-manifests",
"DirAvailable--data-minikube",
"Port-10250",
"FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml",
"FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml",
"FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml",
"FileAvailable--etc-kubernetes-manifests-etcd.yaml",
// So that "none" driver users don't have to reconfigure their machine
"Swap",
// We use --ignore-preflight-errors=CRI since /var/run/dockershim.sock is not present.
// (because we start kubelet with an invalid config)
"CRI",
// enum to differentiate kubeadm command line parameters from kubeadm config file parameters (see the
// KubeadmExtraArgsWhitelist variable below for more info)
const (
KubeadmCmdParam = iota
KubeadmConfigParam = iota
)
// KubeadmExtraArgsWhitelist is a whitelist of supported kubeadm params that can be supplied to kubeadm through
// minikube's ExtraArgs parameter. The list is split into two parts - params that can be supplied as flags on the
// command line and params that have to be inserted into the kubeadm config file. This is because of a kubeadm
// constraint which allows only certain params to be provided from the command line when the --config parameter
// is specified
var KubeadmExtraArgsWhitelist = map[int][]string{
KubeadmCmdParam: {
"ignore-preflight-errors",
"dry-run",
"kubeconfig",
"kubeconfig-dir",
"node-name",
"cri-socket",
"experimental-upload-certs",
"certificate-key",
"rootfs",
},
KubeadmConfigParam: {
"pod-network-cidr",
},
}
type pod struct {
@ -70,7 +82,6 @@ type pod struct {
// PodsByLayer are queries we run when health checking, sorted roughly by dependency layer
var PodsByLayer = []pod{
{"apiserver", "component", "kube-apiserver"},
{"proxy", "k8s-app", "kube-proxy"},
{"etcd", "component", "etcd"},
{"scheduler", "component", "kube-scheduler"},
@ -123,6 +134,7 @@ func (k *Bootstrapper) GetAPIServerStatus(ip net.IP, apiserverPort int) (string,
url := fmt.Sprintf("https://%s:%d/healthz", ip, apiserverPort)
// To avoid: x509: certificate signed by unknown authority
tr := &http.Transport{
Proxy: nil, // To avoid connectiv issue if http(s)_proxy is set.
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: tr}
@ -163,6 +175,21 @@ func (k *Bootstrapper) LogCommands(o bootstrapper.LogOptions) map[string]string
}
}
// createFlagsFromExtraArgs converts kubeadm extra args into flags to be supplied from the commad linne
func createFlagsFromExtraArgs(extraOptions util.ExtraOptionSlice) string {
kubeadmExtraOpts := extraOptions.AsMap().Get(Kubeadm)
// kubeadm allows only a small set of parameters to be supplied from the command line when the --config param
// is specified, here we remove those that are not allowed
for opt := range kubeadmExtraOpts {
if !util.ContainsString(KubeadmExtraArgsWhitelist[KubeadmCmdParam], opt) {
// kubeadmExtraOpts is a copy so safe to delete
delete(kubeadmExtraOpts, opt)
}
}
return convertToFlags(kubeadmExtraOpts)
}
// StartCluster starts the cluster
func (k *Bootstrapper) StartCluster(k8s config.KubernetesConfig) error {
version, err := ParseKubernetesVersion(k8s.KubernetesVersion)
@ -170,40 +197,36 @@ func (k *Bootstrapper) StartCluster(k8s config.KubernetesConfig) error {
return errors.Wrap(err, "parsing kubernetes version")
}
extraOpts, err := ExtraConfigForComponent(Kubeadm, k8s.ExtraOptions, version)
if err != nil {
return errors.Wrap(err, "generating extra configuration for kubelet")
}
extraFlags := convertToFlags(extraOpts)
extraFlags := createFlagsFromExtraArgs(k8s.ExtraOptions)
r, err := cruntime.New(cruntime.Config{Type: k8s.ContainerRuntime})
if err != nil {
return err
}
b := bytes.Buffer{}
preflights := SkipPreflights
preflights = append(preflights, SkipAdditionalPreflights[r.Name()]...)
templateContext := struct {
KubeadmConfigFile string
SkipPreflightChecks bool
Preflights []string
ExtraOptions string
}{
KubeadmConfigFile: constants.KubeadmConfigFile,
SkipPreflightChecks: !VersionIsBetween(version,
semver.MustParse("1.9.0-alpha.0"),
semver.Version{}),
Preflights: preflights,
ExtraOptions: extraFlags,
ignore := []string{
"DirAvailable--etc-kubernetes-manifests", // Addons are stored in /etc/kubernetes/manifests
"DirAvailable--data-minikube",
"FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml",
"FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml",
"FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml",
"FileAvailable--etc-kubernetes-manifests-etcd.yaml",
"Port-10250", // For "none" users who already have a kubelet online
"Swap", // For "none" users who have swap configured
}
if err := kubeadmInitTemplate.Execute(&b, templateContext); err != nil {
return err
ignore = append(ignore, SkipAdditionalPreflights[r.Name()]...)
// Allow older kubeadm versions to function with newer Docker releases.
if version.LT(semver.MustParse("1.13.0")) {
glog.Infof("Older Kubernetes release detected (%s), disabling SystemVerification check.", version)
ignore = append(ignore, "SystemVerification")
}
out, err := k.c.CombinedOutput(b.String())
cmd := fmt.Sprintf("sudo /usr/bin/kubeadm init --config %s %s --ignore-preflight-errors=%s",
constants.KubeadmConfigFile, extraFlags, strings.Join(ignore, ","))
out, err := k.c.CombinedOutput(cmd)
if err != nil {
return errors.Wrapf(err, "kubeadm init: %s\n%s\n", b.String(), out)
return errors.Wrapf(err, "cmd failed: %s\n%s\n", cmd, out)
}
if version.LT(semver.MustParse("1.10.0-alpha.0")) {
@ -214,20 +237,35 @@ func (k *Bootstrapper) StartCluster(k8s config.KubernetesConfig) error {
}
}
if err := waitForPods(k8s, false); err != nil {
return errors.Wrap(err, "wait")
}
glog.Infof("Configuring cluster permissions ...")
if err := util.RetryAfter(100, elevateKubeSystemPrivileges, time.Millisecond*500); err != nil {
return errors.Wrap(err, "timed out waiting to elevate kube-system RBAC privileges")
}
// Make sure elevating privileges didn't screw anything up
if err := waitForPods(k8s, true); err != nil {
return errors.Wrap(err, "wait")
if err := k.adjustResourceLimits(); err != nil {
glog.Warningf("unable to adjust resource limits: %v", err)
}
return nil
}
// adjustResourceLimits makes fine adjustments to pod resources that aren't possible via kubeadm config.
func (k *Bootstrapper) adjustResourceLimits() error {
score, err := k.c.CombinedOutput("cat /proc/$(pgrep kube-apiserver)/oom_adj")
if err != nil {
return errors.Wrap(err, "oom_adj check")
}
glog.Infof("apiserver oom_adj: %s", score)
// oom_adj is already a negative number
if strings.HasPrefix(score, "-") {
return nil
}
glog.Infof("adjusting apiserver oom_adj to -10")
// Prevent the apiserver from OOM'ing before other pods, as it is our gateway into the cluster.
// It'd be preferable to do this via Kubernetes, but kubeadm doesn't have a way to set pod QoS.
if err := k.c.Run("echo -10 | sudo tee /proc/$(pgrep kube-apiserver)/oom_adj"); err != nil {
return errors.Wrap(err, "oom_adj adjust")
}
return nil
}
@ -260,37 +298,37 @@ func addAddons(files *[]assets.CopyableFile, data interface{}) error {
return nil
}
// waitForPods waits until the important Kubernetes pods are in running state
func waitForPods(k8s config.KubernetesConfig, quiet bool) error {
// WaitCluster blocks until Kubernetes appears to be healthy.
func (k *Bootstrapper) WaitCluster(k8s config.KubernetesConfig) error {
// Do not wait for "k8s-app" pods in the case of CNI, as they are managed
// by a CNI plugin which is usually started after minikube has been brought
// up. Otherwise, minikube won't start, as "k8s-app" pods are not ready.
componentsOnly := k8s.NetworkPlugin == "cni"
if !quiet {
console.OutStyle("waiting-pods", "Waiting for:")
}
console.OutStyle(console.WaitingPods, "Verifying:")
client, err := util.GetClient()
if err != nil {
return errors.Wrap(err, "k8s client")
}
// Wait until the apiserver can answer queries properly. We don't care if the apiserver
// pod shows up as registered, but need the webserver for all subsequent queries.
console.Out(" apiserver")
if err := k.waitForAPIServer(k8s); err != nil {
return errors.Wrap(err, "waiting for apiserver")
}
for _, p := range PodsByLayer {
if componentsOnly && p.key != "component" {
continue
}
if !quiet {
console.Out(" %s", p.name)
}
console.Out(" %s", p.name)
selector := labels.SelectorFromSet(labels.Set(map[string]string{p.key: p.value}))
if err := util.WaitForPodsWithLabelRunning(client, "kube-system", selector); err != nil {
return errors.Wrap(err, fmt.Sprintf("waiting for %s=%s", p.key, p.value))
}
}
if !quiet {
console.OutLn("")
}
console.OutLn("")
return nil
}
@ -308,11 +346,13 @@ func (k *Bootstrapper) RestartCluster(k8s config.KubernetesConfig) error {
controlPlane = "control-plane"
}
configPath := constants.KubeadmConfigFile
baseCmd := fmt.Sprintf("sudo kubeadm %s", phase)
cmds := []string{
fmt.Sprintf("sudo kubeadm %s phase certs all --config %s", phase, constants.KubeadmConfigFile),
fmt.Sprintf("sudo kubeadm %s phase kubeconfig all --config %s", phase, constants.KubeadmConfigFile),
fmt.Sprintf("sudo kubeadm %s phase %s all --config %s", phase, controlPlane, constants.KubeadmConfigFile),
fmt.Sprintf("sudo kubeadm %s phase etcd local --config %s", phase, constants.KubeadmConfigFile),
fmt.Sprintf("%s phase certs all --config %s", baseCmd, configPath),
fmt.Sprintf("%s phase kubeconfig all --config %s", baseCmd, configPath),
fmt.Sprintf("%s phase %s all --config %s", baseCmd, controlPlane, configPath),
fmt.Sprintf("%s phase etcd local --config %s", baseCmd, configPath),
}
// Run commands one at a time so that it is easier to root cause failures.
@ -322,23 +362,36 @@ func (k *Bootstrapper) RestartCluster(k8s config.KubernetesConfig) error {
}
}
if err := waitForPods(k8s, false); err != nil {
return errors.Wrap(err, "wait")
if err := k.waitForAPIServer(k8s); err != nil {
return errors.Wrap(err, "waiting for apiserver")
}
// restart the proxy and coredns
if err := k.c.Run(fmt.Sprintf("%s phase addon all --config %s", baseCmd, configPath)); err != nil {
return errors.Wrapf(err, "addon phase")
}
console.OutStyle("reconfiguring", "Updating kube-proxy configuration ...")
if err = util.RetryAfter(5, func() error { return updateKubeProxyConfigMap(k8s) }, 5*time.Second); err != nil {
return errors.Wrap(err, "restarting kube-proxy")
if err := k.adjustResourceLimits(); err != nil {
glog.Warningf("unable to adjust resource limits: %v", err)
}
// Make sure the kube-proxy restart didn't screw anything up.
if err := waitForPods(k8s, true); err != nil {
return errors.Wrap(err, "wait")
}
return nil
}
// waitForAPIServer waits for the apiserver to start up
func (k *Bootstrapper) waitForAPIServer(k8s config.KubernetesConfig) error {
glog.Infof("Waiting for apiserver ...")
return wait.PollImmediate(time.Millisecond*300, time.Minute*3, func() (bool, error) {
status, err := k.GetAPIServerStatus(net.ParseIP(k8s.NodeIP), k8s.NodePort)
glog.Infof("apiserver status: %s, err: %v", status, err)
if err != nil {
return false, err
}
if status != "Running" {
return false, nil
}
return true, nil
})
}
// DeleteCluster removes the components that were started earlier
func (k *Bootstrapper) DeleteCluster(k8s config.KubernetesConfig) error {
cmd := fmt.Sprintf("sudo kubeadm reset --force")
@ -474,6 +527,25 @@ sudo systemctl start kubelet
return nil
}
// createExtraComponentConfig generates a map of component to extra args for all of the components except kubeadm
func createExtraComponentConfig(extraOptions util.ExtraOptionSlice, version semver.Version, componentFeatureArgs string) ([]ComponentExtraArgs, error) {
extraArgsSlice, err := NewComponentExtraArgs(extraOptions, version, componentFeatureArgs)
if err != nil {
return nil, err
}
// kubeadm extra args should not be included in the kubeadm config in the extra args section (instead, they must
// be inserted explicitly in the appropriate places or supplied from the command line); here we remove all of the
// kubeadm extra args from the slice
for i, extraArgs := range extraArgsSlice {
if extraArgs.Component == Kubeadm {
extraArgsSlice = append(extraArgsSlice[:i], extraArgsSlice[i+1:]...)
break
}
}
return extraArgsSlice, nil
}
func generateConfig(k8s config.KubernetesConfig, r cruntime.Manager) (string, error) {
version, err := ParseKubernetesVersion(k8s.KubernetesVersion)
if err != nil {
@ -486,8 +558,7 @@ func generateConfig(k8s config.KubernetesConfig, r cruntime.Manager) (string, er
return "", errors.Wrap(err, "parses feature gate config for kubeadm and component")
}
// generates a map of component to extra args for apiserver, controller-manager, and scheduler
extraComponentConfig, err := NewComponentExtraArgs(k8s.ExtraOptions, version, componentFeatureArgs)
extraComponentConfig, err := createExtraComponentConfig(k8s.ExtraOptions, version, componentFeatureArgs)
if err != nil {
return "", errors.Wrap(err, "generating extra component config for kubeadm")
}
@ -501,6 +572,7 @@ func generateConfig(k8s config.KubernetesConfig, r cruntime.Manager) (string, er
opts := struct {
CertDir string
ServiceCIDR string
PodSubnet string
AdvertiseAddress string
APIServerPort int
KubernetesVersion string
@ -514,6 +586,7 @@ func generateConfig(k8s config.KubernetesConfig, r cruntime.Manager) (string, er
}{
CertDir: util.DefaultCertPath,
ServiceCIDR: util.DefaultServiceCIDR,
PodSubnet: k8s.ExtraOptions.Get("pod-network-cidr", Kubeadm),
AdvertiseAddress: k8s.NodeIP,
APIServerPort: nodePort,
KubernetesVersion: k8s.KubernetesVersion,

19
pkg/minikube/bootstrapper/kubeadm/kubeadm_test.go
View File

@ -153,6 +153,24 @@ func TestGenerateConfig(t *testing.T) {
Key: "scheduler-name",
Value: "mini-scheduler",
},
util.ExtraOption{
Component: Kubeadm,
Key: "ignore-preflight-errors",
Value: "true",
},
util.ExtraOption{
Component: Kubeadm,
Key: "dry-run",
Value: "true",
},
}
extraOptsPodCidr := util.ExtraOptionSlice{
util.ExtraOption{
Component: Kubeadm,
Key: "pod-network-cidr",
Value: "192.168.32.0/20",
},
}
// Test version policy: Last 4 major releases (slightly looser than our general policy)
@ -177,6 +195,7 @@ func TestGenerateConfig(t *testing.T) {
{"crio-options-gates", "crio", false, config.KubernetesConfig{ExtraOptions: extraOpts, FeatureGates: "a=b"}},
{"unknown-component", "docker", true, config.KubernetesConfig{ExtraOptions: util.ExtraOptionSlice{util.ExtraOption{Component: "not-a-real-component", Key: "killswitch", Value: "true"}}}},
{"containerd-api-port", "containerd", false, config.KubernetesConfig{NodePort: 12345}},
{"containerd-pod-network-cidr", "containerd", false, config.KubernetesConfig{ExtraOptions: extraOptsPodCidr}},
{"image-repository", "docker", false, config.KubernetesConfig{ImageRepository: "test/repo"}},
}
for vname, version := range versions {

6
pkg/minikube/bootstrapper/kubeadm/templates.go
View File

@ -85,7 +85,7 @@ etcd:
kubernetesVersion: {{.KubernetesVersion}}
networking:
dnsDomain: cluster.local
podSubnet: ""
podSubnet: {{if .PodSubnet}}{{.PodSubnet}}{{else}}""{{end}}
serviceSubnet: {{.ServiceCIDR}}
---
apiVersion: kubelet.config.k8s.io/v1beta1
@ -177,10 +177,6 @@ RestartSec=10
WantedBy=multi-user.target
`
var kubeadmInitTemplate = template.Must(template.New("kubeadmInitTemplate").Parse(`
sudo /usr/bin/kubeadm init --config {{.KubeadmConfigFile}} {{.ExtraOptions}} {{if .SkipPreflightChecks}}--skip-preflight-checks{{else}}{{range .Preflights}}--ignore-preflight-errors={{.}} {{end}}{{end}}
`))
// printMapInOrder sorts the keys and prints the map in order, combining key
// value pairs with the separator character
//

2
pkg/minikube/bootstrapper/kubeadm/testdata/containerd-api-port__default.yaml vendored
View File

@ -28,7 +28,7 @@ dns:
etcd:
local:
dataDir: /data/minikube
kubernetesVersion: v1.14.1
kubernetesVersion: v1.14.2
networking:
dnsDomain: cluster.local
podSubnet: ""

43
pkg/minikube/bootstrapper/kubeadm/testdata/containerd-pod-network-cidr__default.yaml vendored Normal file
View File

@ -0,0 +1,43 @@
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 1.1.1.1
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: mk
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
certificatesDir: /var/lib/minikube/certs/
clusterName: kubernetes
controlPlaneEndpoint: localhost:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /data/minikube
kubernetesVersion: v1.14.2
networking:
dnsDomain: cluster.local
podSubnet: ""
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"

43
pkg/minikube/bootstrapper/kubeadm/testdata/containerd-pod-network-cidr__new.yaml vendored Normal file
View File

@ -0,0 +1,43 @@
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 1.1.1.1
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: mk
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
extraArgs:
enable-admission-plugins: "NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
certificatesDir: /var/lib/minikube/certs/
clusterName: kubernetes
controlPlaneEndpoint: localhost:8443
dns:
type: CoreDNS
etcd:
local:
dataDir: /data/minikube
kubernetesVersion: v1.14.0
networking:
dnsDomain: cluster.local
podSubnet: ""
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
imageGCHighThresholdPercent: 100
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"

17
pkg/minikube/bootstrapper/kubeadm/testdata/containerd-pod-network-cidr__obsolete.yaml vendored Normal file
View File

@ -0,0 +1,17 @@
apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
noTaintMaster: true
api:
advertiseAddress: 1.1.1.1
bindPort: 8443
controlPlaneEndpoint: localhost
kubernetesVersion: v1.10.0
certificatesDir: /var/lib/minikube/certs/
networking:
serviceSubnet: 10.96.0.0/12
etcd:
dataDir: /data/minikube
nodeName: mk
criSocket: /run/containerd/containerd.sock
apiServerExtraArgs:
admission-control: "Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"

39
pkg/minikube/bootstrapper/kubeadm/testdata/containerd-pod-network-cidr__old.yaml vendored Normal file
View File

@ -0,0 +1,39 @@
apiVersion: kubeadm.k8s.io/v1alpha3
kind: InitConfiguration
apiEndpoint:
advertiseAddress: 1.1.1.1
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: mk
taints: []
---
apiVersion: kubeadm.k8s.io/v1alpha3
kind: ClusterConfiguration
apiServerExtraArgs:
enable-admission-plugins: "Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
certificatesDir: /var/lib/minikube/certs/
clusterName: kubernetes
controlPlaneEndpoint: localhost:8443
etcd:
local:
dataDir: /data/minikube
kubernetesVersion: v1.12.0
networking:
dnsDomain: cluster.local
podSubnet: 192.168.32.0/20
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"

39
pkg/minikube/bootstrapper/kubeadm/testdata/containerd-pod-network-cidr__recent.yaml vendored Normal file
View File

@ -0,0 +1,39 @@
apiVersion: kubeadm.k8s.io/v1alpha3
kind: InitConfiguration
apiEndpoint:
advertiseAddress: 1.1.1.1
bindPort: 8443
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
ttl: 24h0m0s
usages:
- signing
- authentication
nodeRegistration:
criSocket: /run/containerd/containerd.sock
name: mk
taints: []
---
apiVersion: kubeadm.k8s.io/v1alpha3
kind: ClusterConfiguration
apiServerExtraArgs:
enable-admission-plugins: "Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota"
certificatesDir: /var/lib/minikube/certs/
clusterName: kubernetes
controlPlaneEndpoint: localhost:8443
etcd:
local:
dataDir: /data/minikube
kubernetesVersion: v1.13.0
networking:
dnsDomain: cluster.local
podSubnet: 192.168.32.0/20
serviceSubnet: 10.96.0.0/12
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
evictionHard:
nodefs.available: "0%"
nodefs.inodesFree: "0%"
imagefs.available: "0%"

2
pkg/minikube/bootstrapper/kubeadm/testdata/containerd__default.yaml vendored
View File

@ -28,7 +28,7 @@ dns:
etcd:
local:
dataDir: /data/minikube
kubernetesVersion: v1.14.1
kubernetesVersion: v1.14.2
networking:
dnsDomain: cluster.local
podSubnet: ""

5
pkg/minikube/bootstrapper/kubeadm/testdata/crio-options-gates__default.yaml vendored
View File

@ -26,9 +26,6 @@ controllerManager:
extraArgs:
feature-gates: "a=b"
kube-api-burst: "32"
kubeadm:
extraArgs:
feature-gates: "a=b"
scheduler:
extraArgs:
feature-gates: "a=b"
@ -41,7 +38,7 @@ dns:
etcd:
local:
dataDir: /data/minikube
kubernetesVersion: v1.14.1
kubernetesVersion: v1.14.2
networking:
dnsDomain: cluster.local
podSubnet: ""

3
pkg/minikube/bootstrapper/kubeadm/testdata/crio-options-gates__new.yaml vendored
View File

@ -26,9 +26,6 @@ controllerManager:
extraArgs:
feature-gates: "a=b"
kube-api-burst: "32"
kubeadm:
extraArgs:
feature-gates: "a=b"
scheduler:
extraArgs:
feature-gates: "a=b"

2
pkg/minikube/bootstrapper/kubeadm/testdata/crio-options-gates__obsolete.yaml vendored
View File

@ -20,8 +20,6 @@ apiServerExtraArgs:
controllerManagerExtraArgs:
feature-gates: "a=b"
kube-api-burst: "32"
kubeadmExtraArgs:
feature-gates: "a=b"
schedulerExtraArgs:
feature-gates: "a=b"
scheduler-name: "mini-scheduler"

2
pkg/minikube/bootstrapper/kubeadm/testdata/crio-options-gates__old.yaml vendored
View File

@ -24,8 +24,6 @@ apiServerExtraArgs:
controllerManagerExtraArgs:
feature-gates: "a=b"
kube-api-burst: "32"
kubeadmExtraArgs:
feature-gates: "a=b"
schedulerExtraArgs:
feature-gates: "a=b"
scheduler-name: "mini-scheduler"

2
pkg/minikube/bootstrapper/kubeadm/testdata/crio-options-gates__recent.yaml vendored
View File

@ -24,8 +24,6 @@ apiServerExtraArgs:
controllerManagerExtraArgs:
feature-gates: "a=b"
kube-api-burst: "32"
kubeadmExtraArgs:
feature-gates: "a=b"
schedulerExtraArgs:
feature-gates: "a=b"
scheduler-name: "mini-scheduler"

2
pkg/minikube/bootstrapper/kubeadm/testdata/crio__default.yaml vendored
View File

@ -28,7 +28,7 @@ dns:
etcd:
local:
dataDir: /data/minikube
kubernetesVersion: v1.14.1
kubernetesVersion: v1.14.2
networking:
dnsDomain: cluster.local
podSubnet: ""

2
pkg/minikube/bootstrapper/kubeadm/testdata/default__default.yaml vendored
View File

@ -28,7 +28,7 @@ dns:
etcd:
local:
dataDir: /data/minikube
kubernetesVersion: v1.14.1
kubernetesVersion: v1.14.2
networking:
dnsDomain: cluster.local
podSubnet: ""

2
pkg/minikube/bootstrapper/kubeadm/testdata/image-repository__default.yaml vendored
View File

@ -29,7 +29,7 @@ dns:
etcd:
local:
dataDir: /data/minikube
kubernetesVersion: v1.14.1
kubernetesVersion: v1.14.2
networking:
dnsDomain: cluster.local
podSubnet: ""

2
pkg/minikube/bootstrapper/kubeadm/testdata/options__default.yaml vendored
View File

@ -35,7 +35,7 @@ dns:
etcd:
local:
dataDir: /data/minikube
kubernetesVersion: v1.14.1
kubernetesVersion: v1.14.2
networking:
dnsDomain: cluster.local
podSubnet: ""

100
pkg/minikube/bootstrapper/kubeadm/util.go
View File

@ -17,11 +17,8 @@ limitations under the License.
package kubeadm
import (
"bytes"
"encoding/json"
"html/template"
"net"
"strings"
"github.com/golang/glog"
"github.com/pkg/errors"
@ -29,10 +26,8 @@ import (
rbac "k8s.io/api/rbac/v1beta1"
apierr "k8s.io/apimachinery/pkg/api/errors"
meta "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/types"
"k8s.io/apimachinery/pkg/util/strategicpatch"
"k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/minikube/constants"
"k8s.io/minikube/pkg/minikube/service"
"k8s.io/minikube/pkg/util"
@ -130,98 +125,3 @@ func elevateKubeSystemPrivileges() error {
}
return nil
}
const (
kubeconfigConf = "kubeconfig.conf"
kubeProxyConfigmapTmpl = `apiVersion: v1
kind: Config
clusters:
- cluster:
certificate-authority: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
server: https://{{.AdvertiseAddress}}:{{.APIServerPort}}
name: default
contexts:
- context:
cluster: default
namespace: default
user: default
name: default
current-context: default
users:
- name: default
user:
tokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
`
)
// updateKubeProxyConfigMap updates the IP & port kube-proxy listens on, and restarts it.
func updateKubeProxyConfigMap(k8s config.KubernetesConfig) error {
client, err := util.GetClient()
if err != nil {
return errors.Wrap(err, "getting k8s client")
}
selector := labels.SelectorFromSet(labels.Set(map[string]string{"k8s-app": "kube-proxy"}))
if err := util.WaitForPodsWithLabelRunning(client, "kube-system", selector); err != nil {
return errors.Wrap(err, "kube-proxy not running")
}
cfgMap, err := client.CoreV1().ConfigMaps("kube-system").Get("kube-proxy", meta.GetOptions{})
if err != nil {
return &util.RetriableError{Err: errors.Wrap(err, "getting kube-proxy configmap")}
}
glog.Infof("kube-proxy config: %v", cfgMap.Data[kubeconfigConf])
t := template.Must(template.New("kubeProxyTmpl").Parse(kubeProxyConfigmapTmpl))
opts := struct {
AdvertiseAddress string
APIServerPort int
}{
AdvertiseAddress: k8s.NodeIP,
APIServerPort: k8s.NodePort,
}
kubeconfig := bytes.Buffer{}
if err := t.Execute(&kubeconfig, opts); err != nil {
return errors.Wrap(err, "executing kube proxy configmap template")
}
if cfgMap.Data == nil {
cfgMap.Data = map[string]string{}
}
updated := strings.TrimSuffix(kubeconfig.String(), "\n")
glog.Infof("updated kube-proxy config: %s", updated)
// An optimization, but also one that's unlikely, as kubeadm writes the address as 'localhost'
if cfgMap.Data[kubeconfigConf] == updated {
glog.Infof("kube-proxy config appears to require no change, not restarting kube-proxy")
return nil
}
cfgMap.Data[kubeconfigConf] = updated
// Make this step retriable, as it can fail with:
// "Operation cannot be fulfilled on configmaps "kube-proxy": the object has been modified; please apply your changes to the latest version and try again"
if _, err := client.CoreV1().ConfigMaps("kube-system").Update(cfgMap); err != nil {
return &util.RetriableError{Err: errors.Wrap(err, "updating configmap")}
}
pods, err := client.CoreV1().Pods("kube-system").List(meta.ListOptions{
LabelSelector: "k8s-app=kube-proxy",
})
if err != nil {
return errors.Wrap(err, "listing kube-proxy pods")
}
for _, pod := range pods.Items {
// Retriable, as known to fail with: pods "<name>" not found
if err := client.CoreV1().Pods(pod.Namespace).Delete(pod.Name, &meta.DeleteOptions{}); err != nil {
return &util.RetriableError{Err: errors.Wrapf(err, "deleting pod %+v", pod)}
}
}
// Wait for the scheduler to restart kube-proxy
if err := util.WaitForPodsWithLabelRunning(client, "kube-system", selector); err != nil {
return errors.Wrap(err, "kube-proxy not running")
}
return nil
}

86
pkg/minikube/cluster/cluster.go
View File

@ -20,9 +20,12 @@ import (
"encoding/json"
"flag"
"fmt"
"math"
"net"
"os/exec"
"regexp"
"strconv"
"strings"
"time"
"github.com/docker/machine/libmachine"
@ -44,6 +47,17 @@ import (
pkgutil "k8s.io/minikube/pkg/util"
)
// hostRunner is a minimal host.Host based interface for running commands
type hostRunner interface {
RunSSHCommand(string) (string, error)
}
var (
// The maximum the guest VM clock is allowed to be ahead and behind. This value is intentionally
// large to allow for inaccurate methodology, but still small enough so that certificates are likely valid.
maxClockDesyncSeconds = 2.1
)
//This init function is used to set the logtostderr variable to false so that INFO level log info does not clutter the CLI
//INFO lvl logging is displayed due to the kubernetes api calling flag.Set("logtostderr", "true") in its init()
//see: https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/util/logs/logs.go#L32-L34
@ -93,7 +107,7 @@ func StartHost(api libmachine.API, config cfg.MachineConfig) (*host.Host, error)
console.Warning("Alternatively, you may delete the existing VM using `minikube delete -p %s`", cfg.GetMachineName())
console.Out("\n")
} else if exists && cfg.GetMachineName() == constants.DefaultMachineName {
console.OutStyle("tip", "Tip: Use 'minikube start -p <name>' to create a new cluster, or 'minikube delete' to delete this one.")
console.OutStyle(console.Tip, "Tip: Use 'minikube start -p <name>' to create a new cluster, or 'minikube delete' to delete this one.")
}
s, err := h.Driver.GetState()
@ -103,9 +117,9 @@ func StartHost(api libmachine.API, config cfg.MachineConfig) (*host.Host, error)
}
if s == state.Running {
console.OutStyle("running", "Re-using the currently running %s VM for %q ...", h.Driver.DriverName(), cfg.GetMachineName())
console.OutStyle(console.Running, "Re-using the currently running %s VM for %q ...", h.Driver.DriverName(), cfg.GetMachineName())
} else {
console.OutStyle("restarting", "Restarting existing %s VM for %q ...", h.Driver.DriverName(), cfg.GetMachineName())
console.OutStyle(console.Restarting, "Restarting existing %s VM for %q ...", h.Driver.DriverName(), cfg.GetMachineName())
if err := h.Driver.Start(); err != nil {
return nil, errors.Wrap(err, "start")
}
@ -117,18 +131,17 @@ func StartHost(api libmachine.API, config cfg.MachineConfig) (*host.Host, error)
e := engineOptions(config)
glog.Infof("engine options: %+v", e)
err = waitForSSHAccess(h, e)
err = configureHost(h, e)
if err != nil {
return nil, err
}
return h, nil
}
func waitForSSHAccess(h *host.Host, e *engine.Options) error {
// configureHost handles any post-powerup configuration required
func configureHost(h *host.Host, e *engine.Options) error {
// Slightly counter-intuitive, but this is what DetectProvisioner & ConfigureAuth block on.
console.OutStyle("waiting", "Waiting for SSH access ...")
console.OutStyle(console.Waiting, "Waiting for SSH access ...")
if len(e.Env) > 0 {
h.HostOptions.EngineOptions.Env = e.Env
@ -145,11 +158,60 @@ func waitForSSHAccess(h *host.Host, e *engine.Options) error {
if err := h.ConfigureAuth(); err != nil {
return &util.RetriableError{Err: errors.Wrap(err, "Error configuring auth on host")}
}
return ensureSyncedGuestClock(h)
}
return nil
}
// ensureGuestClockSync ensures that the guest system clock is relatively in-sync
func ensureSyncedGuestClock(h hostRunner) error {
d, err := guestClockDelta(h, time.Now())
if err != nil {
glog.Warningf("Unable to measure system clock delta: %v", err)
return nil
}
if math.Abs(d.Seconds()) < maxClockDesyncSeconds {
glog.Infof("guest clock delta is within tolerance: %s", d)
return nil
}
if err := adjustGuestClock(h, time.Now()); err != nil {
return errors.Wrap(err, "adjusting system clock")
}
return nil
}
// guestClockDelta returns the approximate difference between the host and guest system clock
// NOTE: This does not currently take into account ssh latency.
func guestClockDelta(h hostRunner, local time.Time) (time.Duration, error) {
out, err := h.RunSSHCommand("date +%s.%N")
if err != nil {
return 0, errors.Wrap(err, "get clock")
}
glog.Infof("guest clock: %s", out)
ns := strings.Split(strings.TrimSpace(out), ".")
secs, err := strconv.ParseInt(strings.TrimSpace(ns[0]), 10, 64)
if err != nil {
return 0, errors.Wrap(err, "atoi")
}
nsecs, err := strconv.ParseInt(strings.TrimSpace(ns[1]), 10, 64)
if err != nil {
return 0, errors.Wrap(err, "atoi")
}
// NOTE: In a synced state, remote is a few hundred ms ahead of local
remote := time.Unix(secs, nsecs)
d := remote.Sub(local)
glog.Infof("Guest: %s Remote: %s (delta=%s)", remote, local, d)
return d, nil
}
// adjustSystemClock adjusts the guest system clock to be nearer to the host system clock
func adjustGuestClock(h hostRunner, t time.Time) error {
out, err := h.RunSSHCommand(fmt.Sprintf("sudo date -s @%d", t.Unix()))
glog.Infof("clock set: %s (err=%v)", out, err)
return err
}
// trySSHPowerOff runs the poweroff command on the guest VM to speed up deletion
func trySSHPowerOff(h *host.Host) {
s, err := h.Driver.GetState()
@ -162,7 +224,7 @@ func trySSHPowerOff(h *host.Host) {
return
}
console.OutStyle("shutdown", "Powering off %q via SSH ...", cfg.GetMachineName())
console.OutStyle(console.Shutdown, "Powering off %q via SSH ...", cfg.GetMachineName())
out, err := h.RunSSHCommand("sudo poweroff")
// poweroff always results in an error, since the host disconnects.
glog.Infof("poweroff result: out=%s, err=%v", out, err)
@ -174,7 +236,7 @@ func StopHost(api libmachine.API) error {
if err != nil {
return errors.Wrapf(err, "load")
}
console.OutStyle("stopping", "Stopping %q in %s ...", cfg.GetMachineName(), host.DriverName)
console.OutStyle(console.Stopping, "Stopping %q in %s ...", cfg.GetMachineName(), host.DriverName)
if err := host.Stop(); err != nil {
alreadyInStateError, ok := err.(mcnerror.ErrHostAlreadyInState)
if ok && alreadyInStateError.State == state.Stopped {
@ -196,7 +258,7 @@ func DeleteHost(api libmachine.API) error {
trySSHPowerOff(host)
}
console.OutStyle("deleting-host", "Deleting %q from %s ...", cfg.GetMachineName(), host.DriverName)
console.OutStyle(console.DeletingHost, "Deleting %q from %s ...", cfg.GetMachineName(), host.DriverName)
if err := host.Driver.Remove(); err != nil {
return errors.Wrap(err, "host remove")
}
@ -284,7 +346,7 @@ To disable this message, run [minikube config set WantShowDriverDeprecationNotif
func createHost(api libmachine.API, config cfg.MachineConfig) (*host.Host, error) {
preCreateHost(&config)
console.OutStyle("starting-vm", "Creating %s VM (CPUs=%d, Memory=%dMB, Disk=%dMB) ...", config.VMDriver, config.CPUs, config.Memory, config.DiskSize)
console.OutStyle(console.StartingVM, "Creating %s VM (CPUs=%d, Memory=%dMB, Disk=%dMB) ...", config.VMDriver, config.CPUs, config.Memory, config.DiskSize)
def, err := registry.Driver(config.VMDriver)
if err != nil {
if err == registry.ErrDriverNotFound {

19
pkg/minikube/cluster/cluster_test.go
View File

@ -17,8 +17,10 @@ limitations under the License.
package cluster
import (
"fmt"
"os"
"testing"
"time"
"github.com/docker/machine/libmachine/drivers"
"github.com/docker/machine/libmachine/host"
@ -382,3 +384,20 @@ func TestCreateSSHShell(t *testing.T) {
t.Fatalf("Expected ssh session to be run")
}
}
func TestGuestClockDelta(t *testing.T) {
local := time.Now()
h := tests.NewMockHost()
// Truncate remote clock so that it is between 0 and 1 second behind
h.CommandOutput["date +%s.%N"] = fmt.Sprintf("%d.0000", local.Unix())
got, err := guestClockDelta(h, local)
if err != nil {
t.Fatalf("guestClock: %v", err)
}
if got > (0 * time.Second) {
t.Errorf("unexpected positive delta (remote should be behind): %s", got)
}
if got < (-1 * time.Second) {
t.Errorf("unexpectedly negative delta (remote too far behind): %s", got)
}
}

35
pkg/minikube/console/console.go
View File

@ -18,6 +18,7 @@ limitations under the License.
package console
import (
"fmt"
"io"
"os"
"strconv"
@ -34,7 +35,7 @@ import (
//
// console.SetOutFile(os.Stdout)
// console.Out("Starting up!")
// console.OutStyle("status-change", "Configuring things")
// console.OutStyle(console.StatusChange, "Configuring things")
// console.SetErrFile(os.Stderr)
// console.Fatal("Oh no, everything failed.")
@ -58,17 +59,9 @@ type fdWriter interface {
Fd() uintptr
}
// HasStyle checks if a style exists
func HasStyle(style string) bool {
return hasStyle(style)
}
// OutStyle writes a stylized and formatted message to stdout
func OutStyle(style, format string, a ...interface{}) {
outStyled, err := applyStyle(style, useColor, format, a...)
if err != nil {
glog.Errorf("applyStyle(%s): %v", style, err)
}
func OutStyle(style StyleEnum, format string, a ...interface{}) {
outStyled := applyStyle(style, useColor, format, a...)
// escape any outstanding '%' signs so that they don't get interpreted
// as a formatting directive down the line
@ -80,7 +73,7 @@ func OutStyle(style, format string, a ...interface{}) {
func Out(format string, a ...interface{}) {
p := message.NewPrinter(translate.GetPreferredLanguage())
if outFile == nil {
glog.Errorf("no output file has been set")
glog.Warningf("[unset outFile]: %s", fmt.Sprintf(format, a...))
return
}
_, err := p.Fprintf(outFile, format, a...)
@ -95,12 +88,8 @@ func OutLn(format string, a ...interface{}) {
}
// ErrStyle writes a stylized and formatted error message to stderr
func ErrStyle(style, format string, a ...interface{}) {
format, err := applyStyle(style, useColor, format, a...)
if err != nil {
glog.Errorf("applyStyle(%s): %v", style, err)
ErrLn(format, a...)
}
func ErrStyle(style StyleEnum, format string, a ...interface{}) {
format = applyStyle(style, useColor, format, a...)
// escape any outstanding '%' signs so that they don't get interpreted
// as a formatting directive down the line
@ -112,7 +101,7 @@ func ErrStyle(style, format string, a ...interface{}) {
func Err(format string, a ...interface{}) {
p := message.NewPrinter(translate.GetPreferredLanguage())
if errFile == nil {
glog.Errorf("no error file has been set")
glog.Errorf("[unset errFile]: %s", fmt.Sprintf(format, a...))
return
}
_, err := p.Fprintf(errFile, format, a...)
@ -128,22 +117,22 @@ func ErrLn(format string, a ...interface{}) {
// Success is a shortcut for writing a styled success message to stdout
func Success(format string, a ...interface{}) {
OutStyle("success", format, a...)
OutStyle(SuccessType, format, a...)
}
// Fatal is a shortcut for writing a styled fatal message to stderr
func Fatal(format string, a ...interface{}) {
ErrStyle("fatal", format, a...)
ErrStyle(FatalType, format, a...)
}
// Warning is a shortcut for writing a styled warning message to stderr
func Warning(format string, a ...interface{}) {
ErrStyle("warning", format, a...)
ErrStyle(WarningType, format, a...)
}
// Failure is a shortcut for writing a styled failure message to stderr
func Failure(format string, a ...interface{}) {
ErrStyle("failure", format, a...)
ErrStyle(FailureType, format, a...)
}
// SetOutFile configures which writer standard output goes to.

20
pkg/minikube/console/console_test.go
View File

@ -51,23 +51,23 @@ func (f *fakeFile) String() string {
func TestOutStyle(t *testing.T) {
var tests = []struct {
style string
style StyleEnum
message string
params []interface{}
want string
wantASCII string
}{
{"happy", "Happy", nil, "😄 Happy\n", "* Happy\n"},
{"option", "Option", nil, " ▪ Option\n", " - Option\n"},
{"warning", "Warning", nil, "⚠️ Warning\n", "! Warning\n"},
{"fatal", "Fatal: %v", []interface{}{"ugh"}, "💣 Fatal: ugh\n", "X Fatal: ugh\n"},
{"waiting-pods", "wait", nil, "⌛ wait", "* wait"},
{"issue", "http://i/%d", []interface{}{10000}, " ▪ http://i/10000\n", " - http://i/10000\n"},
{"usage", "raw: %s %s", []interface{}{"'%'", "%d"}, "💡 raw: '%' %d\n", "* raw: '%' %d\n"},
{Happy, "Happy", nil, "😄 Happy\n", "* Happy\n"},
{Option, "Option", nil, " ▪ Option\n", " - Option\n"},
{WarningType, "Warning", nil, "⚠️ Warning\n", "! Warning\n"},
{FatalType, "Fatal: %v", []interface{}{"ugh"}, "💣 Fatal: ugh\n", "X Fatal: ugh\n"},
{WaitingPods, "wait", nil, "⌛ wait", "* wait"},
{Issue, "http://i/%d", []interface{}{10000}, " ▪ http://i/10000\n", " - http://i/10000\n"},
{Usage, "raw: %s %s", []interface{}{"'%'", "%d"}, "💡 raw: '%' %d\n", "* raw: '%' %d\n"},
}
for _, tc := range tests {
for _, override := range []bool{true, false} {
t.Run(fmt.Sprintf("%s-override-%v", tc.style, override), func(t *testing.T) {
t.Run(fmt.Sprintf("%s-override-%v", tc.message, override), func(t *testing.T) {
// Set MINIKUBE_IN_STYLE=<override>
os.Setenv(OverrideEnv, strconv.FormatBool(override))
f := newFakeFile()
@ -138,7 +138,7 @@ func TestErrStyle(t *testing.T) {
os.Setenv(OverrideEnv, "1")
f := newFakeFile()
SetErrFile(f)
ErrStyle("fatal", "error: %s", "%s%%%d")
ErrStyle(FatalType, "error: %s", "%s%%%d")
got := f.String()
want := "💣 error: %s%%%d\n"
if got != want {

132
pkg/minikube/console/style.go
View File

@ -17,7 +17,6 @@ limitations under the License.
package console
import (
"fmt"
"strings"
"golang.org/x/text/message"
@ -48,67 +47,67 @@ type style struct {
// styles is a map of style name to style struct
// For consistency, ensure that emojis added render with the same width across platforms.
var styles = map[string]style{
"happy": {Prefix: "😄 "},
"success": {Prefix: "✅ "},
"failure": {Prefix: "❌ "},
"conflict": {Prefix: "💥 ", LowPrefix: lowWarning},
"fatal": {Prefix: "💣 ", LowPrefix: lowError},
"notice": {Prefix: "📌 "},
"ready": {Prefix: "🏄 "},
"running": {Prefix: "🏃 "},
"provisioning": {Prefix: "🌱 "},
"restarting": {Prefix: "🔄 "},
"reconfiguring": {Prefix: "📯 "},
"stopping": {Prefix: "✋ "},
"stopped": {Prefix: "🛑 "},
"warning": {Prefix: "⚠️ ", LowPrefix: lowWarning},
"waiting": {Prefix: "⌛ "},
"waiting-pods": {Prefix: "⌛ ", OmitNewline: true},
"usage": {Prefix: "💡 "},
"launch": {Prefix: "🚀 "},
"sad": {Prefix: "😿 "},
"thumbs-up": {Prefix: "👍 "},
"option": {Prefix: " ▪ ", LowPrefix: lowIndent}, // Indented bullet
"command": {Prefix: " ▪ ", LowPrefix: lowIndent}, // Indented bullet
"log-entry": {Prefix: " "}, // Indent
"crushed": {Prefix: "💔 "},
"url": {Prefix: "👉 ", LowPrefix: lowIndent},
"documentation": {Prefix: "📘 "},
"issues": {Prefix: "⁉️ "},
"issue": {Prefix: " ▪ ", LowPrefix: lowIndent}, // Indented bullet
"check": {Prefix: "✔️ "},
var styles = map[StyleEnum]style{
Happy: {Prefix: "😄 "},
SuccessType: {Prefix: "✅ "},
FailureType: {Prefix: "❌ "},
Conflict: {Prefix: "💥 ", LowPrefix: lowWarning},
FatalType: {Prefix: "💣 ", LowPrefix: lowError},
Notice: {Prefix: "📌 "},
Ready: {Prefix: "🏄 "},
Running: {Prefix: "🏃 "},
Provisioning: {Prefix: "🌱 "},
Restarting: {Prefix: "🔄 "},
Reconfiguring: {Prefix: "📯 "},
Stopping: {Prefix: "✋ "},
Stopped: {Prefix: "🛑 "},
WarningType: {Prefix: "⚠️ ", LowPrefix: lowWarning},
Waiting: {Prefix: "⌛ "},
WaitingPods: {Prefix: "⌛ ", OmitNewline: true},
Usage: {Prefix: "💡 "},
Launch: {Prefix: "🚀 "},
Sad: {Prefix: "😿 "},
ThumbsUp: {Prefix: "👍 "},
Option: {Prefix: " ▪ ", LowPrefix: lowIndent}, // Indented bullet
Command: {Prefix: " ▪ ", LowPrefix: lowIndent}, // Indented bullet
LogEntry: {Prefix: " "}, // Indent
Crushed: {Prefix: "💔 "},
URL: {Prefix: "👉 ", LowPrefix: lowIndent},
Documentation: {Prefix: "📘 "},
Issues: {Prefix: "⁉️ "},
Issue: {Prefix: " ▪ ", LowPrefix: lowIndent}, // Indented bullet
Check: {Prefix: "✔️ "},
// Specialized purpose styles
"iso-download": {Prefix: "💿 "},
"file-download": {Prefix: "💾 "},
"caching": {Prefix: "🤹 "},
"starting-vm": {Prefix: "🔥 "},
"starting-none": {Prefix: "🤹 "},
"resetting": {Prefix: "🔄 "},
"deleting-host": {Prefix: "🔥 "},
"copying": {Prefix: "✨ "},
"connectivity": {Prefix: "📶 "},
"internet": {Prefix: "🌐 "},
"mounting": {Prefix: "📁 "},
"celebrate": {Prefix: "🎉 "},
"container-runtime": {Prefix: "🎁 "},
"Docker": {Prefix: "🐳 "},
"CRI-O": {Prefix: "🎁 "}, // This should be a snow-flake, but the emoji has a strange width on macOS
"containerd": {Prefix: "📦 "},
"permissions": {Prefix: "🔑 "},
"enabling": {Prefix: "🔌 "},
"shutdown": {Prefix: "🛑 "},
"pulling": {Prefix: "🚜 "},
"verifying": {Prefix: "🤔 "},
"verifying-noline": {Prefix: "🤔 ", OmitNewline: true},
"kubectl": {Prefix: "💗 "},
"meh": {Prefix: "🙄 ", LowPrefix: lowWarning},
"embarrassed": {Prefix: "🤦 ", LowPrefix: lowWarning},
"tip": {Prefix: "💡 "},
"unmount": {Prefix: "🔥 "},
"mount-options": {Prefix: "💾 "},
"fileserver": {Prefix: "🚀 ", OmitNewline: true},
ISODownload: {Prefix: "💿 "},
FileDownload: {Prefix: "💾 "},
Caching: {Prefix: "🤹 "},
StartingVM: {Prefix: "🔥 "},
StartingNone: {Prefix: "🤹 "},
Resetting: {Prefix: "🔄 "},
DeletingHost: {Prefix: "🔥 "},
Copying: {Prefix: "✨ "},
Connectivity: {Prefix: "📶 "},
Internet: {Prefix: "🌐 "},
Mounting: {Prefix: "📁 "},
Celebrate: {Prefix: "🎉 "},
ContainerRuntime: {Prefix: "🎁 "},
Docker: {Prefix: "🐳 "},
CRIO: {Prefix: "🎁 "}, // This should be a snow-flake, but the emoji has a strange width on macOS
Containerd: {Prefix: "📦 "},
Permissions: {Prefix: "🔑 "},
Enabling: {Prefix: "🔌 "},
Shutdown: {Prefix: "🛑 "},
Pulling: {Prefix: "🚜 "},
Verifying: {Prefix: "🤔 "},
VerifyingNoLine: {Prefix: "🤔 ", OmitNewline: true},
Kubectl: {Prefix: "💗 "},
Meh: {Prefix: "🙄 ", LowPrefix: lowWarning},
Embarrassed: {Prefix: "🤦 ", LowPrefix: lowWarning},
Tip: {Prefix: "💡 "},
Unmount: {Prefix: "🔥 "},
MountOptions: {Prefix: "💾 "},
Fileserver: {Prefix: "🚀 ", OmitNewline: true},
}
// Add a prefix to a string
@ -120,11 +119,6 @@ func applyPrefix(prefix, format string) string {
return prefix + format
}
func hasStyle(style string) bool {
_, exists := styles[style]
return exists
}
// lowPrefix returns a 7-bit compatible prefix for a style
func lowPrefix(s style) string {
if s.LowPrefix != "" {
@ -137,7 +131,7 @@ func lowPrefix(s style) string {
}
// Apply styling to a format string
func applyStyle(style string, useColor bool, format string, a ...interface{}) (string, error) {
func applyStyle(style StyleEnum, useColor bool, format string, a ...interface{}) string {
p := message.NewPrinter(translate.GetPreferredLanguage())
for i, x := range a {
if _, ok := x.(int); ok {
@ -154,11 +148,11 @@ func applyStyle(style string, useColor bool, format string, a ...interface{}) (s
// Similar to CSS styles, if no style matches, output an unformatted string.
if !ok {
return p.Sprintf(format, a...), fmt.Errorf("unknown style: %q", style)
return p.Sprintf(format, a...)
}
if !useColor {
return applyPrefix(lowPrefix(s), out), nil
return applyPrefix(lowPrefix(s), out)
}
return applyPrefix(s.Prefix, out), nil
return applyPrefix(s.Prefix, out)
}

80
pkg/minikube/console/style_enum.go Normal file
View File

@ -0,0 +1,80 @@
/*
Copyright 2019 The Kubernetes Authors All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package console
type StyleEnum int
const (
Happy StyleEnum = iota
SuccessType
FailureType
Conflict
FatalType
Notice
Ready
Running
Provisioning
Restarting
Reconfiguring
Stopping
Stopped
WarningType
Waiting
WaitingPods
Usage
Launch
Sad
ThumbsUp
Option
Command
LogEntry
Crushed
URL
Documentation
Issues
Issue
Check
ISODownload
FileDownload
Caching
StartingVM
StartingNone
Resetting
DeletingHost
Copying
Connectivity
Internet
Mounting
Celebrate
ContainerRuntime
Docker
CRIO
Containerd
Permissions
Enabling
Shutdown
Pulling
Verifying
VerifyingNoLine
Kubectl
Meh
Embarrassed
Tip
Unmount
MountOptions
Fileserver
)

7
pkg/minikube/constants/constants.go
View File

@ -162,10 +162,10 @@ var DefaultISOURL = fmt.Sprintf("https://storage.googleapis.com/%s/minikube-%s.i
var DefaultISOSHAURL = DefaultISOURL + SHASuffix
// DefaultKubernetesVersion is the default kubernetes version
var DefaultKubernetesVersion = "v1.14.1"
var DefaultKubernetesVersion = "v1.14.2"
// NewestKubernetesVersion is the newest Kubernetes version to test against
var NewestKubernetesVersion = "v1.14.1"
var NewestKubernetesVersion = "v1.14.2"
// OldestKubernetesVersion is the oldest Kubernetes version to test against
var OldestKubernetesVersion = "v1.10.13"
@ -181,9 +181,6 @@ func GetProfileFile(profile string) string {
return filepath.Join(GetMinipath(), "profiles", profile, "config.json")
}
// DockerAPIVersion is the API version implemented by Docker running in the minikube VM.
const DockerAPIVersion = "1.35"
// ReportingURL is the URL for reporting a minikube error
const ReportingURL = "https://clouderrorreporting.googleapis.com/v1beta1/projects/k8s-minikube/events:report?key=AIzaSyACUwzG0dEPcl-eOgpDKnyKoUFgHdfoFuA"

5
pkg/minikube/cruntime/containerd.go
View File

@ -21,6 +21,7 @@ import (
"strings"
"github.com/golang/glog"
"k8s.io/minikube/pkg/minikube/console"
)
// Containerd contains containerd runtime state
@ -34,6 +35,10 @@ func (r *Containerd) Name() string {
return "containerd"
}
func (r *Containerd) Style() console.StyleEnum {
return console.Containerd
}
// Version retrieves the current version of this runtime
func (r *Containerd) Version() (string, error) {
ver, err := r.Runner.CombinedOutput("containerd --version")

6
pkg/minikube/cruntime/crio.go
View File

@ -21,6 +21,7 @@ import (
"strings"
"github.com/golang/glog"
"k8s.io/minikube/pkg/minikube/console"
)
// CRIO contains CRIO runtime state
@ -34,6 +35,11 @@ func (r *CRIO) Name() string {
return "CRI-O"
}
// Name is a human readable name for CRIO
func (r *CRIO) Style() console.StyleEnum {
return console.CRIO
}
// Version retrieves the current version of this runtime
func (r *CRIO) Version() (string, error) {
ver, err := r.Runner.CombinedOutput("crio --version")

3
pkg/minikube/cruntime/cruntime.go
View File

@ -22,6 +22,7 @@ import (
"github.com/golang/glog"
"github.com/pkg/errors"
"k8s.io/minikube/pkg/minikube/console"
)
// CommandRunner is the subset of bootstrapper.CommandRunner this package consumes
@ -44,6 +45,8 @@ type Manager interface {
Active() bool
// Available returns an error if it is not possible to use this runtime on a host
Available() error
// Style is an associated StyleEnum for Name()
Style() console.StyleEnum
// KubeletOptions returns kubelet options for a runtime.
KubeletOptions() map[string]string

4
pkg/minikube/cruntime/cruntime_test.go
View File

@ -23,7 +23,6 @@ import (
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"k8s.io/minikube/pkg/minikube/console"
)
func TestName(t *testing.T) {
@ -47,9 +46,6 @@ func TestName(t *testing.T) {
if got != tc.want {
t.Errorf("Name(%s) = %q, want: %q", tc.runtime, got, tc.want)
}
if !console.HasStyle(got) {
t.Fatalf("console.HasStyle(%s): %v", got, false)
}
})
}
}

6
pkg/minikube/cruntime/docker.go
View File

@ -22,6 +22,7 @@ import (
"strings"
"github.com/golang/glog"
"k8s.io/minikube/pkg/minikube/console"
)
// Docker contains Docker runtime state
@ -35,6 +36,11 @@ func (r *Docker) Name() string {
return "Docker"
}
// Name is a human readable name for Docker
func (r *Docker) Style() console.StyleEnum {
return console.Docker
}
// Version retrieves the current version of this runtime
func (r *Docker) Version() (string, error) {
// Note: the server daemon has to be running, for this call to return successfully

1
pkg/minikube/drivers/hyperv/driver.go
View File

@ -46,6 +46,7 @@ func createHypervHost(config cfg.MachineConfig) interface{} {
d.CPU = config.CPUs
d.DiskSize = int(config.DiskSize)
d.SSHUser = "docker"
d.DisableDynamicMemory = true //default to disable dynamic memory as minikube is unlikely to work properly with dynamic memory
return d
}

14
pkg/minikube/exit/exit.go
View File

@ -46,7 +46,7 @@ const (
// Usage outputs a usage error and exits with error code 64
func Usage(format string, a ...interface{}) {
console.ErrStyle("usage", format, a...)
console.ErrStyle(console.Usage, format, a...)
os.Exit(BadUsage)
}
@ -74,8 +74,8 @@ func WithProblem(msg string, p *problem.Problem) {
console.Fatal(msg)
p.Display()
console.Err("\n")
console.ErrStyle("sad", "If the above advice does not help, please let us know: ")
console.ErrStyle("url", "https://github.com/kubernetes/minikube/issues/new")
console.ErrStyle(console.Sad, "If the above advice does not help, please let us know: ")
console.ErrStyle(console.URL, "https://github.com/kubernetes/minikube/issues/new")
os.Exit(Config)
}
@ -84,12 +84,12 @@ func WithLogEntries(msg string, err error, entries map[string][]string) {
displayError(msg, err)
for name, lines := range entries {
console.OutStyle("failure", "Problems detected in %q:", name)
console.OutStyle(console.FailureType, "Problems detected in %q:", name)
if len(lines) > MaxLogEntries {
lines = lines[:MaxLogEntries]
}
for _, l := range lines {
console.OutStyle("log-entry", l)
console.OutStyle(console.LogEntry, l)
}
}
os.Exit(Software)
@ -101,6 +101,6 @@ func displayError(msg string, err error) {
console.Err("\n")
console.Fatal("%s: %v", msg, err)
console.Err("\n")
console.ErrStyle("sad", "Sorry that minikube crashed. If this was unexpected, we would love to hear from you:")
console.ErrStyle("url", "https://github.com/kubernetes/minikube/issues/new")
console.ErrStyle(console.Sad, "Sorry that minikube crashed. If this was unexpected, we would love to hear from you:")
console.ErrStyle(console.URL, "https://github.com/kubernetes/minikube/issues/new")
}

4
pkg/minikube/logs/logs.go
View File

@ -99,12 +99,12 @@ func FindProblems(r cruntime.Manager, bs bootstrapper.Bootstrapper, runner boots
// OutputProblems outputs discovered problems.
func OutputProblems(problems map[string][]string, maxLines int) {
for name, lines := range problems {
console.OutStyle("failure", "Problems detected in %q:", name)
console.OutStyle(console.FailureType, "Problems detected in %q:", name)
if len(lines) > maxLines {
lines = lines[len(lines)-maxLines:]
}
for _, l := range lines {
console.OutStyle("log-entry", l)
console.OutStyle(console.LogEntry, l)
}
}
}

2
pkg/minikube/machine/cache_binaries.go
View File

@ -77,7 +77,7 @@ func CacheBinary(binary, version, osName, archName string) (string, error) {
options.Checksum = constants.GetKubernetesReleaseURLSHA1(binary, version, osName, archName)
options.ChecksumHash = crypto.SHA1
console.OutStyle("file-download", "Downloading %s %s", binary, version)
console.OutStyle(console.FileDownload, "Downloading %s %s", binary, version)
if err := download.ToFile(url, targetFilepath, options); err != nil {
return "", errors.Wrapf(err, "Error downloading %s %s", binary, version)
}

19
pkg/minikube/problem/err_map.go
View File

@ -93,8 +93,8 @@ var vmProblems = map[string]match{
},
"KVM2_START_NO_IP": {
Regexp: re(`Error in driver during machine creation: Machine didn't return an IP after 120 seconds`),
Advice: "The KVM driver is not providing an IP address to the VM. Try checking your libvirt configuration and/or opening an issue",
URL: "https://fedoraproject.org/wiki/How_to_debug_Virtualization_problems#Networking",
Advice: "Install the latest kvm2 driver and run 'virt-host-validate'",
URL: "https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#kvm2-driver",
Issues: []int{4249, 3566},
},
"KVM2_NETWORK_DEFINE_XML": {
@ -103,12 +103,27 @@ var vmProblems = map[string]match{
URL: "https://forums.gentoo.org/viewtopic-t-981692-start-0.html",
Issues: []int{4195},
},
"KVM2_QEMU_MONITOR": {
Regexp: re(`qemu unexpectedly closed the monitor`),
Advice: "Upgrade to QEMU v3.1.0+, run 'virt-host-validate', or ensure that you are not running in a nested VM environment.",
Issues: []int{4277},
},
"KVM_UNAVAILABLE": {
Regexp: re(`invalid argument: could not find capabilities for domaintype=kvm`),
Advice: "Your host does not support KVM virtualization. Ensure that qemu-kvm is installed, and run 'virt-host-validate' to debug the problem",
URL: "http://mikko.repolainen.fi/documents/virtualization-with-kvm",
Issues: []int{2991},
},
"DRIVER_CRASHED": {
Regexp: re(`Error attempting to get plugin server address for RPC`),
Advice: "The VM driver exited with an error, and may be corrupt. Run 'minikube start' with --alsologtostderr -v=8 to see the error",
URL: "https://github.com/kubernetes/minikube/blob/master/docs/drivers.md",
},
"DRIVER_EXITED": {
Regexp: re(`Unable to start VM: create: creating: exit status 1`),
Advice: "Re-run 'minikube start' with --alsologtostderr -v=8 to see the VM driver error message",
URL: "https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#troubleshooting",
},
"VM_BOOT_FAILED_HYPERV_ENABLED": {
Regexp: re(`VirtualBox won't boot a 64bits VM when Hyper-V is activated`),
Advice: "Disable Hyper-V when you want to run VirtualBox to boot the VM",

10
pkg/minikube/problem/problem.go
View File

@ -51,21 +51,21 @@ type match struct {
// Display problem metadata to the console
func (p *Problem) Display() {
console.ErrStyle("failure", "Error: [%s] %v", p.ID, p.Err)
console.ErrStyle("tip", "Advice: %s", p.Advice)
console.ErrStyle(console.FailureType, "Error: [%s] %v", p.ID, p.Err)
console.ErrStyle(console.Tip, "Advice: %s", p.Advice)
if p.URL != "" {
console.ErrStyle("documentation", "Documentation: %s", p.URL)
console.ErrStyle(console.Documentation, "Documentation: %s", p.URL)
}
if len(p.Issues) == 0 {
return
}
console.ErrStyle("issues", "Related issues:")
console.ErrStyle(console.Issues, "Related issues:")
issues := p.Issues
if len(issues) > 3 {
issues = issues[0:3]
}
for _, i := range issues {
console.ErrStyle("issue", "%s/%d", issueBase, i)
console.ErrStyle(console.Issue, "%s/%d", issueBase, i)
}
}

133
pkg/minikube/proxy/proxy.go Normal file
View File

@ -0,0 +1,133 @@
/*
Copyright 2019 The Kubernetes Authors All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package proxy
import (
"fmt"
"net"
"net/http"
"os"
"strings"
"github.com/golang/glog"
"github.com/pkg/errors"
"k8s.io/client-go/rest"
)
// EnvVars are variables we plumb through to the underlying container runtime
var EnvVars = []string{"HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY"}
// isInBlock checks if ip is a CIDR block
func isInBlock(ip string, block string) (bool, error) {
if ip == "" {
return false, fmt.Errorf("ip is nil")
}
if block == "" {
return false, fmt.Errorf("CIDR is nil")
}
i := net.ParseIP(ip)
if i == nil {
return false, fmt.Errorf("parsed IP is nil")
}
_, b, err := net.ParseCIDR(block)
if err != nil {
return false, errors.Wrapf(err, "Error Parsing block %s", b)
}
if b.Contains(i) {
return true, nil
}
return false, errors.Wrapf(err, "Error ip not in block")
}
// ExcludeIP will exclude the ip from the http(s)_proxy
func ExcludeIP(ip string) error {
return updateEnv(ip, "NO_PROXY")
}
// IsIPExcluded checks if an IP is excluded from http(s)_proxy
func IsIPExcluded(ip string) bool {
return checkEnv(ip, "NO_PROXY")
}
// updateEnv appends an ip to the environment variable
func updateEnv(ip string, env string) error {
if ip == "" {
return fmt.Errorf("IP %s is blank. ", ip)
}
if !isValidEnv(env) {
return fmt.Errorf("%s is not a valid env var name for proxy settings", env)
}
if !checkEnv(ip, env) {
v := os.Getenv(env)
if v == "" {
return os.Setenv(env, ip)
}
return os.Setenv(env, fmt.Sprintf("%s,%s", v, ip))
}
return nil
}
// checkEnv checks if ip in an environment variable
func checkEnv(ip string, env string) bool {
v := os.Getenv(env)
if v == "" {
return false
}
// Checking for IP explicitly, i.e., 192.168.39.224
if strings.Contains(v, ip) {
return true
}
// Checks if included in IP ranges, i.e., 192.168.39.13/24
noProxyBlocks := strings.Split(v, ",")
for _, b := range noProxyBlocks {
if yes, _ := isInBlock(ip, b); yes {
return true
}
}
return false
}
// isValidEnv checks if the env for proxy settings
func isValidEnv(env string) bool {
for _, e := range EnvVars {
if e == env {
return true
}
}
return false
}
// UpdateTransport takes a k8s client *rest.config and returns a config without a proxy.
func UpdateTransport(cfg *rest.Config) *rest.Config {
wt := cfg.WrapTransport // Config might already have a transport wrapper
cfg.WrapTransport = func(rt http.RoundTripper) http.RoundTripper {
if wt != nil {
rt = wt(rt)
}
if ht, ok := rt.(*http.Transport); ok {
ht.Proxy = nil
rt = ht
} else {
glog.Errorf("Error while casting RoundTripper (of type %T) to *http.Transport : %v", rt, ok)
}
return rt
}
return cfg
}

151
pkg/minikube/proxy/proxy_test.go Normal file
View File

@ -0,0 +1,151 @@
/*
Copyright 2019 The Kubernetes Authors All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package proxy
import (
"fmt"
"os"
"testing"
)
func TestIsValidEnv(t *testing.T) {
var testCases = []struct {
env string
want bool
}{
{"", false},
{"HTTPS-PROXY", false},
{"NOPROXY", false},
}
for _, tc := range testCases {
t.Run(tc.env, func(t *testing.T) {
got := isValidEnv(tc.env)
if got != tc.want {
t.Errorf("isValidEnv(\"%v\") got %v; want %v", tc.env, got, tc.want)
}
})
}
}
func TestIsInBlock(t *testing.T) {
var testCases = []struct {
ip string
block string
want bool
wanntAErr bool
}{
{"", "192.168.0.1/32", false, true},
{"192.168.0.1", "192.168.0.1/32", true, false},
{"192.168.0.2", "192.168.0.1/32", false, false},
{"192.168.0.1", "192.168.0.1/18", true, false},
{"abcd", "192.168.0.1/18", false, true},
}
for _, tc := range testCases {
t.Run(fmt.Sprintf("%s in %s", tc.ip, tc.block), func(t *testing.T) {
got, err := isInBlock(tc.ip, tc.block)
gotErr := false
if err != nil {
gotErr = true
}
if gotErr != tc.wanntAErr {
t.Errorf("isInBlock(%v,%v) got error is %v ; want error is %v", tc.ip, tc.block, gotErr, tc.wanntAErr)
}
if got != tc.want {
t.Errorf("isInBlock(%v,%v) got %v; want %v", tc.ip, tc.block, got, tc.want)
}
})
}
}
func TestUpdateEnv(t *testing.T) {
var testCases = []struct {
ip string
env string
wantErr bool
}{
{"192.168.0.13", "NO_PROXY", false},
{"", "NO_PROXY", true},
{"", "", true},
{"192.168.0.13", "", true},
{"192.168.0.13", "NPROXY", true},
}
for _, tc := range testCases {
t.Run(fmt.Sprintf("%s in %s", tc.ip, tc.env), func(t *testing.T) {
origVal := os.Getenv(tc.env)
gotErr := false
err := updateEnv(tc.ip, tc.env)
if err != nil {
gotErr = true
}
if gotErr != tc.wantErr {
t.Errorf("updateEnv(%v,%v) got error is %v ; want error is %v", tc.ip, tc.env, gotErr, tc.wantErr)
}
err = os.Setenv(tc.env, origVal)
if err != nil && tc.env != "" {
t.Errorf("Error reverting the env var (%s) to its original value (%s)", tc.env, origVal)
}
})
}
}
func TestCheckEnv(t *testing.T) {
var testCases = []struct {
ip string
envName string
want bool
mockEnvValue string
}{
{"", "NO_PROXY", false, ""},
{"192.168.0.13", "NO_PROXY", false, ""},
{"192.168.0.13", "NO_PROXY", false, ","},
{"192.168.0.13", "NO_PROXY", true, "192.168.0.13"},
{"192.168.0.13", "NO_PROXY", true, ",192.168.0.13"},
{"192.168.0.13", "NO_PROXY", true, "10.10.0.13,192.168.0.13"},
{"192.168.0.13", "NO_PROXY", true, "192.168.0.13/22"},
{"192.168.0.13", "NO_PROXY", true, "10.10.0.13,192.168.0.13"},
{"192.168.0.13", "NO_PROXY", false, "10.10.0.13/22"},
}
for _, tc := range testCases {
t.Run(fmt.Sprintf("%s in %s", tc.ip, tc.envName), func(t *testing.T) {
originalEnv := os.Getenv(tc.envName)
defer func() { // revert to pre-test env var
err := os.Setenv(tc.envName, originalEnv)
if err != nil {
t.Fatalf("Error reverting env (%s) to its original value (%s) var after test ", tc.envName, originalEnv)
}
}()
// defer os.Setenv(tc.envName, originalEnv)
err := os.Setenv(tc.envName, tc.mockEnvValue) // setting up the test case
if err != nil {
t.Error("Error setting env var for taste case")
}
got := checkEnv(tc.ip, tc.envName)
if got != tc.want {
t.Errorf("CheckEnv(%v,%v) got %v ; want is %v", tc.ip, tc.envName, got, tc.want)
}
})
}
}

4
pkg/minikube/service/service.go
View File

@ -40,6 +40,7 @@ import (
"k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/minikube/console"
"k8s.io/minikube/pkg/minikube/constants"
"k8s.io/minikube/pkg/minikube/proxy"
"k8s.io/minikube/pkg/util"
)
@ -84,6 +85,7 @@ func (*K8sClientGetter) GetClientset(timeout time.Duration) (*kubernetes.Clients
return nil, fmt.Errorf("kubeConfig: %v", err)
}
clientConfig.Timeout = timeout
clientConfig = proxy.UpdateTransport(clientConfig)
client, err := kubernetes.NewForConfig(clientConfig)
if err != nil {
return nil, errors.Wrap(err, "client from config")
@ -257,7 +259,7 @@ func WaitAndMaybeOpenService(api libmachine.API, namespace string, service strin
if urlMode || !isHTTPSchemedURL {
console.OutLn(urlString)
} else {
console.OutStyle("celebrate", "Opening kubernetes service %s/%s in default browser...", namespace, service)
console.OutStyle(console.Celebrate, "Opening kubernetes service %s/%s in default browser...", namespace, service)
if err := browser.OpenURL(urlString); err != nil {
console.Err("browser failed to open url: %v", err)
}

12
pkg/minikube/tunnel/cluster_inspector.go
View File

@ -26,6 +26,7 @@ import (
"github.com/pkg/errors"
"k8s.io/minikube/pkg/minikube/cluster"
"k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/util"
)
type clusterInspector struct {
@ -93,9 +94,14 @@ func getRoute(host *host.Host, clusterConfig config.Config) (*Route, error) {
if ip == nil {
return nil, fmt.Errorf("invalid IP for host %s", hostDriverIP)
}
dnsIP, err := util.GetDNSIP(ipNet.String())
if err != nil {
return nil, err
}
return &Route{
Gateway: ip,
DestCIDR: ipNet,
Gateway: ip,
DestCIDR: ipNet,
ClusterDomain: clusterConfig.KubernetesConfig.DNSDomain,
ClusterDNSIP: dnsIP,
}, nil
}

11
pkg/minikube/tunnel/cluster_inspector_test.go
View File

@ -19,6 +19,8 @@ package tunnel
import (
"testing"
"k8s.io/minikube/pkg/util"
"net"
"reflect"
"strings"
@ -78,10 +80,15 @@ func TestMinikubeCheckReturnsHostInformation(t *testing.T) {
ip := net.ParseIP("1.2.3.4")
_, ipNet, _ := net.ParseCIDR("96.0.0.0/12")
dnsIP, err := util.GetDNSIP(ipNet.String())
if err != nil {
t.Errorf("getdnsIP: %v", err)
}
expectedRoute := &Route{
Gateway: ip,
DestCIDR: ipNet,
Gateway: ip,
DestCIDR: ipNet,
ClusterDNSIP: dnsIP,
}
if s != Running {

37
pkg/minikube/tunnel/route_darwin.go
View File

@ -18,7 +18,9 @@ package tunnel
import (
"fmt"
"io/ioutil"
"net"
"os"
"os/exec"
"regexp"
"strings"
@ -34,6 +36,9 @@ func (router *osRouter) EnsureRouteIsAdded(route *Route) error {
if exists {
return nil
}
if err := writeResolverFile(route); err != nil {
return fmt.Errorf("could not write /etc/resolver/{cluster_domain} file: %s", err)
}
serviceCIDR := route.DestCIDR.String()
gatewayIP := route.Gateway.String()
@ -162,5 +167,37 @@ func (router *osRouter) Cleanup(route *Route) error {
if !re.MatchString(message) {
return fmt.Errorf("error deleting route: %s, %d", message, len(strings.Split(message, "\n")))
}
// idempotent removal of cluster domain dns
resolverFile := fmt.Sprintf("/etc/resolver/%s", route.ClusterDomain)
command = exec.Command("sudo", "rm", "-f", resolverFile)
if err := command.Run(); err != nil {
return fmt.Errorf("could not remove %s: %s", resolverFile, err)
}
return nil
}
func writeResolverFile(route *Route) error {
resolverFile := "/etc/resolver/" + route.ClusterDomain
content := fmt.Sprintf("nameserver %s\nsearch_order 1\n", route.ClusterDNSIP)
// write resolver content into tmpFile, then copy it to /etc/resolver/clusterDomain
tmpFile, err := ioutil.TempFile("", "minikube-tunnel-resolver-")
if err != nil {
return err
}
defer os.Remove(tmpFile.Name())
if _, err = tmpFile.WriteString(content); err != nil {
return err
}
if err = tmpFile.Close(); err != nil {
return err
}
command := exec.Command("sudo", "mkdir", "-p", "/etc/resolver")
if err := command.Run(); err != nil {
return err
}
command = exec.Command("sudo", "cp", "-f", tmpFile.Name(), resolverFile)
if err := command.Run(); err != nil {
return err
}
return nil
}

Some files were not shown because too many files have changed in this diff Show More