From 25743dd198ee095cb75c97c6aa6e75cd39edd927 Mon Sep 17 00:00:00 2001 From: Radoslaw Smigielski Date: Thu, 9 Apr 2020 07:54:11 +0100 Subject: [PATCH 1/6] Remove reference to not existing section on Minikube sigs The only content of docs/networking.md is an URL to non-existing page https://minikube.sigs.k8s.io/docs/reference/networking/ --- docs/networking.md | 2 -- 1 file changed, 2 deletions(-) delete mode 100644 docs/networking.md diff --git a/docs/networking.md b/docs/networking.md deleted file mode 100644 index cc3ba1e46e..0000000000 --- a/docs/networking.md +++ /dev/null @@ -1,2 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/reference/networking/ - \ No newline at end of file From bb6e25f8d9213345c4da29a162dccaead292fbfe Mon Sep 17 00:00:00 2001 From: Radoslaw Smigielski Date: Thu, 9 Apr 2020 14:01:42 +0100 Subject: [PATCH 2/6] Remove unused docs/ directory docs/ directory became obsoleted since massive reorganization made in commit 7559952 --- Makefile | 2 +- docs/README.md | 49 ------------------------- docs/accessing_etcd.md | 1 - docs/addons.md | 1 - docs/alternative_runtimes.md | 1 - docs/building_images_within_the_vm.md | 1 - docs/cache.md | 1 - docs/cli_commands.md | 1 - docs/configuring_kubernetes.md | 1 - docs/contributors/README.md | 1 - docs/contributors/adding_an_addon.md | 1 - docs/contributors/adding_driver.md | 1 - docs/contributors/build_guide.md | 1 - docs/contributors/ci_builds.md | 1 - docs/contributors/minikube_iso.md | 1 - docs/contributors/principles.md | 1 - docs/contributors/releasing_minikube.md | 1 - docs/contributors/roadmap.md | 1 - docs/dashboard.md | 1 - docs/debugging.md | 1 - docs/drivers.md | 1 - docs/env_vars.md | 1 - docs/gpu.md | 1 - docs/host_folder_mount.md | 1 - docs/http_proxy.md | 1 - docs/insecure_registry.md | 1 - docs/offline.md | 1 - docs/openid_connect_auth.md | 1 - docs/persistent_volumes.md | 1 - docs/reusing_the_docker_daemon.md | 1 - docs/syncing-files.md | 1 - docs/tunnel.md | 1 - docs/vmdriver-none.md | 1 - 33 files changed, 1 insertion(+), 81 deletions(-) delete mode 100644 docs/README.md delete mode 100644 docs/accessing_etcd.md delete mode 100644 docs/addons.md delete mode 100644 docs/alternative_runtimes.md delete mode 100644 docs/building_images_within_the_vm.md delete mode 100644 docs/cache.md delete mode 100644 docs/cli_commands.md delete mode 100644 docs/configuring_kubernetes.md delete mode 100644 docs/contributors/README.md delete mode 100644 docs/contributors/adding_an_addon.md delete mode 100644 docs/contributors/adding_driver.md delete mode 100644 docs/contributors/build_guide.md delete mode 100644 docs/contributors/ci_builds.md delete mode 100644 docs/contributors/minikube_iso.md delete mode 100644 docs/contributors/principles.md delete mode 100644 docs/contributors/releasing_minikube.md delete mode 100644 docs/contributors/roadmap.md delete mode 100644 docs/dashboard.md delete mode 100644 docs/debugging.md delete mode 100644 docs/drivers.md delete mode 100644 docs/env_vars.md delete mode 100644 docs/gpu.md delete mode 100644 docs/host_folder_mount.md delete mode 100644 docs/http_proxy.md delete mode 100644 docs/insecure_registry.md delete mode 100644 docs/offline.md delete mode 100644 docs/openid_connect_auth.md delete mode 100644 docs/persistent_volumes.md delete mode 100644 docs/reusing_the_docker_daemon.md delete mode 100644 docs/syncing-files.md delete mode 100644 docs/tunnel.md delete mode 100644 docs/vmdriver-none.md diff --git a/Makefile b/Makefile index 1665b17c58..e0fe592f0a 100755 --- a/Makefile +++ b/Makefile @@ -111,7 +111,7 @@ MINIKUBE_TEST_FILES := ./cmd/... ./pkg/... MARKDOWNLINT ?= markdownlint -MINIKUBE_MARKDOWN_FILES := README.md docs CONTRIBUTING.md CHANGELOG.md +MINIKUBE_MARKDOWN_FILES := README.md CONTRIBUTING.md CHANGELOG.md MINIKUBE_BUILD_TAGS := container_image_ostree_stub containers_image_openpgp MINIKUBE_BUILD_TAGS += go_getter_nos3 go_getter_nogcs diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index 16e26d9369..0000000000 --- a/docs/README.md +++ /dev/null @@ -1,49 +0,0 @@ -# Advanced Topics and Tutorials - -## Cluster Configuration - -* **Alternative Runtimes** ([alternative_runtimes.md](alternative_runtimes.md)): How to run minikube without Docker as the container runtime - -* **Environment Variables** ([env_vars.md](env_vars.md)): The different environment variables that minikube understands - -* **Minikube Addons** ([addons.md](addons.md)): Information on configuring addons to be run on minikube - -* **Configuring Kubernetes** ([configuring_kubernetes.md](configuring_kubernetes.md)): Configuring different Kubernetes components in minikube - -* **Caching Images** ([cache.md](cache.md)): Caching non-minikube images in minikube - -* **GPUs** ([gpu.md](gpu.md)): Using NVIDIA GPUs on minikube - -* **OpenID Connect Authentication** ([openid_connect_auth.md](openid_connect_auth.md)): Using OIDC Authentication on minikube - -### Installation and debugging - -* **Driver installation** ([drivers.md](drivers.md)): In depth instructions for installing the various hypervisor drivers - -* **Debugging minikube** ([debugging.md](debugging.md)): General practices for debugging the minikube binary itself - -### Developing on the minikube cluster - -* **Reusing the Docker Daemon** ([reusing_the_docker_daemon.md](reusing_the_docker_daemon.md)): How to point your docker CLI to the docker daemon running inside minikube - -* **Building images within the VM** ([building_images_within_the_vm.md](building_images_within_the_vm.md)): How to build a container image within the minikube VM - -#### Storage - -* **Persistent Volumes** ([persistent_volumes.md](persistent_volumes.md)): Persistent Volumes in Minikube and persisted locations in the VM - -* **Host Folder Mounting** ([host_folder_mount.md](host_folder_mount.md)): How to mount your files from your host into the minikube VM - -* **Syncing files into the VM** ([syncing-files.md](syncing-files.md)): How to sync files from your host into the minikube VM - -#### Networking - -* **HTTP Proxy** ([http_proxy.md](http_proxy.md)): Instruction on how to run minikube behind a HTTP Proxy - -* **Insecure or Private Registries** ([insecure_registry.md](insecure_registry.md)): How to use private or insecure registries with minikube - -* **Accessing etcd from inside the cluster** ([accessing_etcd.md](accessing_etcd.md)) - -* **Networking** ([networking.md](networking.md)): FAQ about networking between the host and minikube VM - -* **Offline** ([offline.md](offline.md)): Details about using minikube offline diff --git a/docs/accessing_etcd.md b/docs/accessing_etcd.md deleted file mode 100644 index b1593333d5..0000000000 --- a/docs/accessing_etcd.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tasks/accessing-host-resources/ \ No newline at end of file diff --git a/docs/addons.md b/docs/addons.md deleted file mode 100644 index ca879ccab8..0000000000 --- a/docs/addons.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tasks/addons/ diff --git a/docs/alternative_runtimes.md b/docs/alternative_runtimes.md deleted file mode 100644 index d0d648734e..0000000000 --- a/docs/alternative_runtimes.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/reference/runtimes/ diff --git a/docs/building_images_within_the_vm.md b/docs/building_images_within_the_vm.md deleted file mode 100644 index 5b22a32031..0000000000 --- a/docs/building_images_within_the_vm.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tasks/building_within/ diff --git a/docs/cache.md b/docs/cache.md deleted file mode 100644 index 475bb7332c..0000000000 --- a/docs/cache.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tasks/caching diff --git a/docs/cli_commands.md b/docs/cli_commands.md deleted file mode 100644 index b547b26a5d..0000000000 --- a/docs/cli_commands.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/reference/commands/ diff --git a/docs/configuring_kubernetes.md b/docs/configuring_kubernetes.md deleted file mode 100644 index cb7db70ae8..0000000000 --- a/docs/configuring_kubernetes.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/reference/configuration/kubernetes/ diff --git a/docs/contributors/README.md b/docs/contributors/README.md deleted file mode 100644 index a8952b3c5f..0000000000 --- a/docs/contributors/README.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/contributing/ diff --git a/docs/contributors/adding_an_addon.md b/docs/contributors/adding_an_addon.md deleted file mode 100644 index 9fae54108c..0000000000 --- a/docs/contributors/adding_an_addon.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/contributing/addons/ \ No newline at end of file diff --git a/docs/contributors/adding_driver.md b/docs/contributors/adding_driver.md deleted file mode 100644 index fb3ad4579d..0000000000 --- a/docs/contributors/adding_driver.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/contributing/drivers/ diff --git a/docs/contributors/build_guide.md b/docs/contributors/build_guide.md deleted file mode 100644 index dddbe5d7b3..0000000000 --- a/docs/contributors/build_guide.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/contributing/building/ \ No newline at end of file diff --git a/docs/contributors/ci_builds.md b/docs/contributors/ci_builds.md deleted file mode 100644 index 33f7b21f42..0000000000 --- a/docs/contributors/ci_builds.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/contributing/building/ diff --git a/docs/contributors/minikube_iso.md b/docs/contributors/minikube_iso.md deleted file mode 100644 index b9bc175ffa..0000000000 --- a/docs/contributors/minikube_iso.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/contributing/iso/ diff --git a/docs/contributors/principles.md b/docs/contributors/principles.md deleted file mode 100644 index 2caffd4f37..0000000000 --- a/docs/contributors/principles.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/concepts/principles/ diff --git a/docs/contributors/releasing_minikube.md b/docs/contributors/releasing_minikube.md deleted file mode 100644 index 4da4199c09..0000000000 --- a/docs/contributors/releasing_minikube.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/contributing/releasing/ diff --git a/docs/contributors/roadmap.md b/docs/contributors/roadmap.md deleted file mode 100644 index 45e47a31e2..0000000000 --- a/docs/contributors/roadmap.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/contributing/roadmap/ diff --git a/docs/dashboard.md b/docs/dashboard.md deleted file mode 100644 index cb7ff8f2d8..0000000000 --- a/docs/dashboard.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tasks/dashboard/ diff --git a/docs/debugging.md b/docs/debugging.md deleted file mode 100644 index b087d9c57f..0000000000 --- a/docs/debugging.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tasks/debug/ diff --git a/docs/drivers.md b/docs/drivers.md deleted file mode 100644 index 7b69ddd912..0000000000 --- a/docs/drivers.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/reference/drivers/ diff --git a/docs/env_vars.md b/docs/env_vars.md deleted file mode 100644 index b90a70617b..0000000000 --- a/docs/env_vars.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/reference/environment_variables diff --git a/docs/gpu.md b/docs/gpu.md deleted file mode 100644 index 56725b80fb..0000000000 --- a/docs/gpu.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tutorials/nvidia_gpu/ diff --git a/docs/host_folder_mount.md b/docs/host_folder_mount.md deleted file mode 100644 index 689f23f986..0000000000 --- a/docs/host_folder_mount.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tasks/mount/ diff --git a/docs/http_proxy.md b/docs/http_proxy.md deleted file mode 100644 index 789508a20f..0000000000 --- a/docs/http_proxy.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/reference/networking/proxy/ diff --git a/docs/insecure_registry.md b/docs/insecure_registry.md deleted file mode 100644 index c02296e26c..0000000000 --- a/docs/insecure_registry.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tasks/registry/ diff --git a/docs/offline.md b/docs/offline.md deleted file mode 100644 index bca6193eec..0000000000 --- a/docs/offline.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/reference/disk_cache/ diff --git a/docs/openid_connect_auth.md b/docs/openid_connect_auth.md deleted file mode 100644 index cbb2515911..0000000000 --- a/docs/openid_connect_auth.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tutorials/openid_connect_auth/ diff --git a/docs/persistent_volumes.md b/docs/persistent_volumes.md deleted file mode 100644 index df28b852af..0000000000 --- a/docs/persistent_volumes.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/reference/persistent_volumes/ diff --git a/docs/reusing_the_docker_daemon.md b/docs/reusing_the_docker_daemon.md deleted file mode 100644 index 2b76f9534b..0000000000 --- a/docs/reusing_the_docker_daemon.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tasks/docker_daemon/ diff --git a/docs/syncing-files.md b/docs/syncing-files.md deleted file mode 100644 index ea217b2d38..0000000000 --- a/docs/syncing-files.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tasks/sync/ diff --git a/docs/tunnel.md b/docs/tunnel.md deleted file mode 100644 index b92c4c1b16..0000000000 --- a/docs/tunnel.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/tasks/loadbalancer/ diff --git a/docs/vmdriver-none.md b/docs/vmdriver-none.md deleted file mode 100644 index 143ec71b2b..0000000000 --- a/docs/vmdriver-none.md +++ /dev/null @@ -1 +0,0 @@ -This document has moved to https://minikube.sigs.k8s.io/docs/reference/drivers/none/ From 903fe744b199915f58ca69fd458943a1ffea58e5 Mon Sep 17 00:00:00 2001 From: Kenta Iso Date: Fri, 10 Apr 2020 00:15:20 +0900 Subject: [PATCH 3/6] Add output deleteingHost using KIC driver --- cmd/minikube/cmd/delete.go | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/cmd/minikube/cmd/delete.go b/cmd/minikube/cmd/delete.go index 5f4ad12513..22c16062d3 100644 --- a/cmd/minikube/cmd/delete.go +++ b/cmd/minikube/cmd/delete.go @@ -208,7 +208,13 @@ func deleteProfileContainersAndVolumes(name string) { func deleteProfile(profile *config.Profile) error { viper.Set(config.ProfileName, profile.Name) - deleteProfileContainersAndVolumes(profile.Name) + if profile.Config != nil { + // if driver is oci driver, delete containers and volumes + if driver.IsKIC(profile.Config.Driver) { + out.T(out.DeletingHost, `Deleting "{{.profile_name}}" in {{.driver_name}} ...`, out.V{"profile_name": profile.Name, "driver_name": profile.Config.Driver}) + deleteProfileContainersAndVolumes(profile.Name) + } + } api, err := machine.NewAPIClient() if err != nil { From bf784d8bf3602a38a9d7da712be38d28820965eb Mon Sep 17 00:00:00 2001 From: Thomas Stromberg Date: Thu, 9 Apr 2020 13:24:14 -0700 Subject: [PATCH 4/6] Refactor CommandRunner.Copy functions for consistency --- pkg/addons/addons.go | 2 +- pkg/gvisor/enable.go | 2 +- pkg/minikube/assets/vm_assets.go | 34 ++++---- pkg/minikube/bootstrapper/certs.go | 3 +- pkg/minikube/command/command_runner.go | 109 +++++++++++++++++++++++-- pkg/minikube/command/exec_runner.go | 34 ++++---- pkg/minikube/command/fake_runner.go | 4 +- pkg/minikube/command/kic_runner.go | 87 +++++++++++++------- pkg/minikube/command/ssh_runner.go | 107 ++++++------------------ pkg/minikube/machine/filesync_test.go | 2 +- 10 files changed, 227 insertions(+), 157 deletions(-) diff --git a/pkg/addons/addons.go b/pkg/addons/addons.go index 63ae1507df..fd4634d0b4 100644 --- a/pkg/addons/addons.go +++ b/pkg/addons/addons.go @@ -197,7 +197,7 @@ func enableOrDisableAddonInternal(cc *config.ClusterConfig, addon *assets.Addon, if addon.IsTemplate() { f, err = addon.Evaluate(data) if err != nil { - return errors.Wrapf(err, "evaluate bundled addon %s asset", addon.GetAssetName()) + return errors.Wrapf(err, "evaluate bundled addon %s asset", addon.GetSourcePath()) } } else { diff --git a/pkg/gvisor/enable.go b/pkg/gvisor/enable.go index 40f69b2fc9..27d0260406 100644 --- a/pkg/gvisor/enable.go +++ b/pkg/gvisor/enable.go @@ -181,7 +181,7 @@ func copyAssetToDest(targetName, dest string) error { log.Printf("%s asset path: %s", targetName, src) contents, err := ioutil.ReadFile(src) if err != nil { - return errors.Wrapf(err, "getting contents of %s", asset.GetAssetName()) + return errors.Wrapf(err, "getting contents of %s", asset.GetSourcePath()) } if _, err := os.Stat(dest); err == nil { if err := os.Remove(dest); err != nil { diff --git a/pkg/minikube/assets/vm_assets.go b/pkg/minikube/assets/vm_assets.go index e3b4678544..3751a85e08 100644 --- a/pkg/minikube/assets/vm_assets.go +++ b/pkg/minikube/assets/vm_assets.go @@ -29,11 +29,15 @@ import ( "github.com/pkg/errors" ) +// MemorySource is the source name used for in-memory copies +const MemorySource = "memory" + // CopyableFile is something that can be copied type CopyableFile interface { io.Reader GetLength() int - GetAssetName() string + GetSourcePath() string + GetTargetDir() string GetTargetName() string GetPermissions() string @@ -43,15 +47,16 @@ type CopyableFile interface { // BaseAsset is the base asset class type BaseAsset struct { - AssetName string + SourcePath string TargetDir string TargetName string Permissions string + Source string } -// GetAssetName returns asset name -func (b *BaseAsset) GetAssetName() string { - return b.AssetName +// GetSourcePath returns asset name +func (b *BaseAsset) GetSourcePath() string { + return b.SourcePath } // GetTargetDir returns target dir @@ -99,7 +104,7 @@ func NewFileAsset(src, targetDir, targetName, permissions string) (*FileAsset, e r := io.NewSectionReader(f, 0, info.Size()) return &FileAsset{ BaseAsset: BaseAsset{ - AssetName: src, + SourcePath: src, TargetDir: targetDir, TargetName: targetName, Permissions: permissions, @@ -110,7 +115,7 @@ func NewFileAsset(src, targetDir, targetName, permissions string) (*FileAsset, e // GetLength returns the file length, or 0 (on error) func (f *FileAsset) GetLength() (flen int) { - fi, err := os.Stat(f.AssetName) + fi, err := os.Stat(f.SourcePath) if err != nil { return 0 } @@ -119,7 +124,7 @@ func (f *FileAsset) GetLength() (flen int) { // GetModTime returns modification time of the file func (f *FileAsset) GetModTime() (time.Time, error) { - fi, err := os.Stat(f.AssetName) + fi, err := os.Stat(f.SourcePath) if err != nil { return time.Time{}, err } @@ -168,6 +173,7 @@ func NewMemoryAsset(d []byte, targetDir, targetName, permissions string) *Memory TargetDir: targetDir, TargetName: targetName, Permissions: permissions, + SourcePath: MemorySource, }, reader: bytes.NewReader(d), length: len(d), @@ -195,7 +201,7 @@ func MustBinAsset(name, targetDir, targetName, permissions string, isTemplate bo func NewBinAsset(name, targetDir, targetName, permissions string, isTemplate bool) (*BinAsset, error) { m := &BinAsset{ BaseAsset: BaseAsset{ - AssetName: name, + SourcePath: name, TargetDir: targetDir, TargetName: targetName, Permissions: permissions, @@ -218,13 +224,13 @@ func defaultValue(defValue string, val interface{}) string { } func (m *BinAsset) loadData(isTemplate bool) error { - contents, err := Asset(m.AssetName) + contents, err := Asset(m.SourcePath) if err != nil { return err } if isTemplate { - tpl, err := template.New(m.AssetName).Funcs(template.FuncMap{"default": defaultValue}).Parse(string(contents)) + tpl, err := template.New(m.SourcePath).Funcs(template.FuncMap{"default": defaultValue}).Parse(string(contents)) if err != nil { return err } @@ -234,9 +240,9 @@ func (m *BinAsset) loadData(isTemplate bool) error { m.length = len(contents) m.reader = bytes.NewReader(contents) - glog.V(1).Infof("Created asset %s with %d bytes", m.AssetName, m.length) + glog.V(1).Infof("Created asset %s with %d bytes", m.SourcePath, m.length) if m.length == 0 { - return fmt.Errorf("%s is an empty asset", m.AssetName) + return fmt.Errorf("%s is an empty asset", m.SourcePath) } return nil } @@ -249,7 +255,7 @@ func (m *BinAsset) IsTemplate() bool { // Evaluate evaluates the template to a new asset func (m *BinAsset) Evaluate(data interface{}) (*MemoryAsset, error) { if !m.IsTemplate() { - return nil, errors.Errorf("the asset %s is not a template", m.AssetName) + return nil, errors.Errorf("the asset %s is not a template", m.SourcePath) } diff --git a/pkg/minikube/bootstrapper/certs.go b/pkg/minikube/bootstrapper/certs.go index 35833705e0..3099a29b32 100644 --- a/pkg/minikube/bootstrapper/certs.go +++ b/pkg/minikube/bootstrapper/certs.go @@ -117,9 +117,8 @@ func SetupCerts(cmd command.Runner, k8s config.KubernetesConfig, n config.Node) } for _, f := range copyableFiles { - glog.Infof("copying: %s/%s", f.GetTargetDir(), f.GetTargetName()) if err := cmd.Copy(f); err != nil { - return nil, errors.Wrapf(err, "Copy %s", f.GetAssetName()) + return nil, errors.Wrapf(err, "Copy %s", f.GetSourcePath()) } } diff --git a/pkg/minikube/command/command_runner.go b/pkg/minikube/command/command_runner.go index dfd30c5233..06be22eb36 100644 --- a/pkg/minikube/command/command_runner.go +++ b/pkg/minikube/command/command_runner.go @@ -17,12 +17,17 @@ limitations under the License. package command import ( + "bufio" "bytes" "fmt" + "io" + "os" "os/exec" - "path" + "strconv" "strings" + "time" + "github.com/pkg/errors" "k8s.io/minikube/pkg/minikube/assets" ) @@ -55,10 +60,6 @@ type Runner interface { Remove(assets.CopyableFile) error } -func getDeleteFileCommand(f assets.CopyableFile) string { - return fmt.Sprintf("sudo rm %s", path.Join(f.GetTargetDir(), f.GetTargetName())) -} - // Command returns a human readable command string that does not induce eye fatigue func (rr RunResult) Command() string { var sb strings.Builder @@ -84,3 +85,101 @@ func (rr RunResult) Output() string { } return sb.String() } + +// teePrefix copies bytes from a reader to writer, logging each new line. +func teePrefix(prefix string, r io.Reader, w io.Writer, logger func(format string, args ...interface{})) error { + scanner := bufio.NewScanner(r) + scanner.Split(bufio.ScanBytes) + var line bytes.Buffer + + for scanner.Scan() { + b := scanner.Bytes() + if _, err := w.Write(b); err != nil { + return err + } + if bytes.IndexAny(b, "\r\n") == 0 { + if line.Len() > 0 { + logger("%s%s", prefix, line.String()) + line.Reset() + } + continue + } + line.Write(b) + } + // Catch trailing output in case stream does not end with a newline + if line.Len() > 0 { + logger("%s%s", prefix, line.String()) + } + return nil +} + +// fileExists checks that the same file exists on the other end +func fileExists(r Runner, f assets.CopyableFile, dst string) (bool, error) { + // It's too difficult to tell if the file exists with the exact contents + if f.GetSourcePath() == assets.MemorySource { + return false, nil + } + + // get file size and modtime of the source + srcSize := f.GetLength() + srcModTime, err := f.GetModTime() + if err != nil { + return false, err + } + if srcModTime.IsZero() { + return false, nil + } + + // get file size and modtime of the destination + rr, err := r.RunCmd(exec.Command("stat", "-c", "%s %y", dst)) + if err != nil { + if rr.ExitCode == 1 { + return false, nil + } + + // avoid the noise because ssh doesn't propagate the exit code + if strings.HasSuffix(err.Error(), "status 1") { + return false, nil + } + + return false, err + } + + stdout := strings.TrimSpace(rr.Stdout.String()) + outputs := strings.SplitN(stdout, " ", 2) + dstSize, err := strconv.Atoi(outputs[0]) + if err != nil { + return false, err + } + + dstModTime, err := time.Parse(layout, outputs[1]) + if err != nil { + return false, err + } + + if srcSize != dstSize { + return false, errors.New("source file and destination file are different sizes") + } + + return srcModTime.Equal(dstModTime), nil +} + +// writeFile is like ioutil.WriteFile, but does not require reading file into memory +func writeFile(dst string, f assets.CopyableFile, perms os.FileMode) error { + w, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE, perms) + if err != nil { + return errors.Wrap(err, "create") + } + defer w.Close() + + r := f.(io.Reader) + n, err := io.Copy(w, r) + if err != nil { + return errors.Wrap(err, "copy") + } + + if n != int64(f.GetLength()) { + return fmt.Errorf("%s: expected to write %d bytes, but wrote %d instead", dst, f.GetLength(), n) + } + return w.Close() +} diff --git a/pkg/minikube/command/exec_runner.go b/pkg/minikube/command/exec_runner.go index e25346c990..43b49c59f2 100644 --- a/pkg/minikube/command/exec_runner.go +++ b/pkg/minikube/command/exec_runner.go @@ -86,35 +86,31 @@ func (*execRunner) RunCmd(cmd *exec.Cmd) (*RunResult, error) { // Copy copies a file and its permissions func (*execRunner) Copy(f assets.CopyableFile) error { - targetPath := path.Join(f.GetTargetDir(), f.GetTargetName()) - if _, err := os.Stat(targetPath); err == nil { - if err := os.Remove(targetPath); err != nil { - return errors.Wrapf(err, "error removing file %s", targetPath) + dst := path.Join(f.GetTargetDir(), f.GetTargetName()) + if _, err := os.Stat(dst); err == nil { + glog.Infof("found %s, removing ...", dst) + if err := os.Remove(dst); err != nil { + return errors.Wrapf(err, "error removing file %s", dst) } + } + src := f.GetSourcePath() + glog.Infof("cp: %s --> %s (%d bytes)", src, dst, f.GetLength()) + if f.GetLength() == 0 { + glog.Warningf("0 byte asset: %+v", f) } - target, err := os.Create(targetPath) - if err != nil { - return errors.Wrapf(err, "error creating file at %s", targetPath) - } + perms, err := strconv.ParseInt(f.GetPermissions(), 8, 0) if err != nil { return errors.Wrapf(err, "error converting permissions %s to integer", f.GetPermissions()) } - if err := os.Chmod(targetPath, os.FileMode(perms)); err != nil { - return errors.Wrapf(err, "error changing file permissions for %s", targetPath) - } - if _, err = io.Copy(target, f); err != nil { - return errors.Wrapf(err, `error copying file %s to target location: -do you have the correct permissions?`, - targetPath) - } - return target.Close() + return writeFile(dst, f, os.FileMode(perms)) } // Remove removes a file func (*execRunner) Remove(f assets.CopyableFile) error { - targetPath := filepath.Join(f.GetTargetDir(), f.GetTargetName()) - return os.Remove(targetPath) + dst := filepath.Join(f.GetTargetDir(), f.GetTargetName()) + glog.Infof("rm: %s", dst) + return os.Remove(dst) } diff --git a/pkg/minikube/command/fake_runner.go b/pkg/minikube/command/fake_runner.go index 82a6d833df..9da8377a26 100644 --- a/pkg/minikube/command/fake_runner.go +++ b/pkg/minikube/command/fake_runner.go @@ -97,13 +97,13 @@ func (f *FakeCommandRunner) Copy(file assets.CopyableFile) error { if err != nil { return errors.Wrapf(err, "error reading file: %+v", file) } - f.fileMap.Store(file.GetAssetName(), b.String()) + f.fileMap.Store(file.GetSourcePath(), b.String()) return nil } // Remove removes the filename, file contents key value pair from the stored map func (f *FakeCommandRunner) Remove(file assets.CopyableFile) error { - f.fileMap.Delete(file.GetAssetName()) + f.fileMap.Delete(file.GetSourcePath()) return nil } diff --git a/pkg/minikube/command/kic_runner.go b/pkg/minikube/command/kic_runner.go index 754fc64b18..018fe3bfae 100644 --- a/pkg/minikube/command/kic_runner.go +++ b/pkg/minikube/command/kic_runner.go @@ -128,44 +128,73 @@ func (k *kicRunner) RunCmd(cmd *exec.Cmd) (*RunResult, error) { // Copy copies a file and its permissions func (k *kicRunner) Copy(f assets.CopyableFile) error { - src := f.GetAssetName() - if _, err := os.Stat(f.GetAssetName()); os.IsNotExist(err) { - fc := make([]byte, f.GetLength()) // Read asset file into a []byte - if _, err := f.Read(fc); err != nil { - return errors.Wrap(err, "can't copy non-existing file") - } // we have a MemoryAsset, will write to disk before copying + dst := path.Join(path.Join(f.GetTargetDir(), f.GetTargetName())) - tmpFile, err := ioutil.TempFile(os.TempDir(), "tmpf-memory-asset") + // For tiny files, it's cheaper to overwrite than check + if f.GetLength() > 4096 { + exists, err := fileExists(k, f, dst) if err != nil { - return errors.Wrap(err, "creating temporary file") + glog.Infof("existence error for %s: %v", dst, err) } - // clean up the temp file - defer os.Remove(tmpFile.Name()) - if _, err = tmpFile.Write(fc); err != nil { - return errors.Wrap(err, "write to temporary file") + if exists { + glog.Infof("copy: skipping %s (exists)", dst) + return nil } + } - // Close the file - if err := tmpFile.Close(); err != nil { - return errors.Wrap(err, "close temporary file") - } - src = tmpFile.Name() + src := f.GetSourcePath() + if f.GetLength() == 0 { + glog.Warningf("0 byte asset: %+v", f) } perms, err := strconv.ParseInt(f.GetPermissions(), 8, 0) if err != nil { - return errors.Wrapf(err, "converting permissions %s to integer", f.GetPermissions()) + return errors.Wrapf(err, "error converting permissions %s to integer", f.GetPermissions()) } - // Rely on cp -a to propagate permissions - if err := os.Chmod(src, os.FileMode(perms)); err != nil { - return errors.Wrapf(err, "chmod") + if src != assets.MemorySource { + // Take the fast path + fi, err := os.Stat(src) + if err == nil { + if fi.Mode() == os.FileMode(perms) { + glog.Infof("%s (direct): %s --> %s (%d bytes)", k.ociBin, src, dst, f.GetLength()) + return k.copy(src, dst) + } + + // If >1MB, avoid local copy + if fi.Size() > (1024 * 1024) { + glog.Infof("%s (chmod): %s --> %s (%d bytes)", k.ociBin, src, dst, f.GetLength()) + if err := k.copy(src, dst); err != nil { + return err + } + return k.chmod(dst, f.GetPermissions()) + } + } } - dest := fmt.Sprintf("%s:%s", k.nameOrID, path.Join(f.GetTargetDir(), f.GetTargetName())) + glog.Infof("%s (temp): %s --> %s (%d bytes)", k.ociBin, src, dst, f.GetLength()) + tf, err := ioutil.TempFile("", "tmpf-memory-asset") + if err != nil { + return errors.Wrap(err, "creating temporary file") + } + defer os.Remove(tf.Name()) + + if err := writeFile(tf.Name(), f, os.FileMode(perms)); err != nil { + return errors.Wrap(err, "write") + } + return k.copy(tf.Name(), dst) +} + +func (k *kicRunner) copy(src string, dst string) error { + fullDest := fmt.Sprintf("%s:%s", k.nameOrID, dst) if k.ociBin == oci.Podman { - return copyToPodman(src, dest) + return copyToPodman(src, fullDest) } - return copyToDocker(src, dest) + return copyToDocker(src, fullDest) +} + +func (k *kicRunner) chmod(dst string, perm string) error { + _, err := k.RunCmd(exec.Command("sudo", "chmod", perm, dst)) + return err } // Podman cp command doesn't match docker and doesn't have -a @@ -185,11 +214,11 @@ func copyToDocker(src string, dest string) error { // Remove removes a file func (k *kicRunner) Remove(f assets.CopyableFile) error { - fp := path.Join(f.GetTargetDir(), f.GetTargetName()) - if rr, err := k.RunCmd(exec.Command("sudo", "rm", fp)); err != nil { - return errors.Wrapf(err, "removing file %q output: %s", fp, rr.Output()) - } - return nil + dst := path.Join(f.GetTargetDir(), f.GetTargetName()) + glog.Infof("rm: %s", dst) + + _, err := k.RunCmd(exec.Command("sudo", "rm", dst)) + return err } // isTerminal returns true if the writer w is a terminal diff --git a/pkg/minikube/command/ssh_runner.go b/pkg/minikube/command/ssh_runner.go index 9d1f03a04d..ce3cc58522 100644 --- a/pkg/minikube/command/ssh_runner.go +++ b/pkg/minikube/command/ssh_runner.go @@ -17,14 +17,11 @@ limitations under the License. package command import ( - "bufio" "bytes" "fmt" "io" "os/exec" "path" - "strconv" - "strings" "sync" "time" @@ -55,13 +52,16 @@ func NewSSHRunner(c *ssh.Client) *SSHRunner { // Remove runs a command to delete a file on the remote. func (s *SSHRunner) Remove(f assets.CopyableFile) error { + dst := path.Join(f.GetTargetDir(), f.GetTargetName()) + glog.Infof("rm: %s", dst) + sess, err := s.c.NewSession() if err != nil { return errors.Wrap(err, "getting ssh session") } + defer sess.Close() - cmd := getDeleteFileCommand(f) - return sess.Run(cmd) + return sess.Run(fmt.Sprintf("sudo rm %s", dst)) } // teeSSH runs an SSH command, streaming stdout, stderr to logs @@ -150,14 +150,26 @@ func (s *SSHRunner) RunCmd(cmd *exec.Cmd) (*RunResult, error) { // Copy copies a file to the remote over SSH. func (s *SSHRunner) Copy(f assets.CopyableFile) error { dst := path.Join(path.Join(f.GetTargetDir(), f.GetTargetName())) - exists, err := s.sameFileExists(f, dst) - if err != nil { - glog.Infof("Checked if %s exists, but got error: %v", dst, err) + + // For small files, don't bother risking being wrong for no performance benefit + if f.GetLength() > 2048 { + exists, err := fileExists(s, f, dst) + if err != nil { + glog.Infof("existence check for %s: %v", dst, err) + } + + if exists { + glog.Infof("copy: skipping %s (exists)", dst) + return nil + } } - if exists { - glog.Infof("Skipping copying %s as it already exists", dst) - return nil + + src := f.GetSourcePath() + glog.Infof("scp %s --> %s (%d bytes)", src, dst, f.GetLength()) + if f.GetLength() == 0 { + glog.Warningf("0 byte asset: %+v", f) } + sess, err := s.c.NewSession() if err != nil { return errors.Wrap(err, "NewSession") @@ -171,14 +183,13 @@ func (s *SSHRunner) Copy(f assets.CopyableFile) error { // StdinPipe is closed. But let's use errgroup to make it explicit. var g errgroup.Group var copied int64 - glog.Infof("Transferring %d bytes to %s", f.GetLength(), dst) g.Go(func() error { defer w.Close() header := fmt.Sprintf("C%s %d %s\n", f.GetPermissions(), f.GetLength(), f.GetTargetName()) fmt.Fprint(w, header) if f.GetLength() == 0 { - glog.Warningf("%s is a 0 byte asset!", f.GetTargetName()) + glog.Warningf("asked to copy a 0 byte asset: %+v", f) fmt.Fprint(w, "\x00") return nil } @@ -190,7 +201,6 @@ func (s *SSHRunner) Copy(f assets.CopyableFile) error { if copied != int64(f.GetLength()) { return fmt.Errorf("%s: expected to copy %d bytes, but copied %d instead", f.GetTargetName(), f.GetLength(), copied) } - glog.Infof("%s: copied %d bytes", f.GetTargetName(), copied) fmt.Fprint(w, "\x00") return nil }) @@ -208,72 +218,3 @@ func (s *SSHRunner) Copy(f assets.CopyableFile) error { } return g.Wait() } - -func (s *SSHRunner) sameFileExists(f assets.CopyableFile, dst string) (bool, error) { - // get file size and modtime of the source - srcSize := f.GetLength() - srcModTime, err := f.GetModTime() - if err != nil { - return false, err - } - if srcModTime.IsZero() { - return false, nil - } - - // get file size and modtime of the destination - sess, err := s.c.NewSession() - if err != nil { - return false, err - } - - cmd := "stat -c \"%s %y\" " + dst - out, err := sess.CombinedOutput(cmd) - if err != nil { - return false, err - } - outputs := strings.SplitN(strings.Trim(string(out), "\n"), " ", 2) - - dstSize, err := strconv.Atoi(outputs[0]) - if err != nil { - return false, err - } - dstModTime, err := time.Parse(layout, outputs[1]) - if err != nil { - return false, err - } - glog.Infof("found %s: %d bytes, modified at %s", dst, dstSize, dstModTime) - - // compare sizes and modtimes - if srcSize != dstSize { - return false, errors.New("source file and destination file are different sizes") - } - - return srcModTime.Equal(dstModTime), nil -} - -// teePrefix copies bytes from a reader to writer, logging each new line. -func teePrefix(prefix string, r io.Reader, w io.Writer, logger func(format string, args ...interface{})) error { - scanner := bufio.NewScanner(r) - scanner.Split(bufio.ScanBytes) - var line bytes.Buffer - - for scanner.Scan() { - b := scanner.Bytes() - if _, err := w.Write(b); err != nil { - return err - } - if bytes.IndexAny(b, "\r\n") == 0 { - if line.Len() > 0 { - logger("%s%s", prefix, line.String()) - line.Reset() - } - continue - } - line.Write(b) - } - // Catch trailing output in case stream does not end with a newline - if line.Len() > 0 { - logger("%s%s", prefix, line.String()) - } - return nil -} diff --git a/pkg/minikube/machine/filesync_test.go b/pkg/minikube/machine/filesync_test.go index 99a674e8e8..143c3e9ab7 100644 --- a/pkg/minikube/machine/filesync_test.go +++ b/pkg/minikube/machine/filesync_test.go @@ -149,7 +149,7 @@ func TestAssetsFromDir(t *testing.T) { got := make(map[string]string) for _, actualFile := range actualFiles { - got[actualFile.GetAssetName()] = actualFile.GetTargetDir() + got[actualFile.GetSourcePath()] = actualFile.GetTargetDir() } if diff := cmp.Diff(want, got); diff != "" { t.Errorf("files differ: (-want +got)\n%s", diff) From 897af043f9f90e22e08b300e575051b3b2a5c55a Mon Sep 17 00:00:00 2001 From: Medya Gh Date: Thu, 9 Apr 2020 17:08:02 -0700 Subject: [PATCH 5/6] fix broken screenshot link on the README --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index eeb78f1c0b..3f002b3713 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,8 @@ minikube implements a local Kubernetes cluster on macOS, Linux, and Windows. minikube's [primary goals](https://minikube.sigs.k8s.io/docs/concepts/principles/) are to be the best tool for local Kubernetes application development and to support all Kubernetes features that fit. -screenshot +screenshot + ## Features From 0f325f069d4d76692f676011186e88b5e8fb39c6 Mon Sep 17 00:00:00 2001 From: remraz <38458023+remraz@users.noreply.github.com> Date: Fri, 10 Apr 2020 02:24:59 +0200 Subject: [PATCH 6/6] Use Github's GUI for quick contributions Refers to Github's repositories and integrated markdown editor that already exists in Kubernetes's general guidelines. --- site/content/en/docs/contrib/documentation.en.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/site/content/en/docs/contrib/documentation.en.md b/site/content/en/docs/contrib/documentation.en.md index ef0be91b83..f2bab66420 100644 --- a/site/content/en/docs/contrib/documentation.en.md +++ b/site/content/en/docs/contrib/documentation.en.md @@ -12,6 +12,10 @@ minikube's documentation is in [Markdown](https://www.markdownguide.org/cheat-sh In production, the minikube website is served using [Netlify](https://netlify.com/) +## Small or cosmetic contributions + +Use Github's repositories and markdown editor as described by [Kubernetes's general guideline for documentation contributing](https://kubernetes.io/docs/contribute/start/#submit-a-pull-request) + ## Local documentation website To serve documentation pages locally, clone the `minikube` repository and run: