From f42f79e56d5fc9e26d4f5a7b857166c3fb78ce6a Mon Sep 17 00:00:00 2001 From: Shubham Date: Tue, 21 Jul 2020 20:58:54 +0530 Subject: [PATCH] (Addons) Create a new command in every retry Whenever a `kubectl apply` fails while enabling an addon, it is retried with exponential backoff. The command (type `*exec.Cmd`) that this retry function runs in created outside the function - which means that it is reused on every retry. This is a problem because `exec.Cmd` (https://godoc.org/github.com/pkg/exec#Cmd) states that "... Cmd cannot be reused after calling its Run or Start methods." This retry is a common case due to, say, a CRD and its resource being present in the same YAML file of the addon which causes a race condition where the resource is created before its CRD is created in the cluster - this race is fixed by subsequent retries. I've noticed this in the dashboard and the ambassador addon. Due to the above mentioned bug, minikube throws errors like `exec: already started` in every retry and the retry is never successful, manifests are never deployed and addon creation errors out. Related to #8138 #8119 #8372 --- pkg/addons/addons.go | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/pkg/addons/addons.go b/pkg/addons/addons.go index c197913d12..3275946684 100644 --- a/pkg/addons/addons.go +++ b/pkg/addons/addons.go @@ -250,11 +250,9 @@ func enableOrDisableAddonInternal(cc *config.ClusterConfig, addon *assets.Addon, } } - command := kubectlCommand(cc, deployFiles, enable) - // Retry, because sometimes we race against an apiserver restart apply := func() error { - _, err := cmd.RunCmd(command) + _, err := cmd.RunCmd(kubectlCommand(cc, deployFiles, enable)) if err != nil { glog.Warningf("apply failed, will retry: %v", err) }