Loading...
Loading...
While attempting to upgrade a Tanzu Kubernetes Grid (TKG) management cluster version from 1.3.1 to 1.4.0 you see similar to the following in the tanzu CLI output after the first upgrade attempt: updating additional components: 'addons-management/tanzu-addons-manager' ... updating additional components: 'tkr/tkr-controller' ... updating additional components: 'addons-management/core-package-repo' ... Error: failed to deploy additional components after kubernetes upgrade: error while upgrading additional component 'addons-management/standard-package-repo': kubectl apply failed, output: secret/mgmt-cluster-1-9-5-standard-package-repository-crs configured Error from server (InternalError): error when creating "/tmp/kubeapply-211654953": Internal error occurred: failed calling webhook "default.clusterresourceset.addons.cluster.x-k8s.io": Post "https://capi-webhook-service.capi-webhook-system.svc:443/mutate-addons-cluster-x-k8s-io-v1alpha3-clusterresourceset?timeout=30s": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) : exit status 1 You see messages similar to the following in the cert-manager-webhook pod logs: W1001 12:15:35.653078 1 client_config.go:608] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. I1001 12:15:35.653294 1 webhook.go:57] cert-manager/webhook "msg"="using dynamic certificate generating using CA stored in Secret resource" "secret_name"="cert-manager-webhook-ca" "secret_namespace"="cert-manager" I1001 12:15:35.653455 1 server.go:146] cert-manager/webhook "msg"="listening for insecure healthz connections" "address"=":6080" I1001 12:15:35.653488 1 server.go:159] cert-manager/webhook "msg"="listening for secure connections" "address"=":10250" I1001 12:15:35.653509 1 server.go:185] cert-manager/webhook "msg"="registered pprof handlers" I1001 12:15:35.654464 1 reflector.go:207] Starting reflector *v1.Secret (1m0s) from pkg/mod/k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156 E1001 12:15:36.660296 1 dynamic_source.go:88] cert-manager/webhook "msg"="Failed to generate initial serving certificate, retrying..." "error"="failed verifying CA keypair: tls: failed to find any PEM data in certificate input" "interval"=1000000000 E1001 12:15:37.659501 1 dynamic_source.go:88] cert-manager/webhook "msg"="Failed to generate initial serving certificate, retrying..." "error"="failed verifying CA keypair: tls: failed to find any PEM data in certificate input" "interval"=1000000000 E1001 12:15:38.660871 1 dynamic_source.go:88] cert-manager/webhook "msg"="Failed to generate initial serving certificate, retrying..." "error"="failed verifying CA keypair: tls: failed to find any PEM data in certificate input" "interval"=1000000000 I1001 12:15:39.682232 1 dynamic_source.go:199] cert-manager/webhook "msg"="Updated serving TLS certificate" I1001 12:15:45.080062 1 logs.go:58] http: TLS handshake error from 100.96.3.1:54726: remote error: tls: bad certificate Multiple attempts to upgrade the management cluster result in the following output from tanzu CLI: Upgrading management cluster providers... Error: failed to upgrade management cluster providers: failed to get providers upgrade information: failed to parse the management group: failed to find core provider from the current providers
This is a known issue affecting TKG. There is currently no resolution.
On a machine with access to the tanzu CLI, run the tanzu login command to see the available management clusters and which one is the current login context for the CLI.To see the details of the management cluster, run tanzu management-cluster getIssue a command similar to the following to retrieve a kubeconfig for the management cluster tanzu management-cluster kubeconfig get --admin --export-file MC-ADMIN-KUBECONFIG Issue a command similar to the following to set the context of kubectl to the management cluster: kubectl config use-context my-mgmnt-cluster-admin@my-mgmnt-cluster Create a file called provider-definition.yaml with the following content: apiVersion: clusterctl.cluster.x-k8s.io/v1alpha3 kind: Provider metadata: labels: cluster.x-k8s.io/provider: cluster-api clusterctl.cluster.x-k8s.io: "" clusterctl.cluster.x-k8s.io/core: inventory name: cluster-api namespace: capi-system providerName: cluster-api type: CoreProvider version: v0.3.14 Run a command similar to the following: kubectl apply -f <PATH TO provider-definition.yaml>
https://github.com/kubernetes-sigs/cluster-api/issues/5015 https://github.com/kubernetes-sigs/cluster-api/pull/5044
Click on a version to see all relevant bugs
VMware Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.