Upgrade Workload Clusters

Upgrade Path

This page covers the traditional operating system path for workload clusters. For workload clusters that run on Immutable Infrastructure (MicroOS on Huawei DCS, VMware vSphere, or Huawei Cloud Stack), the Kubernetes step lives in the immutable infrastructure documentation — see Upgrading Clusters on Immutable Infrastructure. The Core, Aligned, and Agnostic steps described on this page still apply to immutable-OS clusters; only the way Kubernetes is rolled out differs.

Workload clusters can be upgraded after the global tier has already reached the target ACP Distribution Version. The global tier does not need to be upgraded again if it is already at that Distribution Version.

ACP 4.3 uses the same CVO-based workflow for workload clusters as for the global cluster: confirm prerequisites, run preflight checks, request the upgrade, and observe execution. This workflow has two additional rules:

If the platform uses global DR, both the standby and primary global clusters must reach the target Distribution Version before any workload cluster is upgraded to that Distribution Version.
The target version normally becomes available for a workload cluster only after the global tier has already reached that same Distribution Version.

Workflow

Confirm workload-cluster prerequisites

Before requesting the upgrade, verify that:

The workload cluster version is lower than the current global cluster version.
The global tier has already reached the target ACP Distribution Version.
In global DR environments, both the standby and primary global clusters have already reached that target Distribution Version.

Operator push fallback

The recommended pattern in Upgrade the global cluster pushes operator packages to every cluster during the global upgrade window. If the recommendation was followed, the workload cluster already has every operator package it needs and you can skip this section.

Cluster plugins do not need a per-workload-cluster push. A cluster plugin pushed once to the global tier is installable and upgradable on every cluster in the platform.

If the global window did not push operator packages to this workload cluster — for example, the cluster was offline during the global window or was added later — push each in-use operator package to this workload cluster before the upgrade request:

violet push <path/to/operator-package> \
  --target-catalog-source "platform" \
  --platform-address "https://<your-platform-domain>" \
  --platform-token "<platform_token>" \
  --clusters "<workload-cluster-name>"

You only need to push operator packages for operators actually running on the workload cluster. Aligned plugin packages still on the global side are picked up automatically by CVO during the workload upgrade.

Run preflight checks

Run preflight before requesting the upgrade:

bash upgrade.sh --cluster=<cluster> --preflight

Preflight returns two parts:

Output	Purpose
`Summary`	Shows the overall result, current version, desired version, and desired image.
`Checks`	Shows the result of each individual validation item.

If preflight does not pass, do not submit the upgrade request until all blocking items are resolved.

For preflight block handling patterns, see Handle preflight blocks when needed.

Request the upgrade

After preflight passes, request the upgrade through one of the following entry points. The two entry points are equivalent; pick the one that fits your operating model.

Web Console

ACP CLI

Start the upgrade from the workload cluster in the Web Console. The request follows a two-step flow:

In Step 1, review the RPCH list.
Click Acknowledge to continue to Step 2.
In Step 2, review Current Version and Target Version. The page does not display a plugin list or a warning panel at this stage.
The target version is the current global cluster version and cannot be selected manually in the Web Console.
Click Start Upgrade.
After the request is submitted, the page shows that the upgrade request has been submitted and the action enters an in-progress state.

Use ACP CLI when your current ACP session can access the target workload cluster. To avoid ambiguity, specify the target cluster explicitly with --cluster. ACP CLI is not used to upgrade the global cluster.

# Request upgrade to the highest version currently published in availableUpdates
ac adm upgrade --cluster=<cluster> --to-latest

# Request upgrade to a specific target version
ac adm upgrade --cluster=<cluster> --to=4.3.0

ACP CLI submits the requested target for the specified workload cluster. Whether the upgrade can proceed is still determined by the cluster's available upgrade targets and the upgrade controller's preflight checks.

If you are new to ACP CLI, see Getting Started with ACP CLI. For session and cluster selection, see Managing CLI Profiles. For the complete AC CLI upgrade workflow (metadata preparation, available updates, request modes, and status interpretation), see Upgrading Clusters. For full command and flag syntax, see AC CLI Administrator Command Reference.

Observe progress

After the upgrade request is submitted, use cvsh.status to track current version, desired version, preflight results, stages, and history:

kubectl get cvsh <cluster> -n cpaas-system
kubectl get cvsh <cluster> -n cpaas-system -o yaml

If your current ACP context points to the target workload cluster, you can also check the CLI-reported status:

# Show summary, preflight, and stage progress for the target cluster upgrade
ac adm upgrade status

For detailed semantics of ac adm upgrade status output (preflight and stage interpretation), see Upgrading Clusters.

Upgrade Agnostic plugins from Marketplace

CVO drives Core and Aligned plugins. Agnostic plugins are outside CVO's scope and must be upgraded individually after this workload cluster has reached the target Distribution Version.

For each in-use Agnostic plugin on this workload cluster:

In the Web Console, switch to Administrator view.
Navigate to Marketplace > Cluster Plugins for Agnostic cluster plugins, or to the operator workflow for Agnostic operators.
Select the target plugin or operator and trigger the upgrade.

Whether each Agnostic plugin needs to be upgraded in this window depends on its own Kubernetes compatibility — see the plugin's release notes for compatibility with the new Kubernetes version that the workload cluster has reached.

If the Marketplace does not offer the target version of an operator on this workload cluster, complete the Operator push fallback for that operator first and then retry the Marketplace upgrade.

When troubleshooting, inspect conditions, preflight details, and history first:

kubectl -n cpaas-system get cvsh <cluster> \
  -o jsonpath='{range .status.conditions[*]}{.type}{"\t"}{.status}{"\t"}{.reason}{"\t"}{.message}{"\n"}{end}'

kubectl -n cpaas-system get cvsh <cluster> \
  -o jsonpath='{.status.preflight.observedAt}{"\n"}{range .status.preflight.checks[*]}{.name}{"\t"}{.policy}{"\t"}{.state}{"\t"}{.reason}{"\t"}{.message}{"\n"}{end}'

kubectl -n cpaas-system get cvsh <cluster> \
  -o jsonpath='{range .status.history[*]}{.version}{"\t"}{.state}{"\t"}{.startedTime}{"\t"}{.completionTime}{"\n"}{end}'

#Upgrade Workload Clusters

#TOC

#Workflow

#Confirm workload-cluster prerequisites

#Operator push fallback

#Run preflight checks

#Request the upgrade

#Observe progress

#Upgrade Agnostic plugins from Marketplace

#Related Documentation