cluster-update-advisor

Cluster Update Advisor

Assess whether an OpenShift cluster is ready to update, and — when the update is feasible — produce a concrete, ordered remediation plan.

Input

The proposal request already contains everything needed to start: current and target version metadata, the update path classification (Recommended or Conditional), and pre-collected cluster readiness JSON gathered by the Cluster Version Operator, embedded between ```json markers under the "Cluster Readiness Data" heading. Do not re-collect cluster data. Parse that JSON to begin.

{
  "current_version": "4.21.5",
  "target_version": "4.21.8",
  "checks": {
    "cluster_conditions":     { "_status": "ok", "summary": {...}, ... },
    "operator_health":        { "_status": "ok", "summary": {...}, ... },
    "api_deprecations":       { "_status": "ok", "summary": {...}, ... },
    "node_capacity":          { "_status": "ok", "summary": {...}, ... },
    "pdb_drain":              { "_status": "ok", "summary": {...}, ... },
    "etcd_health":            { "_status": "ok", "summary": {...}, ... },
    "network":                { "_status": "ok", "summary": {...}, ... },
    "crd_compat":             { "_status": "ok", "summary": {...}, ... },
    "olm_operator_lifecycle": { "_status": "ok", "summary": {...}, "operators": [...] }
  }
}

Every check has _status (ok or error) and a summary for quick parsing. A check with _status: error is a gap in visibility — note it; it lowers confidence.

Output format

Output must conform to the RemediationOption schema enforced by the OlsAgent CR's outputSchema field; the operator handles structured-output compliance. Plan actions are typed pre-check, mutation, wait, or post-check, and appear in that order, with long running verification commands beyond starting cluster update to be included in the verification field of the RemediationOption. When the update cannot proceed, structure the RemediationOption to include the causes in within the RemediationOption. If any single cause is blocking the cluster update, also assess if a cluster update ignoring that factor is possible as a viable path, generating a RemediationOption. In such cases also include a plan blocked on user update instructing user on steps to take or what conditions need to be met or waited for in order to perform a safe cluster update. If a plan makes destructive changes to a cluster, include steps to restore any removed components or backed up data if appropriate.

Assess update readiness

Apply any organization-specific policy from the system prompt (thresholds, scheduling preferences, risk tolerance); otherwise use sensible defaults. Walk each check's summary and detail data:

• Compare numeric thresholds (node headroom, etcd backup age)
• Evaluate conditional update risks against actual cluster state — never dismiss them
• Identify compounding risks (e.g. paused MCP + cert expiry)
• Estimate update duration (~10 min/node)

Classify each finding

For checks not listed: blocker if the issue would cause data loss, a performance regression, or a failed update; warning if it would cause temporary disruption or a slow update.

Determine if any operators need upgrading

.checks.olm_operator_lifecycle.operators[] tells you what is installed and how it is managed — name, package, namespace, channel, available_channels, installed_version, installed_csv, install_plan_approval, state. Use it in the product-lifecycle skill as needed and to name resources in commands. It does not decide what needs upgrading: summary.incompatible_with_target is a hint that work is likely needed, not a per-operator verdict, and available_channels lists what exists for OLMv0 operators rather than what is compatible. For operators either missing lifecycle data or marked incompatible use the product-lifecycle skill should be the source of truth.

Split by OLM API type. An entry with installed_csv or install_plan_approval is OLMv0 (Subscription-managed); everything else is OLMv1 (ClusterExtension). Build one JSON array per type from package and installed_version.

When using the product-lifecycle skill call it once for the whole cluster, with the target OCP as major.minor (4.21, not 4.21.8) and always --compatibleonly

Lifecycle data in the output is purely informational, do not use it to disqualify an otherwise valid update if no other update is available. Operators may be installed past their End of life date if compatible with target ocp version and an alternative is unavailable. Management of Operators without data in the PLC api is orthogonal to the cluster update, missing PLC data should be reported as a warning.

Decide per operator from its entry in results — never from a version number, a channel name, or the mere existence of a newer release:

ocp_compatible: false with a reachable upgrade is the only trigger for an operator upgrade; those operators are the only ones that get commands in the plan. If an installed incompatible operator can be addressed with an update, do not consider the operator a blocker. Currently installed operator versions being incompatible with current or target OCP is not a blocker if and only if the operator can be updated to a version compatible with the target OCP version - cluster updates not meeting this precondition are invalid.

Decide

Do not include findings that can be fixed without user input such as incompatible operators or configuration that can be fixed with an update or minor configuration change as blockers. Report the decision with the evidence behind it. If the decision is escalate, or block with no viable remediation, stop here — explain why in the diagnosis and produce no plan. Only do this if no alternate plan is possible. Do not generate remediation plans to other target OCP versions than the one specified.

Build the remediation plan

Proceed only when the update is feasible: the decision is recommend, warn, or block with actionable remediation. Produce a single RemediationOption.

Commands

Every command must be concrete and executable: resolve every value from the olm-check output and the readiness data before emitting it, and never leave a placeholder variable in the plan. Patch only the fields that actually change.

Choosing the target version: the lowest version that is compatible with the target OCP, strictly higher than the installed version, and not End of life in its product lifecycle status.

Pre-checks

Validate before any mutation — never skip this phase:

• Cluster is not already updating (Progressing=False)
• No critical operator is degraded
• Target version exists in available or conditional updates

These are the only pre-checks to include in the plan. Other readiness dimensions (MCP health, node capacity, PDB configuration, etcd) were already assessed from the pre-collected cluster data and must not be re-verified as execution-time pre-checks.

OLMv0 operator update (Subscription / InstallPlan)

RBAC required:

• patch on subscriptions.operators.coreos.com and installplans.operators.coreos.com
• delete on installplans.operators.coreos.com

Each source_catalogs entry gives a catalog, a catalog_namespace, and a versions map of full SemVer → {channels, bundle}. From the chosen version's entry:

1. Take the channel only from that version's own channels list. Never infer a channel from a version number, and never reuse available_channels from the readiness data — it is not filtered for compatibility.
2. Take the catalog and the channel from the same source_catalogs entry.
3. If several channels carry the chosen version, prefer stable over fast, candidate or preview; among version-scoped channels (stable-6.5) prefer the one pinned to the chosen version's minor over a floating one, so the update lands on the intended version.
4. Do not set spec.startingCSV — it applies at install time only and is ignored on update. Use the bundle value for verification and in the action description instead.
5. Never synthesize a CSV name or bundle name from the package, version, or channel name. The only valid CSV name is the bundle value.

If the subscription is already on the right catalog, patch the channel alone (and skip the patch entirely if the channel already matches):

oc patch subscriptions.operators.coreos.com ${SUBSCRIPTION_NAME} -n ${NAMESPACE} --type=merge -p '{"spec":{"channel":"${TARGET_CHANNEL}"}}'

If the chosen version is only available from a different catalog, patch the source and channel in one operation so the subscription is never left inconsistent:

oc patch subscriptions.operators.coreos.com ${SUBSCRIPTION_NAME} -n ${NAMESPACE} --type=merge -p '{"spec":{"channel":"${TARGET_CHANNEL}","source":"${CATALOG}","sourceNamespace":"${CATALOG_NAMESPACE}"}}'

Then wait for the InstallPlan, approve it when install_plan_approval is Manual for any InstallPlan in the namespace (omit the approve step if all InstallPlans have Automatic approval, keep the waits), and verify. This may require deleting an existing InstallPlan if the subscription already has one in RequiresApproval status. ${INSTALLPLAN_NAME} comes from the subscription's .status.installplan.name after the channel patch:

# Wait for the subscription to generate an InstallPlan
oc wait subscriptions.operators.coreos.com ${SUBSCRIPTION_NAME} -n ${NAMESPACE} --for=jsonpath='{.status.installplan.name}' --timeout=60s

# Required only if an InstallPlan in the namespace has manual approval
# Ensure the InstallPlan is the correct one before approving
oc patch installplans.operators.coreos.com ${INSTALLPLAN_NAME} -n ${NAMESPACE} --type=merge -p '{"spec":{"approved":true}}'

# Wait for the operator update to complete, then confirm the expected bundle is installed
oc wait installplans.operators.coreos.com ${INSTALLPLAN_NAME} -n ${NAMESPACE} --for='jsonpath={.status.phase}=Complete' --timeout=300s
oc get subscriptions.operators.coreos.com ${SUBSCRIPTION_NAME} -n ${NAMESPACE} -o jsonpath='{.status.installedCSV}'

** Wait on the InstallPlan phase only.**: A compatible verision on a channel only means it may be reached. Depending on the OLMv1 catalog source itself, a specific operator version may be skipped in favor of a later version in the channel, or only be reachable through a multiple operator updates. If installPlan approcal is Automatic, this update proceeds till at the head of the channel. If Manual, each step of the update will need an installPlan approval. Do not have strict checks or waits on specific values of .status.installedCSV or .status.currentCSV, these are unreliable.

OLMv1 operator update (ClusterExtension)

RBAC required:

• patch on clusterextensions.olm.operatorframework.io

Set a version range derived from available_versions, excluding every incompatible version; OLMv1 resolves to the newest version in range. If the only version olm-check returns is the installed one, do not patch the ClusterExtension.

# Patch the clusterextension to include the expected version in its version range
oc patch clusterextensions.olm.operatorframework.io ${EXTENSION_NAME} --type=merge -p '{"spec":{"source":{"catalog":{"version":"${VERSION_RANGE}"}}}}'

# Wait for the update, then confirm the resolved version is inside the compatible range
oc wait clusterextensions.olm.operatorframework.io ${EXTENSION_NAME} --for=condition=Installed=True --timeout=300s
oc get clusterextensions.olm.operatorframework.io ${EXTENSION_NAME} -o jsonpath='{.status.install.bundle.version}'

Platform update

${TARGET_VERSION} is the chosen target from the cluster's available updates. All plans that include this are Irreversible

# Requires: patch on the version instance of clusterversions.config.openshift.io
oc adm upgrade --to=${TARGET_VERSION}

Post-checks

Verify the update has started:

oc wait --for=jsonpath='{.status.desired.version}=${TARGET_VERSION}' clusterversions.config.openshift.io version --timeout=60s

Longer running checks beyond checking for cluster update to start should be done in the verification section in the RemediationOption to avoid execution timeouts.

Optionally inform whether cluster update is progressing or complete.

oc get clusterversion/version -o='jsonpath={.status.conditions[?(@.type=="Progressing")].status}'

RBAC

For each oc command in the plan, derive the Kubernetes PolicyRule it requires from the fully qualified resource name (resource, apiGroup, verbs, and resourceNames where applicable), and include the complete list in the remediation option. Include permissions required for reading and modifying resources as needed. Ensure patch permissions for clusterversions.config.openshift.io for actual cluster update.

Hard rules

• Never recommend skipping an update version unless the readiness data shows that path exists, and never recommend force-updating. If the standard path is blocked, report it as blocked.
• Never treat missing lifecycle data as compatible. ocp_compatible: null, an error entry, or a packagemanifest_error means unknown — report it; do not guess, and do not silently skip the operator. A remediation plan may be offered without the operator update but the status of missing lifecycle data must be reported.
• Never dry-run with --dry-run=client; use --dry-run=server so validation hits the real cluster API.
• Ensure emitted commands are syntactically correct.
• If no valid remediation plan can be produced, say so and give the root cause.