Upgrading Anthos Service Mesh

This page is for platform administrators.

This page describes how to upgrade Anthos Service Mesh.

Platform admins can upgrade Anthos Service Mesh. Upgrading Anthos Service Mesh is a revision-based or canary upgrade where both the new and previous versions of the control plane are running as you test the new version with a small percentage of your workloads. This approach is safer than an in-place upgrade, where the new version of the control plane replaces the previous version.

Redeploying the Anthos Service Mesh control plane components takes about 5 to 10 minutes to complete. You must also inject new sidecar proxies into all of your workloads so that they are updated with the current Anthos Service Mesh version. The time it takes to update the sidecar proxies might be about 100 Pods per minute, but this depends on many factors, such as the number of Pods, the number of nodes, deployment scaling settings, Pod disruption budgets, and other configuration settings.

Prerequisites

Ensure that you meet the following conditions before upgrading:

  • You have a target user cluster managed by the admin cluster.
  • Your target user cluster has an existing Anthos Service Mesh installation that has upgrades available.

View your existing installation

Confirm that Anthos Service Mesh is installed on your cluster.

Console

In the Management Center console, go to the Service Mesh menu to see the list of clusters with Anthos Service Mesh installed.

API

You can find the following custom resources on an admin cluster describing your current Anthos Service Mesh installation:

  • A ServiceMeshBinding object which binds a ServiceMeshFeatureSpec object with the ID of your user cluster. Run the following command to find the object. As this command might return multiple objects, you must find the object with the target cluster's ID in its spec.

    KUBECONFIG=ADMIN_KUBECONFIG kubectl get servicemeshbinding -n anthos-management-center
    

    Here is an example of the content of the object:

    apiVersion: managementcenter.anthos.cloud.google.com/v1alpha1
    kind: ServiceMeshBinding
    metadata:
     name: service-mesh-binding-sample
     namespace: anthos-management-center
    spec:
     configs:
     - configRef:
         name: PREVIOUS_SERVICE_MESH_SPEC
         namespace: anthos-management-center
       placement:
         clusterIDs:
         - TARGET_CLUSTER_ID
    
  • A ServiceMeshFeatureSpec object which defines the spec of your Anthos Service Mesh installation. Its namespace and name are referenced in the config.configRef field in the ServiceMeshBinding object described in the previous example. Run the following command to view the object.

    KUBECONFIG=ADMIN_KUBECONFIG kubectl get servicemeshfeaturespec PREVIOUS_SERVICE_MESH_SPEC -n anthos-management-center -o yaml
    

    Here is an example of the content of the object:

    apiVersion: managementcenter.anthos.cloud.google.com/v1alpha1
    kind: ServiceMeshFeatureSpec
    metadata:
     name:PREVIOUS_SERVICE_MESH_SPEC
     namespace: anthos-management-center
    spec:
     version: 1.7.3-asm.6
    

Install a newer version of Anthos Service Mesh

You can update to a newer version of Anthos Service Mesh by using the management center console or the API.

Console

  1. In the Management Center console, go to the Dashboard menu.
  2. Click the Upgrades tab to see all available upgrades.
  3. Click View Upgrades for the cluster that you want to upgrade.
  4. In the Available upgrades list, find the target Anthos Service Mesh version that you want to upgrade to and click Upgrade to open the Anthos Service Mesh upgrade page. Available upgrades page
  5. Confirm the target version's information, such as the version, description, and a link to the release notes and click Continue.
  6. In the pop-up dialog, click Upgrade to start installing the target version of Anthos Service Mesh on the target cluster.
  7. The Service Mesh page opens where you can see that the target version is being installed.

API

Perform the following steps using kubectl commands on the admin cluster to install a newer version of Anthos Service Mesh:

  1. Create a new ServiceMeshFeatureSpec object for the new version. Here is an sample object for version 1.8.3-asm.2:

    apiVersion: managementcenter.anthos.cloud.google.com/v1alpha1
    kind: ServiceMeshFeatureSpec
    metadata:
     name: NEW_SERVICE_MESH_SPEC
     namespace: anthos-management-center
    spec:
     version: 1.8.3-asm.2
    
  2. Update the existing ServiceMeshBinding object to insert a new entry to bind the new ServiceMeshFeatureSpec with the cluster:

     apiVersion: managementcenter.anthos.cloud.google.com/v1alpha1
     kind: ServiceMeshBinding
     metadata:
      name: service-mesh-binding-sample
      namespace: anthos-management-center
     spec:
      configs:
      - configRef:
          name: PREVIOUS_SERVICE_MESH_SPEC
          namespace: anthos-management-center
        placement:
          clusterIDs:
          - TARGET_CLUSTER_ID
      - configRef:
          name: NEW_SERVICE_MESH_SPEC
          namespace: anthos-management-center
        placement:
          clusterIDs:
          - TARGET_CLUSTER_ID
    
  3. The newer version of Anthos Service Mesh installs on the admin cluster. You can check whether the installation is ready using the following command:

     KUBECONFIG=ADMIN_KUBECONFIG kubectl get servicemeshbinding -n anthos-management-center -o jsonpath="{range .items[*].status.bindingItemStatuses[*]}{'\n'}{['clusterID', 'configRef', 'conditions']}" | grep NEW_SERVICE_MESH_SPEC | grep TARGET_CLUSTER_ID
    

    Replace the TARGET_CLUSTER_ID and NEW_SERVICE_MESH_SPEC with the values configured in the ServiceMeshBinding object.

    Here's an example of the output:

    target-user-cluster-1 {"name":"new-spec-q64x7","namespace":"anthos-management-center"} [{"lastTransitionTime":"2021-05-27T18:06:27Z","message":"","observedGeneration":1,"reason":"Healthy","status":"True","type":"Ready"}]
    

    The status: true of the Ready condition indicates that the installation is complete.

The previous version of Anthos Service Mesh is not removed automatically. You must manually migrate your workloads to the new version of Anthos Service Mesh and uninstall the previous version.

Migrate your workloads

Perform the following steps on your user cluster to migrate your workloads:

  1. Get the revision label of your previous version and new version of Anthos Service Mesh. To find these values:

    Console

    1. In the Management Center console, go to the Service Mesh menu.
    2. In the cluster list, find the revision label in the Revision column for the cluster.

    API

    Run the following command:

     KUBECONFIG=USER_KUBECONFIG kubectl get pod -n istio-system -l app=istiod -L istio.io/rev
    

    Here's an example of the output:

    NAME                                READY   STATUS    RESTARTS   AGE   REV
    istiod-asm-173-6-756d989c58-p84b9   1/1     Running   0          13h   asm-173-6
    istiod-asm-173-6-756d989c58-t6vh8   1/1     Running   0          13h   asm-173-6
    istiod-asm-183-2-76f64c6bc7-2qnxq   1/1     Running   0          17h   asm-183-2
    istiod-asm-183-2-76f64c6bc7-8nrpz   1/1     Running   0          17h   asm-183-2
    

    In the output, the revision label is under the REV column. In this example, the value is asm-183-2. This label is used later to trigger sidecar injection for the new Anthos Service Mesh version.

  2. Add the revision label to a namespace. In the following command, change REVISION to the value that matches the revision label of the new Anthos Service Mesh version.

    KUBECONFIG=USER_KUBECONFIG kubectl label namespace NAMESPACE istio.io/rev=REVISION istio-injection- --overwrite
    
  3. Restart the workloads to trigger re-injection, for example to restart all deployments in a namespace:

    KUBECONFIG=USER_KUBECONFIG kubectl rollout restart deployment -n NAMESPACE
    
  4. Verify that your Pods are configured to point to the new version of istiod.

    KUBECONFIG=USER_KUBECONFIG kubectl get pods -n NAMESPACE -l istio.io/rev=REVISION
    
  5. Test your application to verify that the workloads are working correctly.

  6. If you have workloads in other namespaces, repeat steps 2 to 5 to label the namespace and restart the workloads.

Delete the previous version of Anthos Service Mesh

If your application is working as expected, you can uninstall the previous version of Anthos Service Mesh by using the Management Center or the API.

Console

  1. In the Management Center console, go to the Service Mesh menu.
  2. In the Versions column, click the link of the previous Anthos Service Mesh version.
  3. On the Edit Service Mesh page, click Remove Service Mesh.
  4. A pop-up dialog shows the result of a confidence check before deletion. The dialog lists all of the namespaces and Pods that still depend on the previous version of Anthos Service Mesh by checking theistio.io/rev label on namespaces and Pods. If the check finds any of these dependencies, you cannot proceed until you migrate these Pods or namespaces.
  5. If the confidence check passes, click Remove to start uninstalling Anthos Service Mesh on the target cluster.

After you submit the deletion request, the Service Mesh page opens and the previous version of Anthos Service Mesh is no longer visible. The uninstallation of the previous Anthos Service Mesh resources might take several minutes to complete.

API

  1. Before you delete the previous version of Anthos Service Mesh, verify that there are no Pods or namespaces that depend on that version of Anthos Service Mesh.

    1. Run the following command to verify any namespace dependencies.

      KUBECONFIG=USER_KUBECONFIG kubectl get namespace -l istio.io/rev=PREVIOUS_REVISION  
      

      Replace PREVIOUS_REVISION with the revision label of the previous Anthos Service Mesh version.

    2. Run the following command to verify any Pod dependencies.

      KUBECONFIG=USER_KUBECONFIG kubectl get pod -l istio.io/rev=PREVIOUS_REVISION --field-selector metadata.namespace!=istio-system --all-namespaces
      

    If the queries return empty results, proceed to the next step. Otherwise, migrate the workload first before proceeding.

  2. Update the ServiceMeshBinding object on the admin cluster to unbind the previous feature spec from the target cluster. Here's an example of the ServiceMeshBinding object after removing the PREVIOUS_SERVICE_MESH_SPEC entry:

    apiVersion: managementcenter.anthos.cloud.google.com/v1alpha1
    kind: ServiceMeshBinding
    metadata:
     name: service-mesh-binding-sample
     namespace: anthos-management-center
    spec:
     configs:
     - configRef:
         name: NEW_SERVICE_MESH_SPEC
         namespace: anthos-management-center
       placement:
         clusterIDs:
         -  TARGET_CLUSTER_ID
    
  3. After the ServiceMeshBinding is updated, the removal of the previous Anthos Service Mesh version starts. The uninstallation might take several minutes to complete. Run the following command to verify that the previous version is removed.

    KUBECONFIG=USER_KUBECONFIG kubectl get deployment -n istio-system -l istio.io/rev=PREVIOUS_REVISION
    

    This command returns an empty result if the uninstallation is complete.