Anthos private mode known issues

This page lists known issues with Anthos private mode, along with ways that you can avoid or recover from these issues.

Unable to bind a service account to the preset anthos-platform-admin ClusterRole

You cannot successfully bind a service account to the preset anthos-platform-admin or anthos-platform-admin-read-only ClusterRole. Some permissions must be dynamically granted to the subjects by the controller, but the controller currently only support users and groups.

Pod connectivity failures and reverse path filtering

Anthos private mode configures reverse path filtering on nodes to disable source validation (net.ipv4.conf.all.rp_filter=0). If therp_filter setting is changed to 1 or 2, Pods will fail due to out-of-node communication timeouts.

Reverse path filtering is set with rp_filter files in the IPv4 configuration folder (net/ipv4/conf/all). This value might also be overridden by sysctl, which stores reverse path filtering settings in a network security configuration file, such as /etc/sysctl.d/60-gce-network-security.conf.

To restore Pod connectivity, either set net.ipv4.conf.all.rp_filter back to 0 manually, or restart the anetd Pod to set net.ipv4.conf.all.rp_filter back to 0. To restart the anetd Pod, use the following commands to locate and delete the anetd Pod and a new anetd Pod will start up in its place:

kubectl get pods -n kube-system
kubectl delete pods -n kube-system ANETD_XYZ

Replace ANETD_XYZ with the name of the anetd Pod.

Redirect loop when accessing the management center UI

If the browser is in a redirect loop when accessing Management Center Console after setting up identity providers and signing in with the identity provider, there might be an error in the OIDC settings. See Reset Authentication config.

This typically occurs when the username claim or the group claim does not exist in the requested OIDC scopes. Check the JWT from the OIDC provider to verify that the correct username claim or group claim is used when setting up identity providers.

Client IDs must be unique when configuring OIDC

This issue might manifest in a redirect loop after successfully authenticating with the OIDC provider. Check the identity provider configuration to see if there are other identity providers that use the same client ID:

KUBECONFIG=${ADMIN_KUBECONFIG} kubectl get clientconfig -n kube-public default -oyaml

If there are duplicate client IDs, ask your identity provider to create a new client ID.

Manually refresh web pages after authorization token expires

If you see a web page showing errors and it has been at least one hour (or less depending on your OIDC provider settings) since your last login, click the refresh button in your browser to hard refresh the web page with a new authorization token. You might be prompted by your OIDC provider to log in again.

Failure when creating the admin cluster

If you have a problem creating the admin cluster where the kube-proxy Pod in the kind cluster does not start, try manually setting nf_conntrack_max on the admin workstation. For example:

sudo sysctl -w net.netfilter.nf_conntrack_max=131072

What's next