The GCP service account has Artifact registry reader role in the project and the kubernetes service account has roles/workloadIdentity.user on GCP service account.

However pod run always fails with

Failed to pull image "us-central1-docker.pkg.dev/my-project-id/ar-bucket-name/my-docker-image:latest": failed to pull and unpack image "us-central1-docker.pkg.dev/my-project-id/ar-bucket-name/my-docker-image:latest": failed to resolve reference "us-central1-docker.pkg.dev/my-project-id/ar-bucket-name/my-docker-image:latest": failed to authorize: failed to fetch oauth token: unexpected status from GET request to /v2/token?scope=repository%3Amy-project-id%2Far-bucket-name%2Fmy-docker-image%3Apull&service=us-central1-docker.pkg.dev: 403 Forbidden
  Warning  Failed     12s (x3 over 51s)  kubelet            Error: ErrImagePull

i AM permissions on my GCP service account:

gcloud projects get-iam-policy my-project-id  \
--flatten="bindings[].members" \
--format='table(bindings.role)' \
--filter="bindings.members:[email protected]"
ROLE
roles/artifactregistry.reader

Adding workload identity user to ksa

gcloud iam service-accounts add-iam-policy-binding [email protected] \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:my-project-id.svc.id.goog[k8s-namespace/pod-service-account-name]"
Updated IAM policy for serviceAccount [[email protected]].
bindings:
- members:
  - serviceAccount:my-project-id.svc.id.goog[k8s-namespace/pod-service-account-name]
  role: roles/iam.workloadIdentityUser
etag: BwYuueR7rFQ=
version: 1

The service account for the pod

kubectl describe pod $pod -n k8s-namespace
Name:             my-pod-name
Namespace:        k8s-namespace
Priority:         0
Service Account:  pod-service-account-name
. . . 

Despite the KSA having impersonation role on the GCP service account that has the artifact registry reader role, I even tried directly giving the artifact registry role to the KSA

principal://iam.googleapis/projects/122066631103/locations/global/workloadIdentityPools/my-project-id.svc.id.goog/subject/ns/k8s-namespace/sa/pod-service-account-name

Yet it made no difference.