Azure container instance failing to authenticate with user assigned identitty

Question

Azure container instance failing to authenticate with user assigned identitty

Vengadakrishnan Lakshmanan 0

I’ve enabled virtual nodes in AKS and deployed the same application on both the Linux user node pool and the virtual nodes. The application needs to read values from Azure App Configuration using a user-assigned managed identity. I’ve assigned this identity to both the ACI (behind the virtual nodes) and the user node pool, and granted it App Configuration Data Reader permissions. The app works fine when running on the user node pool, but fails on the virtual node (ACI). While checking the logs, I see an authentication error. Logs attached along with this case
Unhandled exception. Azure.Identity.CredentialUnavailableException: DefaultAzureCredential failed to retrieve a token from the included credentials. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/defaultazurecredential/troubleshoot

- EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot

- WorkloadIdentityCredential authentication unavailable. The workload options are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/workloadidentitycredential/troubleshoot

- ManagedIdentityCredential authentication unavailable. No response received from the managed identity endpoint.

- Visual Studio Token provider can't be accessed at /root/.IdentityService/AzureServiceAuth/tokenprovider.json

- Azure CLI not installed

- PowerShell is not installed.

- Azure Developer CLI could not be found.

---> System.AggregateException: Multiple exceptions were encountered while attempting to authenticate. (EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot) (WorkloadIdentityCredential authentication unavailable. The workload options are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/workloadidentitycredential/troubleshoot) (ManagedIdentityCredential authentication unavailable. No response received from the managed identity endpoint.) (Visual Studio Token provider can't be accessed at /root/.IdentityService/AzureServiceAuth/tokenprovider.json) (Azure CLI not installed) (PowerShell is not installed.) (Azure Developer CLI could not be found.)

---> Azure.Identity.CredentialUnavailableException: EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot

at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)

at Azure.Identity.EnvironmentCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.EnvironmentCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)

--- End of inner exception stack trace ---

---> (Inner Exception #1) Azure.Identity.CredentialUnavailableException: WorkloadIdentityCredential authentication unavailable. The workload options are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/workloadidentitycredential/troubleshoot

at Azure.Identity.WorkloadIdentityCredential.GetTokenCoreAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)

at Azure.Identity.WorkloadIdentityCredential.GetTokenCoreAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.WorkloadIdentityCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)<---

---> (Inner Exception #2) Azure.Identity.CredentialUnavailableException: ManagedIdentityCredential authentication unavailable. No response received from the managed identity endpoint.

---> System.Threading.Tasks.TaskCanceledException: The operation was cancelled because it exceeded the configured timeout of 0:00:01. Network timeout can be adjusted in ClientOptions.Retry.NetworkTimeout.

---> System.Threading.Tasks.TaskCanceledException: A task was canceled.

at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)

at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)

at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)

at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)

--- End of inner exception stack trace ---

at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)

at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)

at Azure.Core.Pipeline.HttpClientTransport.ProcessSyncOrAsync(HttpMessage message, Boolean async)

at Azure.Core.Pipeline.HttpPipelineTransportPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)

at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)

--- End of inner exception stack trace ---

at Azure.Core.Pipeline.ResponseBodyPolicy.ThrowIfCancellationRequestedOrTimeout(CancellationToken originalToken, CancellationToken timeoutToken, Exception inner, TimeSpan timeout)

at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)

at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)

at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)

at Azure.Identity.ManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)

at Azure.Identity.ImdsManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)

--- End of inner exception stack trace ---

at Azure.Identity.ImdsManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)

at Azure.Identity.ManagedIdentityClient.AuthenticateCoreAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)

at Azure.Identity.ManagedIdentityClient.AppTokenProviderImpl(AppTokenProviderParameters parameters)

at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.SendTokenRequestToAppTokenProviderAsync(ILoggerAdapter logger, CancellationToken cancellationToken)

at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.GetAccessTokenAsync(CancellationToken cancellationToken, ILoggerAdapter logger)

at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.ExecuteAsync(CancellationToken cancellationToken)

at Microsoft.Identity.Client.Internal.Requests.RequestBase.<>c__DisplayClass11_1.<<RunAsync>b__1>d.MoveNext()

--- End of stack trace from previous location ---

at Microsoft.Identity.Client.Utils.StopwatchService.MeasureCodeBlockAsync(Func`1 codeBlock)

at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken)

at Microsoft.Identity.Client.ApiConfig.Executors.ConfidentialClientExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenForClientParameters clientParameters, CancellationToken cancellationToken)

at Azure.Identity.AbstractAcquireTokenParameterBuilderExtensions.ExecuteAsync[T](AbstractAcquireTokenParameterBuilder`1 builder, Boolean async, CancellationToken cancellationToken)

at Azure.Identity.MsalConfidentialClient.AcquireTokenForClientCoreAsync(String[] scopes, String tenantId, String claims, Boolean enableCae, Boolean async, CancellationToken cancellationToken)

at Azure.Identity.MsalConfidentialClient.AcquireTokenForClientAsync(String[] scopes, String tenantId, String claims, Boolean enableCae, Boolean async, CancellationToken cancellationToken)

at Azure.Identity.ManagedIdentityClient.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)

at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)

at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.ManagedIdentityCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)<---

---> (Inner Exception #3) Azure.Identity.CredentialUnavailableException: Visual Studio Token provider can't be accessed at /root/.IdentityService/AzureServiceAuth/tokenprovider.json

---> System.IO.DirectoryNotFoundException: Could not find a part of the path '/root/.IdentityService/AzureServiceAuth/tokenprovider.json'.

at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirError)

at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode, Boolean failForSymlink, Boolean& wasSymlink, Func`4 createOpenException)

at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, UnixFileMode openPermissions, Int64& fileLength, UnixFileMode& filePermissions, Boolean failForSymlink, Boolean& wasSymlink, Func`4 createOpenException)

at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)

at System.IO.StreamReader.ValidateArgsAndOpenPath(String path, Encoding encoding, Int32 bufferSize)

at System.IO.File.ReadAllText(String path, Encoding encoding)

at Azure.Identity.FileSystemService.ReadAllText(String path)

at Azure.Identity.VisualStudioCredential.GetTokenProviderContent(String tokenProviderPath)

--- End of inner exception stack trace ---

at Azure.Identity.VisualStudioCredential.GetTokenProviderContent(String tokenProviderPath)

at Azure.Identity.VisualStudioCredential.GetTokenProviders(String tokenProviderPath)

at Azure.Identity.VisualStudioCredential.GetTokenImplAsync(TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)

at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)

at Azure.Identity.VisualStudioCredential.GetTokenImplAsync(TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)

at Azure.Identity.VisualStudioCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)<---

---> (Inner Exception #4) Azure.Identity.CredentialUnavailableException: Azure CLI not installed

at Azure.Identity.AzureCliCredential.RequestCliAccessTokenAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)

at Azure.Identity.AzureCliCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)

at Azure.Identity.AzureCliCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.AzureCliCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)<---

---> (Inner Exception #5) Azure.Identity.CredentialUnavailableException: PowerShell is not installed.

at Azure.Identity.AzurePowerShellCredential.CheckForErrors(String output, Int32 exitCode)

at Azure.Identity.AzurePowerShellCredential.RequestAzurePowerShellAccessTokenAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)

at Azure.Identity.AzurePowerShellCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)

at Azure.Identity.AzurePowerShellCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.AzurePowerShellCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)<---

---> (Inner Exception #6) Azure.Identity.CredentialUnavailableException: Azure Developer CLI could not be found.

at Azure.Identity.AzureDeveloperCliCredential.RequestCliAccessTokenAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)

at Azure.Identity.AzureDeveloperCliCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)

at Azure.Identity.AzureDeveloperCliCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.AzureDeveloperCliCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)<---

--- End of inner exception stack trace ---

at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)

at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)

at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Identity.DefaultAzureCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)

at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.SetResultOnTcsFromCredentialAsync(TokenRequestContext context, TaskCompletionSource`1 targetTcs, Boolean async, CancellationToken cancellationToken)

at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetAuthHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async)

at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.TokenRequestState.GetCurrentHeaderValue(Boolean async, Boolean checkForCompletion, CancellationToken cancellationToken)

at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetAuthHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async)

at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AuthenticateAndAuthorizeRequestAsync(HttpMessage message, TokenRequestContext context)

at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)

at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)

at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)

at Microsoft.Extensions.Configuration.AzureAppConfiguration.UserAgentHeaderPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)

at Azure.Core.Pipeline.HttpPipeline.SendRequestAsync(Request request, CancellationToken cancellationToken)

at Azure.Data.AppConfiguration.ConfigurationClient.GetConfigurationSettingsPageAsync(SettingSelector selector, String pageLink, CancellationToken cancellationToken)

at Azure.Core.PageResponseEnumerator.FuncAsyncPageable`1.AsPages(String continuationToken, Nullable`1 pageSizeHint)+MoveNext()

at Azure.Core.PageResponseEnumerator.FuncAsyncPageable`1.AsPages(String continuationToken, Nullable`1 pageSizeHint)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()

at Azure.AsyncPageable`1.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()

at Azure.AsyncPageable`1.GetAsyncEnumerator(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()

at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<>c__DisplayClass26_2.<<LoadAll>b__3>d.MoveNext()

--- End of stack trace from previous location ---

at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<>c__DisplayClass26_2.<<LoadAll>b__3>d.MoveNext()

--- End of stack trace from previous location ---

at Microsoft.Extensions.Configuration.AzureAppConfiguration.TracingUtils.CallWithRequestTracing(Boolean tracingEnabled, RequestType requestType, RequestTracingOptions requestTracingOptions, Func`1 clientCall)

at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.LoadAll(Boolean ignoreFailures, CancellationToken cancellationToken)

at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.Load()

at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)

at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()

at Microsoft.Extensions.Hosting.HostBuilder.InitializeAppConfiguration()

at Microsoft.Extensions.Hosting.HostBuilder.Build()

Himanshu Shekhar 1,935 Reputation points Microsoft External Staff Moderator

2025-09-08T10:40:57.81+00:00

Hello Vengadakrishnan Lakshmanan,

Could you please provide an update on this post?

Kindly let us know if the suggested steps helps or you need further assistance on this issue.

Vengadakrishnan Lakshmanan 0

Hi @Himanshu Shekhar , I created the virtual node pods using below manifest file. When I applied the manifest, the pod was scheduled in AKS and an ACI instance was automatically created to run behind the pod. After that, I manually attached the user-assigned managed identity to the ACI.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ***-fallback
  namespace: ***
  labels:
    app: ***-fallback
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ***-fallback
  template:
    metadata:
      labels:
        app: ***-fallback
        service: ***-service
      annotations:
        "Microsoft.ContainerInstance/userAssignedManagedIdentity": "/subscriptions/***/resourcegroups/***/providers/Microsoft.ManagedIdentity/userAssignedIdentities/****"
    spec:
      imagePullSecrets:
        - name: acr-***
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: type
                    operator: In
                    values:
                      - virtual-kubelet
      tolerations:
        - key: virtual-kubelet.io/provider
          operator: Exists
        - key: azure.com/aci
          operator: Exists
          effect: NoSchedule
      containers:
        - name: api
          image: ***.azurecr.io/webapp.***:123
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: ***
          env:
            - name: ASPNETCORE_ENVIRONMENT
              value: ***
           resources:
            requests:
              memory: "1536Mi"
              cpu: "100m"

User's image

when the AKS cluster was created, a managed identity was automatically generated and assigned to my node pool. For the service account with workload identity, I also configured it to use this same managed identity.

User's image

Himanshu Shekhar 1,935 Reputation points Microsoft External Staff Moderator

2025-09-12T14:42:13.89+00:00

Ack Vengadakrishnan Lakshmanan

Microsoft Entra Workload ID uses Service Account Token Volume Projection (a service account) to enable pods to use a Kubernetes identity - https://dori-uw-1.kuma-moon.com/en-us/azure/aks/workload-identity-overview?tabs=dotnet.

The ACI instance behind your virtual node needs proper managed identity configuration at the container group level and DefaultAzureCredential is failing through all authentication methods when running on AC.

Need to confirm your pod is actually running on a virtual nod

kubectl get pod <pod-name> -n <namespace> -o wide

Verify the ACI instance has the managed identity properly attached:

az container show --resource-group <node-resource-group> --name <aci-container-group-name> --query "identity"

please ensure the managed identity is properly configured on the ACI container group:

List container groups in the node resource group

az container list --resource-group MC_<rg><cluster><location> --query "[].{Name:name,Identity:identity}" -o table

Check specific container group

az container show --resource-group MC_<rg><cluster><location> --name <container-group> --query "identity"

From within your pod, test if the managed identity endpoint is accessible:

Execute into your pod

kubectl exec -it <pod-name> -n <namespace> -- /bin/bash

Check that your managed identity has the required permissions:

Get the managed identity details :

az identity show --resource-group <rg> --name <identity-name>

List role assignments for the identity : az role assignment list --assignee <principal-id> --all

For Fixing ACI Managed Identity Configuration:

Find container groups created by virtual nodes

az container list --resource-group MC_<rg><cluster><location> -o table

Need to manually assign managed identity:

az container update --resource-group MC_<rg>_<cluster>_location> --name <aci-container-group-name>--assign-identity <managed-identity-resource-id>

Need to verify the assignment

az container show --resource-group MC_<rg><cluster><location> --name <aci-container-group-name> --query "identity"

Kindly let us know if the suggested steps helps or you need further assistance on this issue.
Himanshu Shekhar 1,935 Reputation points Microsoft External Staff Moderator

2025-09-16T12:02:26.4333333+00:00

Hello Vengadakrishnan Lakshmanan

Could you please provide an update on this post?

Kindly let us know if the suggested steps helps or you need further assistance on this issue.
Himanshu Shekhar 1,935 Reputation points Microsoft External Staff Moderator

2025-09-17T13:45:44.2433333+00:00

Hello Vengadakrishnan Lakshmanan

Could you please provide an update on this post?

Kindly let us know if the suggested steps helps or you need further assistance on this issue.

1 answer

Your answer

Himanshu Shekhar 1,935 Reputation points Microsoft External Staff Moderator

2025-09-08T10:40:57.81+00:00

Hello Vengadakrishnan Lakshmanan,

Could you please provide an update on this post?

Kindly let us know if the suggested steps helps or you need further assistance on this issue.
Vengadakrishnan Lakshmanan 0 Reputation points

2025-09-09T09:45:15.35+00:00

Hi @Himanshu Shekhar , I created the virtual node pods using below manifest file. When I applied the manifest, the pod was scheduled in AKS and an ACI instance was automatically created to run behind the pod. After that, I manually attached the user-assigned managed identity to the ACI.

apiVersion: apps/v1 kind: Deployment metadata: name: ***-fallback namespace: *** labels: app: ***-fallback spec: replicas: 1 selector: matchLabels: app: ***-fallback template: metadata: labels: app: ***-fallback service: ***-service annotations: "Microsoft.ContainerInstance/userAssignedManagedIdentity": "/subscriptions/***/resourcegroups/***/providers/Microsoft.ManagedIdentity/userAssignedIdentities/****" spec: imagePullSecrets: - name: acr-*** affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: type operator: In values: - virtual-kubelet tolerations: - key: virtual-kubelet.io/provider operator: Exists - key: azure.com/aci operator: Exists effect: NoSchedule containers: - name: api image: ***.azurecr.io/webapp.***:123 imagePullPolicy: IfNotPresent ports: - containerPort: *** env: - name: ASPNETCORE_ENVIRONMENT value: *** resources: requests: memory: "1536Mi" cpu: "100m"

when the AKS cluster was created, a managed identity was automatically generated and assigned to my node pool. For the service account with workload identity, I also configured it to use this same managed identity.
Himanshu Shekhar 1,935 Reputation points Microsoft External Staff Moderator

2025-09-16T12:02:26.4333333+00:00

Hello Vengadakrishnan Lakshmanan

Could you please provide an update on this post?

Kindly let us know if the suggested steps helps or you need further assistance on this issue.
Himanshu Shekhar 1,935 Reputation points Microsoft External Staff Moderator

2025-09-17T13:45:44.2433333+00:00

Hello Vengadakrishnan Lakshmanan

Could you please provide an update on this post?

Kindly let us know if the suggested steps helps or you need further assistance on this issue.

Answer 1

Hi Vengadakrishnan Lakshmanan

Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.

WorkloadIdentityCredential → “not fully configured.”

AKS Workload Identity is not supported on the virtual nodes addon (Virtual Kubelet/ACI).

So, even if WI works on your VMSS node pool, pods running on the ACI virtual node can’t use it. That’s why it shows WorkloadIdentityCredential authentication unavailable. This article helps to understand Microsoft Entra Workload ID, and reviews the options available for planning the project strategy and potential migration from Microsoft Entra pod-managed identity : Use a Microsoft Entra Workload ID on AKS - Azure Kubernetes Service | Microsoft Learn

When a container has a Managed Identity (MI), the SDK pings to IMDS (169.254.169.254) to get a token. In Azure Container Instances (ACI), MI only works if the whole container group has a system-assigned or user-assigned identity. If no identity is attached, IMDS won’t return a token, and your app will just get time out.

How to use managed identities with Azure Container Instances: Enable managed identity in container group - Azure Container Instances | Microsoft Learn

For confirming whether pod really sits on the virtual node (ACI): kubectl get pod <pod> -n <ns> -o wide

Virtual nodes in AKS use Virtual Kubelet to run your pod inside an Azure Container Instances (ACI) container group instead of on a regular AKS VM node.

Create and configure an Azure Kubernetes Services (AKS) cluster to use virtual nodes using Azure CLI: https://dori-uw-1.kuma-moon.com/en-us/azure/aks/virtual-nodes-cli

Microsoft Entra Workload ID uses Service Account Token Volume Projection (a service account) to enable pods to use a Kubernetes identity - https://dori-uw-1.kuma-moon.com/en-us/azure/aks/workload-identity-overview?tabs=dotnet

The ACI instance behind your virtual node needs proper managed identity configuration at the container group level and DefaultAzureCredential is failing through all authentication methods when running on AC.

Need to confirm your pod is actually running on a virtual nod

kubectl get pod <pod-name> -n <namespace> -o wide

Verify the ACI instance has the managed identity properly attached:

az container show --resource-group <node-resource-group> --name <aci-container-group-name> --query "identity"

please ensure the managed identity is properly configured on the ACI container group:

List container groups in the node resource group

az container list --resource-group MC_<rg><cluster><location> --query "[].{Name:name,Identity:identity}" -o table

Check specific container group

az container show --resource-group MC_<rg><cluster><location> --name <container-group> --query "identity"

From within your pod, test if the managed identity endpoint is accessible:

Execute into your pod

kubectl exec -it <pod-name> -n <namespace> -- /bin/bash

Check that your managed identity has the required permissions:

Get the managed identity details :

az identity show --resource-group <rg> --name <identity-name>

List role assignments for the identity : az role assignment list --assignee <principal-id> --all

For Fixing ACI Managed Identity Configuration:

Find container groups created by virtual nodes

az container list --resource-group MC_<rg><cluster><location> -o table

Need to manually assign managed identity:

az container update --resource-group MC_<rg>_<cluster>_location> --name <aci-container-group-name>--assign-identity <managed-identity-resource-id>

Need to verify the assignment

az container show --resource-group MC_<rg><cluster><location> --name <aci-container-group-name> --query "identity"

Kindly let us know if the suggested steps helps or you need further assistance on this issue.

Regards

Himanshu

Share via

Azure container instance failing to authenticate with user assigned identitty

1 answer

Your answer