|
| 1 | +# Troubleshooting Azure Key Vault SDK Issues |
| 2 | +The Azure Key Vault SDKs for Java use a common HTTP pipeline and authentication to create, update, and delete secrets, keys, and certificates in Key Vault and Managed HSM. This troubleshooting guide contains steps for diagnosing issues common to these SDKs. |
| 3 | + |
| 4 | +For any package-specific troubleshooting guides, see any of the following: |
| 5 | + |
| 6 | +* [Troubleshooting Azure Key Vault Administration SDK Issues](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/keyvault/azure-security-keyvault-administration/TROUBLESHOOTING.md) |
| 7 | +* [Troubleshooting Azure Key Vault Certificates SDK Issues](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/keyvault/azure-security-keyvault-certificates/TROUBLESHOOTING.md) |
| 8 | +* [Troubleshooting Azure Key Vault Keys SDK Issues](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/keyvault/azure-security-keyvault-keys/TROUBLESHOOTING.md) |
| 9 | +* [Troubleshooting Azure Key Vault Secrets SDK Issues](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/keyvault/azure-security-keyvault-secrets/TROUBLESHOOTING.md) |
| 10 | + |
| 11 | +## Table of Contents |
| 12 | +* [Troubleshooting Authentication Issues](#troubleshooting-authentication-issues) |
| 13 | + * [HTTP 401 Errors](#http-401-errors) |
| 14 | + * [Frequent HTTP 401 Errors in Logs](#frequent-http-401-errors-in-logs) |
| 15 | + * [AKV10032: Invalid issuer](#akv10032-invalid-issuer) |
| 16 | + * [Other Authentication Issues](#other-authentication-issues) |
| 17 | + * [HTTP 403 Errors](#http-403-errors) |
| 18 | + * [Operation Not Permitted](#operation-not-permitted) |
| 19 | + * [Access Denied to First Party Service](#access-denied-to-first-party-service) |
| 20 | +* [Other Service Errors](#other-service-errors) |
| 21 | + * [HTTP 429: Too Many Requests](#http-429-too-many-requests) |
| 22 | + |
| 23 | +## Troubleshooting Authentication Issues |
| 24 | +### HTTP 401 Errors |
| 25 | +HTTP 401 errors may indicate problems authenticating. |
| 26 | + |
| 27 | +#### Frequent HTTP 401 Errors in Logs |
| 28 | +Most often, this is expected. Azure Key Vault issues a challenge for initial requests that force authentication. You may see these errors most often during application startup, but may also see these periodically during the application's lifetime when authentication tokens are near expiration. |
| 29 | + |
| 30 | +If you are not seeing subsequent exceptions from the Key Vault SDKs, authentication challenges are likely the cause. |
| 31 | + |
| 32 | +#### AKV10032: Invalid issuer |
| 33 | +You may see an error similar to: |
| 34 | + |
| 35 | +```text |
| 36 | +com.azure.core.exception.HttpResponseException: Status code 401, "{"error":{"code":"Unauthorized","message":"AKV10032: Invalid issuer. Expected one of https://sts.windows.net/{tenant 1}/, found https://sts.windows.net/{tenant 2}/."}}" |
| 37 | +``` |
| 38 | + |
| 39 | +This is most often caused by being logged into a different tenant than the Key Vault authenticates. See our [DefaultAzureCredential] documentation to see the order credentials are read. You may be logged into a different tenant for one credential that gets read before another credential. For example, you might be logged into Visual Studio under the wrong tenant even though you're logged into the Azure CLI under the right tenant. |
| 40 | + |
| 41 | +Automatic tenant discovery support has been added when referencing package `azure-identity` version 1.4.0 or newer, and any of the following Key Vault SDK package versions or newer: |
| 42 | + |
| 43 | +Package | Minimum Version |
| 44 | +--- | --- |
| 45 | +`azure-security-keyvault-administration` | 4.1.0-beta.3 |
| 46 | +`azure-security-keyvault-certificates` | 4.3.0-beta.3 |
| 47 | +`azure-security-keyvault-keys` | 4.4.0-beta.5 |
| 48 | +`azure-security-keyvault-secrets` | 4.4.0-beta.3 |
| 49 | + |
| 50 | +Upgrading to the package versions should resolve any "Invalid Issuer" errors as long as the application or user is a member of the resource's tenant. |
| 51 | + |
| 52 | +#### Other Authentication Issues |
| 53 | +If you are using the `azure-identity` package - which contains [DefaultAzureCredential] - to authenticate requests to Azure Key Vault, please see our [troubleshooting guide](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/identity/azure-identity/TROUBLESHOOTING.md). |
| 54 | + |
| 55 | +### HTTP 403 Errors |
| 56 | +HTTP 403 errors indicate the user is not authorized to perform a specific operation in Key Vault or Managed HSM. |
| 57 | + |
| 58 | +#### Operation Not Permitted |
| 59 | +You may see an error similar to: |
| 60 | + |
| 61 | +```text |
| 62 | +com.azure.core.exception.HttpResponseException: Status code 403, {"error":{"code":"Forbidden","message":"Operation decrypt is not permitted on this key.","innererror":{"code":"KeyOperationForbidden"}}} |
| 63 | +``` |
| 64 | + |
| 65 | +The operation and inner `code` may vary, but the rest of the text will indicate which operation is not permitted. This error indicates that the authenticated application or user does not have permissions to perform that operation, though the cause may vary. |
| 66 | + |
| 67 | +1. Check that the application or user has the appropriate permissions: |
| 68 | + * [Access policies](https://docs.microsoft.com/azure/key-vault/general/assign-access-policy) (Key Vault) |
| 69 | + * [Role-Based Access Control (RBAC)](https://docs.microsoft.com/azure/key-vault/general/rbac-guide) (Key Vault and Managed HSM) |
| 70 | +2. If the appropriate permissions are assigned to your application or user, make sure you are authenticating as that user. |
| 71 | + If using the [DefaultAzureCredential] a different credential might've been used than one you expected. |
| 72 | + [Enable logging](https://docs.microsoft.com/azure/developer/java/sdk/logging-overview) and you will see which credential the [DefaultAzureCredential] used as shown below, and why previously-attempted credentials were rejected. |
| 73 | + |
| 74 | + ```text |
| 75 | + [ERROR] c.azure.identity.EnvironmentCredential : Azure Identity => ERROR in EnvironmentCredential: Missing required environment variable AZURE_CLIENT_ID |
| 76 | + [ERROR] c.azure.identity.EnvironmentCredential : EnvironmentCredential authentication unavailable. Environment variables are not fully configured. |
| 77 | + [INFO] c.azure.identity.DefaultAzureCredential : Azure Identity => Attempted credential EnvironmentCredential is unavailable. |
| 78 | + [ERROR] c.a.i.implementation.IdentityClient : ManagedIdentityCredential authentication unavailable. Connection to IMDS endpoint cannot be established, connect timed out. |
| 79 | + [ERROR] c.a.identity.ManagedIdentityCredential : Azure Identity => ERROR in getToken() call for scopes [https://management.core.windows.net//.default]: ManagedIdentityCredential authentication unavailable. Connection to IMDS endpoint cannot be established, connect timed out. |
| 80 | + [INFO] c.azure.identity.DefaultAzureCredential : Azure Identity => Attempted credential ManagedIdentityCredential is unavailable. |
| 81 | + [ERROR] c.a.identity.SharedTokenCacheCredential : Azure Identity => ERROR in getToken() call for scopes [https://management.core.windows.net//.default]: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache. |
| 82 | + [INFO] c.azure.identity.DefaultAzureCredential : Azure Identity => Attempted credential SharedTokenCacheCredential is unavailable. |
| 83 | + [ERROR] com.azure.identity.IntelliJCredential : Azure Identity => ERROR in getToken() call for scopes [https://management.core.windows.net//.default]: Unrecognized field "tenantId" (class com.azure.identity.implementation.IntelliJAuthMethodDetails), not marked as ignorable (4 known properties: "authMethod", "azureEnv", "accountEmail", "credFilePath"]) |
| 84 | + ``` |
| 85 | + |
| 86 | +#### Access Denied to First Party Service |
| 87 | +You may see an error similar to: |
| 88 | + |
| 89 | +```text |
| 90 | +com.azure.core.exception.HttpResponseException: Status code 403, {"error":{"code":"Forbidden","message":"Access denied to first party service. ...","innererror":{"code":"AccessDenied"}}} |
| 91 | +``` |
| 92 | + |
| 93 | +The error `message` may also contain the tenant ID (`tid`) and application ID (`appid`). This error may occur because: |
| 94 | + |
| 95 | +1. You have the **Allow trust services** option enabled and are trying to access the Key Vault from a service not on |
| 96 | + [this list](https://docs.microsoft.com/azure/key-vault/general/overview-vnet-service-endpoints#trusted-services) of |
| 97 | + trusted services. |
| 98 | +2. You are authenticated against a Microsoft Account (MSA) in Visual Studio or another credential provider. See |
| 99 | + [above](#operation-not-permitted) for troubleshooting steps. |
| 100 | + |
| 101 | +## Other Service Errors |
| 102 | +To troubleshoot additional HTTP service errors not described below, see [Azure Key Vault REST API Error Codes](https://docs.microsoft.com/azure/key-vault/general/rest-error-codes). |
| 103 | + |
| 104 | +### HTTP 429: Too Many Requests |
| 105 | +If you get an exception or see logs that describe HTTP 429, you may be making too many requests to Key Vault too quickly. |
| 106 | + |
| 107 | +Possible solutions include: |
| 108 | + |
| 109 | +1. Use a singleton for any `CertificateClient`, `KeyClient`, `CryptographyClient`, or `SecretClient` and their async counterparts in your application for a single Key Vault. |
| 110 | +2. Use a single instance of [DefaultAzureCredential] or other credential you use to authenticate your clients for each Key Vault or Managed HSM endpoint you need to access. |
| 111 | +3. You could cache a certificate, key, or secret in memory for a time to reduce calls to retrieve them. |
| 112 | +4. Use [Azure App Configuration](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/appconfiguration/azure-data-appconfiguration/README.md) for storing non-secrets and references to Key Vault secrets. Storing all app configuration in Key Vault will increase the likelihood of requests being throttled as more application instances are started. |
| 113 | +5. If you are performing encryption or decryption operations, consider using wrap and unwrap operations |
| 114 | + for a symmetric key which may also improve application throughput. |
| 115 | + |
| 116 | +See our [Azure Key Vault throttling guide](https://docs.microsoft.com/azure/key-vault/general/overview-throttling) for more information. |
| 117 | + |
| 118 | +[DefaultAzureCredential]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/identity/azure-identity/README.md#defaultazurecredential |
0 commit comments