Secure APIM and Azure OpenAI with managed identity

Okay, you may have read somewhere that API keys are insecure, or you may have heard about managed identities. But what are they, and why are they better than API keys? Let me answer that question and show you a practical example of how to use managed identities in Azure.

References

What is a Managed Identity?

A feature of Azure that allows applications to securely authenticate to cloud services without having to manage credentials such as API keys.

Managed identities in Azure work by providing: *Automatically managed ID* Used when an application connects to resources that support Azure Active Directory (Azure AD) authentication.

> Okay, so I don’t have to create this identity myself. So far, so good.

Types of managed identities

Here are the details on how it works: Azure has two types of managed identities:

System Assigned Management ID

– creation: Automatically created when you enable it on Azure resources such as virtual machines or app services.

– Life cycle: Tied to the lifecycle of the resource. When the resource is deleted, the managed identity is also deleted.

– usage: Only resources that can request tokens from Azure AD¹ using that ID can use this ID.

> Okay, so it’s tied to a specific resource, I see. What about other types?

Custom Management ID:

– creation: Created as a standalone Azure resource.

– Life cycle: It is managed separately from the resources that use it.

– usage: You can share the same ID by assigning it to multiple Azure resources.

> Okay, this is more flexible and can be shared across resources. And there are different types, so explain how they actually work, the process.

Process – How it works

The process from authentication to resource access is as follows:

1. proof:

– When an application needs to access a resource, it uses a managed identity to request a token from Azure AD.

– Azure AD authenticates the managed identity and issues a token.

> Okay, so I’ve got the token, but how do I actually use it to access resources?

2. Resource Access:

– The application uses the token to access the desired resource (e.g. Azure Key Vault, Azure Storage).

– The resource validates the token and grants access based on the permissions assigned to the managed identity.

> Yes, basically it’s like an intermediary that provides tokens to securely access resources.

Real-world example: Azure Open AI services + APIM

Okay, let’s look at how to use managed identities in a real-world scenario. Let’s take the Azure Open AI service as an example. So you’re starting to generate text for your application using the Azure Open AI service, and it’s fantastic. You’re currently authenticating your application with the service using an API key, and your technical lead has mentioned that this isn’t the most secure way to do it. Instead, use managed identities.

> Okay, how hard can it be? Let’s see if we can plan the steps.

Oh, and remember that you’ll need additional features like error management, load balancing, usage tracking, etc., so you’ll need to factor that into your planning too.

You think about it and decide that you need Azure API Management to handle additional features and Azure Open AI Services. But how do you enable Managed Identity in this setup?

-0- Create cloud resources

You need to create an Azure API Management instance and an Azure Open AI Service. You can do this in the Azure portal or by using Bicep or ARM templates.

We’ll look at a real-world example shortly, but for now, let’s assume these resources have been created.

-1- Azure API Management (APIM)

Here’s how to enable managed identity for APIM:

Activate Managed ID:

– Go to the Azure Portal.

– Go to your Azure API Management instance.

– Select Security > Managed Identity > Set Status. ~ in.

Add a managed identity policy to your API

To use a managed identity, you need to add a policy to your API. You can do this by adding the following policy to your API:

– Navigate to APIs in your Azure API Management instance.

– Add the following to your API in design mode:

    @("Bearer " + (string)context.Variables["managed-id-access-token"])
​

The above policy retrieves a token from the managed identity and sets it as the authorization header for API calls to Azure Open AI services.

-2- Azure Open AI Service: Enable Managed Identity for Azure Open AI

There are two steps to enabling managed identity for Azure Open AI services:

Activate Managed ID:

– Go to the Azure Portal.

– Go to your Azure Open AI service instance.

– Enable managed identity for your service by selecting Resource Management > Identity > System Assigned. Once there, you can enable the system assigned managed identity.

Assigning Permissions:

Since APIM calls Azure Open AI services, you need to assign permissions for APIM to call Azure Open AI services.

In the Azure Portal for resources:

– Select Access Control (IAM) > + Add > Add Role Assignment > “Search”.Cognitive Service User”

– Choose **next**Check the managed ID and “+ Select Member“. Select your APIM instance here and click. **Choose**.

Okay, now we have a managed identity that can be used in both APIM and Azure Open AI services, and we have assigned it the necessary permissions to call Azure Open AI services from APIM.

-3- Try it

For the practical part, let’s use an existing Azure sample. [Azure sample for Azure Open AI and APIM](Things to do)

Open a terminal:

– wren `azd authentication login`This will allow you to log in to your Azure account.

– run `azd up`This will deploy the resource to your Azure account.

– Go to `source` Folder in the cloned repository.

– run `Install npm`This will install the required packages.

– run `npm start`

– When you type text and select the arrow to send the message, the generated text will be displayed.

Infrastructure as Code

So far, we’ve shown you how to manually set up managed identities for APIM and Azure Open AI Services via the Azure Portal, but you can also do this using Infrastructure as Code (IaC) tools like Bicep or ARM templates. Let’s show you how to do this using Bicep.

Biceps File for APIM

Let’s learn how to enable and create a managed identity for APIM using Bicep.

resource apimService 'Microsoft.ApiManagement/service@2023-09-01-preview' = {
  name: name
  location: location
  tags: union(tags, { 'azd-service-name': name })
  sku: {
    name: sku
    capacity: (sku == 'Consumption') ? 0 : ((sku == 'Developer') ? 1 : skuCount)
  }
  properties: {
    publisherEmail: publisherEmail
    publisherName: publisherName
    // Custom properties are not supported for Consumption SKU
    customProperties: sku == 'Consumption' ? {} : {
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_128_GCM_SHA256': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_256_CBC_SHA256': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_128_CBC_SHA256': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_256_CBC_SHA': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_128_CBC_SHA': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TripleDes168': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Tls10': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Tls11': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Ssl30': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Tls10': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Tls11': 'false'
      'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Ssl30': 'false'
    }
  }
  identity: {
    type: 'SystemAssigned'
  }
}

From a managed identity perspective, you can see that we’ve added: `Identity` Set properties for APIM resources to type `System allocated`.

Bicep files for Azure Open AI Service

For Azure Open AI services, you can enable managed identity using the following Bicep code:

resource openAI 'Microsoft.CognitiveServices/accounts@2021-04-30' = {
  name: 'your-openai-resource-name'
  location: 'your-location'
  sku: {
    name: 'S0'
  }
  kind: 'OpenAI'
  properties: {
    // Add other necessary properties here
  }
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    publicNetworkAccess: 'Disabled'
    networkAcls: {
      defaultAction: 'Deny'
    }
    disableLocalAuth: true
  }
}

As you can see, we added: `Identity` Sets the property for the Azure Open AI service resource to type. `System allocated`Same as APIM.

Another thing to keep in mind for security is:

– that `NetworkAcls` The properties are set as follows: `Reject` Basically, this is a security feature that restricts access to Azure Open AI services to only specified resources.

– also, `disableLocalAuth` was set to `True` Disable local authentication.

Then, assign the permissions required for APIM to call Azure Open AI services.

resource roleAssignment 'Microsoft.Authorization/roleAssignments@2020-04-01-preview' = {
  name: guid(openAI.id, 'cognitive-services-openai-user-role')
  properties: {
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'c1c469a3-0a2d-4bba-b0e1-0eaf1d3d728b') // Role ID for Cognitive Services OpenAI User
    principalId: openAI.identity.principalId
    principalType: 'ServicePrincipal'
    scope: openAI.id
  }
}

Please note the following: `range` is set to `openAI.id`Azure Open AI service resource ID. That is, the APIM instance can only call Azure Open AI services.

Scope Discussion

“When assigningCognitive Services OpenAI Users“Roles and scopes for APIM managed identities can be set at the Cognitive Services account level or the resource group level, depending on your requirements.

– Cognitive Service Account Level: This is more restrictive and ensures that the managed identity can only access specific Azure OpenAI resources. This is useful if you want to limit access to only one resource.

– Resource group level: This is broader and allows the managed identity to access all resources within the specified resource group. This is useful if you have multiple Azure OpenAI resources within the same resource group and want to more easily manage access.

However, if you want to allow other resources to call Azure Open AI services, you can scope it at the subscription or resource group level. Here’s how to add it to a role assignment at the resource group level:

resource roleAssignment 'Microsoft.Authorization/roleAssignments@2020-04-01-preview' = {
  name: guid(apimIdentity.id, resourceGroup().id, 'cognitive-services-openai-user-role')
  properties: {
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'c1c469a3-0a2d-4bba-b0e1-0eaf1d3d728b') // Role ID for Cognitive Services OpenAI User
    principalId: apimIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    scope: resourceGroup().id
  }
}

Subscriptions in Azure API Management are a way to control access to your APIs. When publishing APIs through APIM, you can use subscription keys to secure them. Here’s a quick overview:

• subscribe: This is a container for a pair of subscription keys (primary and secondary). Developers need a valid subscription key to make API calls.
• Subscription ID: Each subscription has a unique identifier called a subscription ID.

How do subscriptions relate to APIM resources?

Subscription scope: Subscriptions can be associated with different scopes within an APIM instance.

• Product range: A subscription can be tied to a specific product, which is a collection of one or more APIs. By subscribing to a product, a developer gains access to all the APIs within that product.
• API Scope: Subscriptions can also be associated with individual APIs, allowing more granular control over access.

resource api1 'Microsoft.ApiManagement/service/apis@2020-06-01-preview' = {
  parent: apimService
  name: apiName
  properties: {
    displayName: apiName
    apiType: 'http'
    path: apiSuffix
    format: 'openapi+json-link'
    value: 'https://raw.githubusercontent.com/Azure/azure-rest-api-specs/main/specification/cognitiveservices/data-plane/AzureOpenAI/inference/preview/2024-03-01-preview/inference.json'
    subscriptionKeyParameterNames: {
      header: 'api-key'
    }
    
  }
  resource apimDiagnostics 'diagnostics@2023-05-01-preview' = {
    name: 'applicationinsights' // Use a supported diagnostic identifier
    properties: {
      loggerId: '/subscriptions/${subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.ApiManagement/service/${apimService.name}/loggers/${apimLogger.name}'
      metrics: true
    }
  }
}

// Creating a product for the API. Products are used to group APIs and apply policies to them
resource product 'Microsoft.ApiManagement/service/products@2020-06-01-preview' = {
  parent: apimService
  name: productName
  properties: {
    displayName: productName
    description: productDescription
    state: 'published'
    subscriptionRequired: true
  }
}

// Create PRODUCT-API association the API with the product
resource productApi1 'Microsoft.ApiManagement/service/products/apis@2020-06-01-preview' = {
  parent: product
  name: api1.name
}

// Creating a user for the API Management service
resource user 'Microsoft.ApiManagement/service/users@2020-06-01-preview' = {
  parent: apimService
  name: 'userName'
  properties: {
    firstName: 'User'
    lastName: 'Name'
    email: 'user@example.com'
    state: 'active'
  }
}

// Creating a subscription for the API Management service
// NOTE: the subscription is associated with the user and the product, AND the subscription ID is what will be used in the request to authenticate the calling client
resource subscription 'Microsoft.ApiManagement/service/subscriptions@2020-06-01-preview' = {
  parent: apimService
  name: 'subscriptionAIProduct'
  properties: {
    displayName: 'Subscribing to AI services'
    state: 'active'
    ownerId: user.id
    scope: product.id
  }
}

let body = {
  "model":"gpt-35-turbo","messages":[
  {
    "role":"system","content":"You're a helpful assistant"
  },
  {
    "role":"user","content":prompt
  }
]};

return fetch(URL_CHAT, {
  method: "POST",
  headers: {
    "api-key": process.env.SUBSCRIPTION_KEY,
    "Content-Type": "application/json"
  },
  body: JSON.stringify(body)
})

Summary – Benefits

To summarize what you learned in this article, managed identities offer several advantages over traditional API keys.

– No credential management: Eliminates the need to store and manage credentials in your code.

– Enhanced security: The risk of credential leaks is reduced since credentials are not exposed.

– Simplified access control: Managed identities can be assigned specific roles and permissions to provide granular access control.

To use a managed identity, you must enable managed identities on your Azure resources and configure the required permissions.

We hope you can now sleep well at night knowing that your applications are now more secure with managed identities. 🙂