California Consumer Privacy Act (CCPA) Opt-Out Icon

When building an HTTP API, you may want to execute long-running tasks synchronously in your request handlers. This approach can lead to slow responses, timeouts, and resource exhaustion. If a request times out or the connection is lost, the client has no way of knowing whether the task has completed. For CPU-bound tasks, this approach can also cause the server to hang, making it unresponsive to other requests.

In this post, we’ll look at how to build an asynchronous HTTP API using Azure Container Apps. We’ll create a simple API that implements: Asynchronous request-response pattern: Asynchronous processing performed on APIs and tasks hosted in container apps. This approach provides a much more robust and scalable solution for long-running tasks.

Long-running API requests in Azure container apps

Azure Container Apps is a serverless container platform. It is ideal for hosting a variety of workloads, including HTTP APIs.

Like other serverless and PaaS platforms, Azure Container Apps is designed for short-lived requests. The current maximum ingress timeout is 4 minutes. As an auto-scaling platform, it is designed to scale dynamically based on the number of incoming requests. When scaling down, replicas are removed. Long-running requests can be abruptly terminated when replicas serving requests are removed.

Working with Azure Container Apps

Azure Container Apps has two types of resources: apps and tasks. Apps are long-running services that respond to HTTP requests or events. Tasks are tasks that run until they are completed and can be triggered by schedules or events.

Tasks can also be triggered programmatically. This is ideal for implementing asynchronous processing in HTTP APIs. The API can start executing a task to process a request and return a response immediately. The task can then take as long as it takes to complete processing. The client can poll the status endpoint in the app to check if the task is complete and get the result.

Asynchronous request-response pattern

Asynchronous request-response is a common pattern for handling long-running tasks in HTTP APIs. Instead of waiting for the task to complete, the API returns a status code indicating that the task has started. The client can then poll the API to determine when the task has completed.

Here’s how the pattern applies to Azure Container Apps:

1. A client initiates a task by sending a request to the API (hosted as a container app).
2. The API stores the request (in this example we use Azure Cosmos DB), initiates a job to process the operation, and returns: 202 Accepted With status code Location A header pointing to the status endpoint.
3. The client polls the status endpoint. While the job is in progress, the status endpoint returns: 200 OK With status code Retry-After A header that indicates when the client should poll again.
4. When the job is completed, the status endpoint returns: 303 See Other With status code Location A header pointing to the result. The client automatically follows the redirect to get the result.

Asynchronous HTTP API App

The source code can be found here. GitHub Repository.

The API is a simple Node.js app that uses Fastify. It shows how to build an asynchronous HTTP API that accepts orders and offloads order processing to tasks. The app has a few simple endpoints.

POST / ORDER

This endpoint accepts an order from the body. It stores the order in Cosmos DB in a “pending” state and starts executing a job to process the order.

fastify.post('/orders', async (request, reply) => {
    const orderId = randomUUID()

    // save order to Cosmos DB
    await container.items.create({
        id: orderId,
        status: 'pending',
        order: request.body,
    })

    // start job execution
    await startProcessorJobExecution(orderId)

    // return 202 Accepted with Location header
    reply.code(202).header('Location', '/orders/status/' + orderId).send()
})

We’ll look at this task later in this article. In the code snippet above, startProcessorJobExecution A function that starts a task execution. Starts a task using the Azure Container Apps Management SDK.

const credential = new DefaultAzureCredential()
const containerAppsClient = new ContainerAppsAPIClient(credential, subscriptionId)

// ...

async function startProcessorJobExecution(orderId) {
    // get the existing job's template
    const { template: processorJobTemplate } = 
        await containerAppsClient.jobs.get(resourceGroupName, processorJobName)

    // add the order ID to the job's environment variables
    const environmentVariables = processorJobTemplate.containers[0].env
    environmentVariables.push({ name: 'ORDER_ID', value: orderId })
    const jobStartTemplate = { template: processorJobTemplate }

    // start the job execution with the modified template
    const jobExecution = await containerAppsClient.jobs.beginStartAndWait(
        resourceGroupName, processorJobName, {
            template: processorJobTemplate,
        }
    )
}

The job uses the order ID as an environment variable. To set the environment variable, start the job execution with a modified template that includes the order ID.

Use managed identities for authentication with both the Azure Container Apps Management SDK and the Cosmos DB SDK.

GET /order/status/:orderID

The previous endpoint returns: 202 Accepted With status code Location A header pointing to this status endpoint. Clients can poll this endpoint to check the order status.

This request handler retrieves the order from Cosmos DB. If the order is still pending, it returns: 200 OK With status code Retry-After A header indicating when the client should poll again. When the order is completed, it returns: 303 See Other With status code Location A header pointing to the results.

fastify.get('/orders/status/:orderId', async (request, reply) => {
    const { orderId } = request.params

    // get the order from Cosmos DB
    const { resource: item } = await container.item(orderId, orderId).read()

    if (item === undefined) {
        reply.code(404).send()
        return
    }

    if (item.status === 'pending') {
        reply.code(200).headers({
            'Retry-After': 10,
        }).send({ status: item.status })
    } else {
        reply.code(303).header('Location', '/orders/' + orderId).send()
    }
})

GET /order/:orderID

This endpoint returns the order processing results. Once the order is complete, the status endpoint redirects to this resource. It retrieves and returns the order from Cosmos DB.

fastify.get('/orders/:orderId', async (request, reply) => {
    const { orderId } = request.params

    // get the order from Cosmos DB
    const { resource: item } = await container.item(orderId, orderId).read()

    if (item === undefined || item.status === 'pending') {
        reply.code(404).send()
        return
    }

    if (item.status === 'completed') {
        reply.code(200).send({ id: item.id, status: item.status, order: item.order })
    } else if (item.status === 'failed') {
        reply.code(500).send({ id: item.id, status: item.status, error: item.error })
    }
})

Order Processing Tasks

The order processor job is another Node.js app. Since this is just a demo, it will wait a while, update the order status in Cosmos DB, and exit. In a real scenario, the job could process the order, update the order status, and send a notification.

Deploy as a task in Azure Container Apps. POST /orders The above endpoint starts a job execution. The job updates the order status in Cosmos DB using the order ID as an environment variable.

Like the API app, this operation authenticates to Azure Cosmos DB using a managed identity.

The code is the same GitHub Repository.

import { DefaultAzureCredential } from '@azure/identity'
import { CosmosClient } from '@azure/cosmos'

const credential = new DefaultAzureCredential()
const client = new CosmosClient({ 
    endpoint: process.env.COSMOSDB_ENDPOINT, 
    aadCredentials: credential
})

const database = client.database('async-api')
const container = database.container('statuses')

const orderId = process.env.ORDER_ID

const orderItem = await container.item(orderId, orderId).read()
const orderResource = orderItem.resource

if (orderResource === undefined) {
    console.error('Order not found')
    process.exit(1)
}

// simulate processing time
const orderProcessingTime = Math.floor(Math.random() * 30000)
console.log(`Processing order ${orderId} for ${orderProcessingTime}ms`)
await new Promise(resolve => setTimeout(resolve, orderProcessingTime))

// update order status in Cosmos DB
orderResource.status="completed"
orderResource.order.completedAt = new Date().toISOString()
await orderItem.item.replace(orderResource)
console.log(`Order ${orderId} processed`)

HTTP Client

To call an API and wait for the result, here’s a simple JavaScript function that works like this: fetch However, it waits for the task to complete. It also accepts a callback function that is called whenever the status endpoint is polled, allowing you to record the status or update your UI.

async function fetchAndWait() {
    const input = arguments[0]
    let init = arguments[1]
    let onStatusPoll = arguments[2]
    // if arguments[1] is not a function
    if (typeof init === 'function') {
        init = undefined
        onStatusPoll = arguments[1]
    }
    onStatusPoll = onStatusPoll || (async () => {})

    // make the initial request
    const response = await fetch(input, init)

    if (response.status !== 202) {
        throw new Error(`Something went wrong\nResponse: ${await response.text()}\n`)
    }

    const responseOrigin = new URL(response.url).origin
    let statusLocation = response.headers.get('Location')
    // if the Location header is not an absolute URL, construct it
    statusLocation = new URL(statusLocation, responseOrigin).href

    // poll the status endpoint until it's redirected to the final result
    while (true) {
        const response = await fetch(statusLocation, {
            redirect: 'follow'
        })

        if (response.status !== 200 && !response.redirected) {
            const data = await response.json()
            throw new Error(`Something went wrong\nResponse: ${JSON.stringify(data, null, 2)}\n`)
        }

        // redirected, return final result and stop polling
        if (response.redirected) {
            const data = await response.json()
            return data
        }

        // the Retry-After header indicates how long to wait before polling again
        const retryAfter = parseInt(response.headers.get('Retry-After')) || 10

        // call the onStatusPoll callback so we can log the status or update the UI
        await onStatusPoll({
            response,
            retryAfter,
        })

        await new Promise(resolve => setTimeout(resolve, retryAfter * 1000))
    }
}

To use this function, call it like this: fetch. Pass an additional argument, which is a callback function that is called whenever the status endpoint is polled.

const order = await fetchAndWait('/orders', {
    method: 'POST',
    headers: {
        'Content-Type': 'application/json'
    },
    body: JSON.stringify({
        "customer": "Contoso",
        "items": [
            {
                "name": "Apple",
                "quantity": 5
            },
            {
                "name": "Banana",
                "quantity": 3
            },
        ],
    })
}, async ({ response, retryAfter }) => {
    const { status } = await response.json()
    const requestUrl = response.url
    messagesDiv.innerHTML += `Order status: ${status}; retrying in ${retryAfter} seconds (${requestUrl})\n`
})

// display the final result
document.querySelector('#order').innerHTML = JSON.stringify(order, null, 2)

If you run this in your browser, you can open the developer tools and see all the HTTP requests that are being made.

The portal also provides a view of the job execution history.

conclusion

The asynchronous request-response pattern lets you build robust, scalable HTTP APIs that handle long-running tasks. Azure Container Apps Tasks lets you offload processing to task execution that doesn’t consume resources in your API app. This powerful approach allows your API to respond quickly and handle many requests simultaneously.