Azure App Service Linux WebJobs: A Comprehensive Guide

introduction

WebJobs is a feature of Azure App Service that allows you to run background tasks or scripts in App Service. It is particularly useful for running tasks such as scheduled jobs, data processing, or background maintenance tasks. WebJobs can run continuously, on demand, or on a scheduled basis, and can be written in a variety of programming languages.

The purpose of this blog post is to walk you through the process of setting up and deploying a WebJob on Azure App Service running Linux. We will walk you through the steps required to create and deploy a WebJob, configure settings, and manage it effectively. By the end of this blog, you will have a working WebJob integrated with Azure App Service that can perform background tasks or process tasks as needed.

Prerequisites

• Azure Subscription: You need an active Azure subscription to create and manage Azure resources.
• Linux App Service: App Service is required to deploy WebJobs. To run a WebJob, the App Service stack must match the WebJob. For example, a Python-based WebJob requires a Python App Service stack.
• Integrated Development Environment (IDE): An IDE or text editor that you can use to develop WebJobs.
• File Compression Tool: Packages WebJob files into ZIP archives for deployment.

Types of web jobs

When choosing which type of WebJob to use in Azure App Service, it is important to consider the nature of the work you need to do. Ongoing webjob Designed to run continuously in the background, they are ideal for tasks that need to be constantly active. These WebJobs are ideal for ongoing tasks such as background processing, monitoring, or maintaining a steady stream of jobs. They remain active as long as the App Service is running, and are automatically restarted if something goes wrong or stops unexpectedly.

In contrast, Triggered Web Job They are suitable for tasks that need to run at predefined times or respond to events, as they are executed based on a specific trigger or schedule. For example, you can use a Triggered WebJob to respond to specific tasks, such as scheduled data updates, batch processing, or the arrival of new data. Choosing between a Continuous or Triggered WebJob depends on whether your job requires continuous operation or is event-driven, allowing you to tailor the WebJob type to your specific needs.

Create a web job

In this section, we will create a Python WebJob that reads and updates data from a CSV file. This example will help you understand the basic structure of a WebJob. Below is an edited version of my webjob.py.

import pandas as pd
import os, datetime, logging

# Configure logging
logging.basicConfig(
    filename="/home/LogFiles/webjob.log",
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s"
)


def insert_record(file_path):
    print('--- Inserting Record ---')

    try:
        # Determine the next value
        value = get_next_value(file_path)
        
        # Create a new DataFrame with the new record
        new_record = pd.DataFrame({'Timestamp': [datetime.now()], 'Value': [value]})
        
        if os.path.exists(file_path):
            # Load the existing CSV file
            df = pd.read_csv(file_path)
            logging.info('Loaded existing CSV file successfully.')
            # Append the new record
            df = pd.concat([df, new_record], ignore_index=True)
        else:
            # If the file does not exist, create a new DataFrame
            df = new_record
            logging.info('CSV  file not found. Created new file with columns: Timestamp, Value.')
        
        # Save the updated DataFrame back to Excel
        df.to_csv(file_path, index=False)
        logging.info('Saved updated CSV file successfully with value: %d', value)
    
    except Exception as e:
        logging.error('Error processing record: %s', e)

if __name__ == "__main__":
    print('--- WebJob Execution Started ---')
    file_path="/home/site/wwwroot/data.csv"
    insert_record(file_path)
    print('--- WebJob Execution Ended ---')

Webjob distribution

To deploy a WebJob to App Service Linux, you need to create a shell script, package it, and then deploy it through the Azure Portal.

Create a shell script (script.sh) with the command to start the WebJob script. For Python WebJob, use the script below to install dependencies and run the script.

#!/bin/bash
/opt/python/3.10.14/bin/pip3 install pandas
/opt/python/3.10.14/bin/python3.10 webjob.py

Zip the webjob.py and script.sh files together. Make sure the files are in the root of the compressed archive.

Webjob distribution requires basic authentication. Enable SCM Basic Authentication Posting Credentials In the configuration blade of the app service.

Now go to the WebJob Balde in App Service to add and configure a WebJob. Click the “Add” button to create a new WebJob. In the configuration panel:

• WebJob File Upload: Select the prepared webjob.zip file.
• Setting the WebJob type: Select the WebJob type. Select “Continuous” for a job that runs continuously, or “Triggered” for a job that runs on demand or on a schedule.
• Schedule configuration (If using Triggered WebJobs): Specify a cron expression for scheduling the job. For example, to run every 5 minutes, use 0 */5 * * *.
• Click “OK” or “Save” to upload the WebJob and apply the configuration.

Testing and Logging

For triggered web tasks, you can manually run them by selecting the web task and clicking “Run”.

On App Service Linux, WebJobs are deployed to /tmp/jobs/. To check the logs, you can click the Logs button to view them. You can navigate to /home/data/jobs/triggered/.. Here you can find a directory corresponding to each execution of the WebJob. Inside each directory you can find the files ‘status’ and ‘output_log.txt.’

The status file represents the schedule set for the WebJob along with the results of the WebJob execution. You can also find the start and end times of the execution.

The output_log.txt file contains general information and status updates during the run. It also contains a run log.

Reviewing these logs can help you determine the operational status of your WebJob, diagnose problems, and understand execution behavior.

conclusion

WebJobs on Azure App Service Linux provide a powerful way to run background tasks and automate processes within your applications. Following the steps outlined will help you seamlessly integrate WebJobs into your Linux-based App Service. With a strong understanding of WebJobs, you can leverage this functionality to enhance the functionality of your applications and maintain high performance across your services.

Additional Links

Running background tasks with WebJobs – Azure App Service | Microsoft Learn

Github Repository: App Service Linux WebJob (github.com)