New Custom and Managed Python images in Azure Data Explorer

Azure Data Explorer (ADX) supports executing Python code embedded in KQL queries using: python() plugin . The plugin runtime is hosted in a sandbox, an isolated and secure environment hosted on ADX’s existing compute nodes. This sandbox contains the language engine and common math and science packages. The plugin extends the KQL base functionality with a vast archive of OSS packages, allowing ADX users to run advanced algorithms such as: Machine Learning, Statistical Tests, Time series analysis And there’s more as part of a KQL query.

We’re constantly working to improve the Python plugin capabilities, and today we’re introducing new managed Python images and the option to fully customize your Python images to include the Python packages you need.

• Managed Images A Python environment built and maintained by the Kusto team, containing a specific Python engine and set of packages. We’re adding more today. Python 3.11.7 and Python 3.11.7 DL (Includes Torch and TensorFlow, and both images include the latest Python engine 3.11.7 and packages (you can review the full contents of the images by clicking on the links).
• Custom Images If you need additional packages or different versions of the Python engine and/or packages, you can create a specific Python image. You can create a custom image from scratch by specifying the Python engine and providing a requirements text file with a full list of packages, or by selecting a base existing image and providing a minimum requirements file with only the additional packages to be installed on top of the base image.

Defining a custom Python image in ADX portaL

For more information, see: Create a custom image – Azure Data Explorer | Microsoft Learn and Python Plugin Package – Kusto | Microsoft Learn

Try it out and share your feedback!