Optimizing Azure Table Storage: Automated Data Clean-up using a PowerShell script with Azure Automat by info.odysseyx@gmail.com September 5, 2024 written by info.odysseyx@gmail.com September 5, 2024 0 comment 8 views 8 script The purpose of this blog is to manage Table Storage data efficiently. Let’s say you have a large Azure Table Storage that accumulates logs of various applications or old, unused data. Over time, this data grows significantly and you need to periodically clean up old items to maintain performance and manage costs. You decided to automate this process using Azure Automation. However, the lifecycle management policy is limited to the Blob service only. Scheduling a PowerShell script allows you to efficiently delete old data from Azure Table Storage without manual intervention. This approach ensures that your storage remains optimized and your applications run smoothly. Here is a PowerShell script to delete table entities based on timestamp: Connect-AzAccount-ID $SubscriptionID = “xxxxxxxxxxxxxxxxxx”$AzureContext = Set-AzContext –Subscription ID $Subscription IDUpdate-AzConfig -DisplaySecretsWarning $false $StorageAccount = “xxxxxxxxxxxxxxxxxx”$StorageAccountKey = “xxxxxxxxxxxxxxxxxx” $ctx = New-AzStorageContext -StorageAccountName $StorageAccount -StorageAccountKey $StorageAccountKey $alltablename = (Get-AzStorageTable –Context $ctx).cloudtable foreach ($table in $alltablename) {$tabledata = Get-AzTableRow -Table $table -CustomFilter “Timestamp gt DateTime ‘YYYY-MM-DDThh:mm:ssZ’ ” | Remove-AzTableRow -Table $table} #disclaimer# The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS, WITHOUT WARRANTY OF ANY KIND. MICROSOFT DISCLAIMS ALL IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK ARISING OUT OF THE USE OR PERFORMANCE OF THE SAMPLE Scripts AND THE DOCUMENTATION REMAINS WITH YOU. IN NO EVENT SHALL MICROSOFT, ITS AUTHORS, THE OWNER OF THIS REPOSITORY, OR ANY OTHER PERSON INVOLVED IN THE CREATION, PRODUCTION, OR DELIVERING OF THE SAMPLE Scripts BE LIABLE FOR ANY DAMAGES WHATSOEVER (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF DATA, #Without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss arising out of the use of or inability to use the sample script or documentation (except for damages for such damages), even if Microsoft has been advised of the possibility of such damages. Here are the steps to schedule a PowerShell script in Azure Automation: Follow the link to create an Azure Automation account.: Add a module to your Azure Automation account: Creating a PowerShell Runbook: Go to your Automation account in the Azure Portal. In “Process Automation”, select “Runbook”. Click “Create Runbook”. Enter a name for the runbook, select “PowerShell” as the runbook type, and click “Create”. Once the runbook is created, on the “Edit PowerShell Runbook” page, Enter your PowerShell script and click “Publish”. Schedule a Runbook: Go to the runbook in question and select the “Link to Schedule” option. Select the option “Link schedule to Runbook” and select the appropriate schedule. The Schedule option allows you to create a new schedule by specifying a name, description, start date, time, time zone, and recurrence information. Runbook monitoring: Runbook execution can be monitored by going to: job Under the section Process Automation In your automation account. Here you can check the status of your runbook jobs, view job details, and troubleshoot issues. memo: Managed identities for Microsoft Entra IDs allow you to securely access other Microsoft Entra protected resources without having to manage credentials in your runbooks. These identities are automatically managed by Azure, so you don’t need to manually provision or rotate secrets. Managed identities are the default authentication method for runbooks and are set as the default for your Automation account, ensuring secure and streamlined access to the resources you need. See: Use system-assigned managed identity for your Azure Automation account | Microsoft Learn These steps will help you schedule PowerShell scripts in Azure Automation. If you have any further questions or need further assistance, feel free to ask! References:- Source link Share 0 FacebookTwitterPinterestEmail info.odysseyx@gmail.com previous post Coming soon: DP-3020: Develop data-driven applications with Azure SQL Database next post Enhancing Data Security and Digital Trust in the Cloud using Azure Apps, Azure Storage and Azure Key Vault. You may also like Copilot for Microsoft Fabric – Starter Series Healthcare Focus September 12, 2024 More ways to sell through the marketplace with professional services September 11, 2024 Two upcoming Copilot and M365 for SMB Community offerings September 11, 2024 Copilot for Microsoft 365 Adoption Trainings September 11, 2024 Omdia’s perspective on Microsoft’s SSE solution September 11, 2024 Extend Viva Connections with pre-built 3rd party Adaptive cards September 11, 2024 Leave a Comment Cancel Reply Save my name, email, and website in this browser for the next time I comment.