What’s new in FinOps toolkit 0.5 – August 2024

Whether you’re a FinOps practitioner, someone passionate about driving cloud efficiency and maximizing the value you get from the cloud, or someone who’s been asked to look at ways to reduce costs, there’s something for you in your FinOps toolkit. In August, we added Power BI report support to Cost Management exports without deploying the FinOps hub. We’ve expanded optimization options in workbooks. We’ve improved optimization, security, and resiliency in Azure Optimization Engine. We’ve added a new FOCUS document to help you compare against actual/amortized data, and there are a number of smaller fixes and improvements across the board.

If you missed last month’s update, here’s what you need to know: July Updatedo!

New to the FinOps Toolkit?

You may not know it yet, but the FinOps Toolkit is a collection of open source tools and resources to help you learn, adopt, and implement FinOps in the Microsoft Cloud. The toolkit is based on the FinOps Implementation Guide, which helps you get started with FinOps, whether you use the basic tools in the Azure Portal, find ways to automate and extend those tools, or build your own FinOps tools and reports. To learn more about the toolkit, how to provide feedback, or how to contribute, visit the FinOps Toolkit site.

Comparing FOCUS and Actual/Amortized Data

As people move from the actual and depreciated cost dataset to the FinOps Open Cost and Usage Specification (FOCUS), we often get questions about how one column maps to the other. We provide scenario-based documentation to help. Update your existing reports to use FOCUS. likewise How to convert actual/depreciated data to FOCUSHowever, we occasionally get questions about how to compare specific numbers in FOCUS to actual or amortized cost data sets. To facilitate this, we have introduced a new article to help you compare FOCUS data to actual or amortized cost data that you can view in the Azure portal or other tools.

This document provides one table for comparison with actual cost data and another table for comparison with depreciated cost data. Most of the columns are the same, but the difference is in the difference between effective cost after purchases and refunds and commitment discount purchases are depreciated.

Check out the full article for more details. FOCUS Data Verification.

Screenshot of a document that helps you compare FOCUS cost data to actual cost data and depreciated cost data.

What’s new in Power BI

One of the early goals of the FinOps Toolkit was to provide a scalable solution for cost reporting. This is one of the reasons why the FinOps Hub was created. Of course, the FinOps Hub has a broader vision beyond cost reporting, which is not something that everyone needs. For this reason, we continued to include Power BI reports that work with the Cost Management connector. However, this connector is currently in maintenance support and will no longer receive updates due to inherent limitations in how the connector works. Going forward, the new recommendation for cost reporting in Power BI is to export the data to a storage account and leverage the Azure Data Lake Storage Gen2 connector. Additionally, with the ability to overwrite previous monthly reports, Cost Management exports now provide everything you need to report costs directly on top of the raw Cost Management exports. With this, one of the key features we included in the FinOps Toolkit 0.5 release was to support Power BI reports that are linked to raw exports in storage without using the FinOps Hub.

While there are several reasons to use the FinOps Hub, including managed exports, multi-tenant support, and a rich roadmap covering many FinOps capabilities, most organizations will find that connecting Power BI reports directly to exported datasets in their repository is the simplest option to get started with Power BI.

To connect a Power BI report to storage, simply copy the Data Lake storage endpoint from the Azure portal and paste it into the “Export storage URL” parameter in Power BI. Add the target container and folder path to simplify data refresh, and close and apply the changes.

Uncompressed exports have additional size limitations that can result in timeouts or errors during data refresh. For best performance, we recommend using compressed Parquet exports whenever possible.

If you are familiar with the FinOps Hub, you will notice that there are two storage parameters, one for the hub and one for the export. This is to support exporting cost details and other data sets that are not yet supported by the FinOps Hub. You can use both of them as per your needs. Just set the hub storage URL to the ingestion container and the export storage URL to the container where the other exports are published. The only important thing to remember is that the raw export should overwrite the previous export so that data is not duplicated. If you find that you have more costs and usage than expected, check if the “Overwrite data” setting is turned on in the export properties and if not, enable it.

We have removed the Cost Management connector for Power BI, with additional support for raw exports that include all Cost Management datasets (i.e. reservation recommendations included in the Rate Optimization report). To view reservation recommendations, you must create an export and set the appropriate export storage URL in Power BI.

In addition to supporting raw export functionality in Power BI, the following other updates have been made to Power BI reports:

• The list cost and contract cost figures have been updated to facilitate partial (not complete) cost reduction calculations. It is important to note that this is not a complete cost reduction. Support for the price list dataset will be included in a future release to fully backfill the missing price/cost data.
• We’ve made some minor updates to the Cost Summary DQ page to identify rows where the Unique ID cannot be identified and rows where the billing and pricing currencies are different.
• We have merged the Shared and Single Reservation Recommendations pages in the Rate Optimization report into a unified Reservation Recommendations page. This requires you to export reservation recommendations from Cost Management to view the data.
• Fixed a bug in Cost Management exports where committed usage was being displayed as a “Standard” price category. This could result in older reports being misclassified. To correct the underlying data, re-export FOCUS 1.0 data sets exported before August 1, 2024.

Let us know what you want to see next. Power BI reports are one of the most used tools in the FinOps toolkit, but also one we hear about the least. We want to know what you can do to streamline reporting, deployment, and overall management of your FinOps solutions, not just Power BI reports.

What’s New in the FinOps Workbook

Each month, we look at ways to improve the FinOps workbooks to make your cloud environment more optimized and easier to manage. In August, we added new queries and improved usability in both workbooks.

at Optimization Workbook:

• New feature: Identify VMs by processor architecture type.
• New feature: Find SQL elastic pool instances without databases.
• New feature: Detect premium disks for non-running VMs.
• New feature: Export
• UPDATE: Identify the point where the reservation is interrupted event occurs.
• UPDATE: Azure Hybrid Benefit core count for Virtual Machine Scale Sets has been revised.
• Update: Idle disk queries now ignore disks used by AKS Pods.
• Update: Excluded Block Blob storage accounts from the v1 storage account list.
• Update: Management Group filter has been removed to simplify subscription filtering.

at Governance Workbook:

• Update: Management Group filter has been removed to simplify subscription filtering.

We evaluate changes to the workbook monthly. Please let me know What do you want to see next!

What’s new in Azure Optimization Engine

August brings improvements to recommendations, security, and flexibility to the Azure Optimization Engine (AOE).

First, we Improved Unused Premium Disk Recommendations Provides more accurate and actionable insights, including updated meter names in cost data and support for ZRS disks that provide improved performance and reliability. These changes help you balance resiliency and maximize the value you get from your cloud solutions.

Next, we updated our default security settings, considering how important security is to cloud solutions. Enable Microsoft Entra ID authentication for SQL databases only.. This update removes the password from the initial deployment, making it easier and more secure.

Finally, we have also expanded our support. Add additional Entra ID tenants (or directory) options added. Collect Microsoft Customer Agreement (MCA) Billing ProfilesFixed a bug related to MCA subscription cost data. Use the new Register-MultitenantAutomationSchedules PowerShell script to configure additional tenants.

We hope these improvements will help you maximize efficiency, security, and resiliency in your cloud solutions. We are always looking for new opportunities like this, so let us know what you would like to see next!

Other new and notable updates

Each release includes many small improvements and bug fixes, so it would be difficult to cover everything in detail, but I wanted to point out a few other small things that you might be interested in.

~ inside FinOps Hub:

• We’ve expanded our cost estimation article to include Power BI pricing and a link to a pricing calculator.
• Various minor bug fixes and improvements.

at FinOps Toolkit PowerShell Module:

• Added more filtering options to Get-FinOpsCostExport.
• Various minor bug fixes and improvements.

~ inside Open data:

• 16 new region values ​​have been added to remap existing regions.
• Four new pricing units added and one updated.
• 48 new resource types added and 32 updated.
• 20 new service mappings added and 8 updated.

For more information, see: FinOps Toolkit Change Log.

What’s next

Here are some things we’ll be looking at over the next few months:

• Power BI reports will be expanded with more FinOps capabilities, especially new governance and workload optimization reports coming soon.
• The FinOps hub will add support for more datasets, private endpoints, and large datasets in Azure Data Explorer.
• The FinOps integration documentation is constantly being updated, expanded with more FinOps features, and added costs in the FinOps hub.
• Azure Optimization Engine will continue to receive minor updates as we plan the next major release of the tool.
• In each release, we try to choose at least one of the following: Most voted issues (Based on 👍 Voting) will continue to evolve based on your feedback, so please keep sending us your feedback!

For more details, please see: FinOps Toolkit RoadmapAnd please Let us know If there’s something you’d like to see in future releases, whether you’re using our base product, automating and extending it, or using a custom solution, we’re here to help you adopt and implement FinOps more easily.