OpenAI Assistants Interactive Visualizations Using Chart.js

Welcome to the next article in the AI ​​Futures blog series!

I am Rob ChambersIn this post, we’ll take you on an exciting journey to power your customers’ interactive visualization solutions. chart.js and Azure AI.

introduction

A customer recently contacted us with an interesting question. How to use AI to generate charts from data? they were at first Azure OpenAI Assistant with code interpreterYou can build and run Python code in real time to create static images. The static image approach works out of the box, but the customer wanted a more interactive solution that would allow them to create more vivid and dynamic charts.

Below is an example of a customer input prompt:

Display this data using a bar-chart:

Label  Brown Pink Yellow  Sum
A        339  433    126  898
B         48  421    222  691

our approach

To address the need for interactive and dynamic charts, I had a few interactive conversations with: GPT-4oExplore different options. In the end, we chose Chart.js, a popular JavaScript library for creating vibrant, responsive charts. Chart.js perfectly matched our customer’s requirements, allowing them to create interactive charts that were fully integrated into their web application.

Once we chose Chart.js, we had to decide how we could combine its functionality with an AI chatbot. We knew that we could use the Azure AI CLI to create basic web pages that already supported markdown and code highlighting. marked and highlight.js. You can simply extend this sample to include Chart.js. We decided to use a Markdown rendering process to support new languages. 'chartjs'It contains a JSON object representing the main parameters of the Chart.js chart. To make this work, we needed to update OpenAI Assistant to understand how to export this new Markdown language. Here are some of ours: .ai/data/assistant.instructions file:

You are a helpful assistant.  

One of your special skills is that you can make charts.  
You can do that because you now support a new markdown language called 'chartjs' which contains a JSON object that represents a Chart.js chart.  

Here's an example of what the markdown for a Chart.js chart looks like:
```chartjs
{
  "type": "bar",
  ...
}

Using the Azure AI CLI, we updated OpenAI Assistant with new instructions and created a simple HTML/JavaScript web application that can be run locally.

ai chat assistant update --instructions "@assistant.instructions"
ai dev new openai-asst-webpage-with-functions --javascript

Edit web application

Now that OpenAI Assistant was ready to use, we needed to update the main web application to support the new features. 'chartjs' Markdown language.

The AI ​​Futures team always strives to make the most of AI in the development process. GitHub Co-Pilot inside VS Code Or run the Azure AI CLI interactively. This time I used the AI ​​CLI to read the existing code, make suggestions and changes, and then save the file back to disk. I used VS Code’s built-in diff viewer to review updates, accepting or rejecting changes as needed. You can view a prompt summarizing the changes you’ve made in your GitHub repository. here.

Here’s how to call the AI ​​CLI to read, understand, and modify your web application:

cd openai-asst-webpage-with-functions-js
ai chat --interactive --built-in-functions --system @SYSTEMPROMPT.md --user @USERPROMPT.md

that '--built-in-functions' Options are the key to giving AI CLI and LLM access to read and write files on disk.

Chart.js integration

Here’s what we (AI CLI and I) did to integrate Chart.js into our web application:

Extension marked to handle chart “Language” block

we are markd A library that handles new language blocks. If the language of the code block is: 'chartjs'Returns a new div with a class. 'chartjs' Here we use Chart.js to render the chart.

marked.setOptions({
  highlight: function (code, lang) {
    if (lang === 'chartjs') {
      let chart = createNewChartJsChart(code);
      return `
${chart}
`;
    }    let hl = lang === undefined || lang === '' ? hljs.highlightAuto(code).value : hljs.highlight(lang, code).value;
    return `
${hl}
`;
  }
});

chart rendering

that createNewChartJsChart The above function parses the JSON, creates the required HTML canvas elements, and postpones populating the chart until the DOM elements are available. Once the DOM elements are ready populateNewChartNow A function is called to render the chart using Chart.js.

function populateNewChartNow(chart) {
  let ctx = document.getElementById(chart.id).getContext('2d');
  new Chart(ctx, {
    type: chart.config.type,
    data: chart.config.data,
    options: chart.config.options
  });
}

Conclusion and Call to Action

The end result is a dynamic and conversational chatbot that uses Chart.js to generate charts from data, making the customer experience more engaging and insightful. Customers can now provide data in prompts and receive interactive visualizations in response.

The journey to integrate Chart.js and enhance AI chatbots demonstrates the power of Azure AI not only in the form of innovative and engaging end-user solutions, but also in the development of those experiences.

Are you ready to transform your AI applications with dynamic charts?

Please check the following link for more details:

https://github.com/robch/openai-asst-webpage-with-functions-and-chart-js

https://github.com/Azure/azure-ai-cli

https://thebookof.ai

https://ai.azure.com