How to Choose the Right Models for Your Apps

-Gen AI has forever changed the way we interact with apps and data. So how do you find and integrate the right AI model for your app? In fact, integrating Gen AI into your app development process can make your apps smarter, more efficient, and more user-friendly. Responses to user input are more personalized, engaging, and natural. Gen AI models can infer large amounts of data and interactions, making it easier to scale apps to accommodate the needs of a growing user base without compromising app performance. Additionally, combining and coordinating multiple AI models with different functional components of your app makes it easy to automate repetitive tasks and processes. Over the next few minutes, we’ll walk you through how to choose the right AI model for your app, and as part of that, we’ll explain how to compare models, as well as options for deploying and minimizing inference costs, all in the studio. This applies not only to Azure AI, but also to your code. First, let’s look at model selection.

-There are now more choices than ever before for integrating different classes of AI models into your apps. From large-scale language models capable of sophisticated inference based on vast open knowledge of billions or even trillions of parameters hosted on the Azure supercomputer infrastructure, to powerful quantized small-scale language models like Phi that can also run locally and offline. All about Microsoft’s model family. The studio provides an ever-expanding, central location for you to choose the best AI models as you develop your apps. Azure AI’s model catalog currently hosts more than 1,700 models, including premium models and hundreds of public models organized into collections. There are also large regional language models, such as Core42 JAIS, which supports Arabic spoken languages, or Mistral Large, which focuses on European spoken languages. All models available on Azure have been tested to meet Microsoft’s rigorous security and compliance standards. More information about this can be found at aka.ms/AzureAImodelcontrols.

-We also use a hidden layer model scanner to scan models for embedded malware, backdoors, common vulnerabilities and exposures to detect tampering and compromise across model layers before they are hosted in Azure. Importantly, your options with Azure AI services extend to how you can access these models with your favorite tools and languages ​​through the Azure AI Model Inference API. This API performs well because it works across all models using a unified API schema. Switching between models is easy. It is also integrated with LLM app development tools such as LangChain, Semantic Kernel, Azure AI Prompt Flow, and more. You can also choose an embedding model for vector generation, such as OpenAI or ADA from Cohere.

-Next, since you have a lot of choices, let’s choose an AI model that suits your needs. It’s important here to clearly define your app’s use case and the specific tasks it needs to perform. Here in Azure AI Studio, you can start by filtering your model with an inference task. For example, if natural language processing is a priority for a task like completing a chat, you might see recommendations for models like OpenAI ChatGPT, various SLMs from Microsoft like the Phi model, or models like Meta’s Llama or Mistral as options. For audio-centric tasks like speech recognition or generating speech from text, you might consider OpenAI Whisper, and for computer vision tasks like converting text to images to generate contextual images from text prompts, you might consider DALL-E 3 and Stability AI appears as a potential option.

-Now, if you need higher precision and domain knowledge, you can proactively find ready-made models here, such as Nixtla’s TimeGEN model for time series forecasting and anomaly detection. Additionally, if you and your team have the expertise, you can start with a basic base model and fine-tune the model you want right in Azure AI. That said, ultimately the most important thing on everyone’s mind is cost. Here you can choose from hundreds of free, open models to optimize your app budget, and even if you start there, you can move to more performant models as needed. This is where serverless comes into play through the as-a-service model. An API option that provides serverless pricing for dozens of base models with paid inference input and output tokens that literally pay as you go.

-Or you can choose to run hundreds of open models on hosted hardware using pay-per-GPU managed computing. The trade-offs, depending on your use case, lie in the quality of the model and the sophistication of the model itself, as well as their impact on the cost of inference. As you start building your app prototype, the good news is that Studio in the Azure AI service makes it really easy to make the decision to choose the right AI model for your app. One way is to choose one and experiment. Of course, this requires an Azure subscription and access to Azure AI, and if you choose to deploy, your Azure subscription will be associated with the Azure Marketplace and you may be billed for its use.

-Here, Studio Playground makes it easy to test your deployed model by writing system messages that tell the model the purpose and style of the response, and you can use sample prompts to test the output based on that model. Open world knowledge. You can also continue this experiment by adding your own data and testing the model response in the context of your data. However, prompts and completions are not shared with the model provider or used to train the model; they are private data. This means you will likely want to compare multiple models, and this is where model benchmarks come into play. For example, if you want to build an app primarily for completing chats, you can filter the list in the model catalog and then move on to benchmarking models scored based on multiple industry datasets to classify each model across multiple categories. The first is model accuracy. That’s literally it. The middle line is based on the average of various benchmarks.

-Next, model consistency evaluates how well the model produces smooth, natural responses, while model basis looks at how well the model references the source material in the underlying training set. Model fluency measures the language proficiency of the answer, model relevance scores how well the model meets expectations based on the prompt, and model similarity measures the similarity between the source data sentences and the generated response. For example, if you want to optimize model consistency, you can also select the Meta Llama 3.1 model and learn more about the model and pricing.

-You can also apply model benchmarks to select an embedding model to generate vectors. Here, the data is given numeric coordinate values, allowing similar terms to be mapped based on contextual similarity. It is then used in vector-based search to retrieve the underlying data for the model in Retrieval Augmented Generation, and the most common data is also compared in model benchmarks, which determine which embedding model is best in categories such as classification and clustering. You can check if it works. , and more.

-In addition to the studio, you can also compare different models using the Azure AI model inference package, allowing you to test models on your own data in your preferred coding environment. The only difference is the target URI and key on the endpoint, which makes switching between models easy. For example, here we set up three different notebooks using three different models to test answers generated from custom data. Running them all at the same time with the same prompt can help provide like-for-like comparisons, and once you’ve selected a model and run the app, you can continue to evaluate how well it performs in your code. There are built-in evaluators for relevance, fluency, consistency and fundamentals, each scored on a scale of 1 to 5, with scores averaged over several runs. Additionally, the Application Insights dashboard allows you to visualize model performance and other key metrics across multiple runs over time, including detailed evaluation score trends and token usage per model over time. This can help you evaluate costs along with the model period. This is useful when testing multiple models.

-Now you know the essential steps to evaluate AI models based on your use case, from initial considerations to comparing models to exploring deployment options with Azure AI. In addition to the model selection we provide, you can also benefit from responsible AI control through content filters that operate on prompted input and generated response output. Overall, the Azure platform provides the scalability, intelligence, and security needed for Gen AI apps. We provide. It includes extensive global data center coverage, seamless integration with other Microsoft products, and of course one of the most comprehensive suites of AI and machine learning tools.

-To learn more and get started, check out ai.azure.com and subscribe to Microsoft Mechanics for additional commentary and technical updates. Thank you for watching.