The Future of AI Is: Model Choice

When I was first asked to think about what the future of AI might look like for developers, my answer was instinctive. You should start with model selection! Today I would like to explore this topic in more detail. This is the first of a multi-part series I want to take you through. From catalog to code to cloudBuild intelligent applications on Azure AI.

paradox of choice

Language models are the core of generative AI applications. The model we choose affects the quality of the application response and the cost of developing the solution. Our decisions can help us extend our reach to the edges (using small language models) or improve the precision of special applications (using fine-tuning). Choice makes the difference between a promising prototype and a scaled product. However, choosing a model can be difficult.

2023 thesis “Harnessing the Power of the LLM in Practice: A Survey of ChatGPT and Beyond”) As shown below, we have charted a fairly sparse LLM environment using a small number of model choices (e.g. GPT-4, Llama, and Claude). This allowed us to quickly become productive using model features and APIs and easily choose which one best suited our scenario requirements.

Fast forward to 2024 and we Over 1 million community created variations for Hugging Face The underlying model ecosystem is growing rapidly from providers such as Cohere, Mistral, AI21 Labs, Jais, Nixtla, Google, and Microsoft. we are Small Language Model (SLM) Extend your use of AI apps to the edge Professional models (by domain) Features range from multilingual (Jais) to time series forecasting (Nixtla), healthcare, and more.

This is paradox of choice With so many choices, we become afraid to choose one for fear of missing out on a better option that suits our needs. How do you overcome analysis paralysis? In this blog post, we’ll look at a three-part solution that can help alleviate these problems.

• find: Prevents decision fatigue. Use a structured process to shortlist your selections.
• evaluation: Find the optimal fit. Evaluate final candidates using appropriate tools and metrics.
• development: Make the switch easily. Use model-agnostic APIs when coding your apps.

Any generative AI application design architecture begins with these fundamental questions: What model should I use to make my scenario a reality? As a developer looking at the end-to-end application lifecycle, there are three things to consider when choosing a model:

1. idea -Can I do it? prototype What are my application scenarios using this model?
2. increase -Can I do it? optimize What models improve response quality and safety?
3. operationalization -Can I do it? scale Will you use the model in a production deployment?

Our problem today is The model ecosystem is fragmented. Baseline models are published along with documentation and samples on provider-hosted sites, while community-generated variants are published on model hubs such as Hugging Face. Developers obtain different levels of information from each source. This means the burden of discovery falls on the developer, who must visit different model playgrounds, use different SDKs, and find the right metrics to compare options before choosing one.

this happens decision fatigue Where fear interferes with our ability to make decisions, there may be better choices unknown to us. What we need structured process Model selection helps filter out irrelevant options until you get a manageable subset that can be evaluated against specific application requirements. comprehensive platform, good night Azure AI Studio (See below) We provide the tools and support you need to make the process seamless. Let’s jump in!

Let’s take a look back at the three-part solution we talked about earlier and understand the questions we’re trying to answer and the challenges we face at each step.

• find It’s about find All options to choose from. To do this, you first need: know Once you know that a particular option exists in this growing ecosystem, trust Meets privacy, security, and safety standards for enterprise use.
• evaluation It’s about filtering Make your selections until you have a manageable shortlist that can be evaluated more carefully. For this asking the right questionsIt eliminates unacceptable choices, preferably in the correct order. Identify the right criteria Comparing the remaining model choices will help you zoom in and make your final decision.
• assignment It’s about fitting Apply that model to the scenario’s application architecture. by decoupling In task-based applications, selecting a model allows developers to swap models without having to rewrite code. The model must then be tested so that it can iterate from the initial prompt to a functional prototype.

This is where having a comprehensive platform that streamlines your end-to-end workflow helps! that Azure AI Platform With a wealth of tools, turnkey services, and easy integrations, we provide a one-stop shop to move from model selection to managed solutions in an integrated manner. It provides three main features to meet your model selection requirements:

These features map directly to our needs. find, evaluation & assignment

find: “There is something like this. model For that purpose!”

that Azure AI Model Catalog Contains over 1.7K. Models carefully curated with the help of trusted partners and customer feedback to support enterprise scenarios. Reflecting the growing interest in multilingual capabilities, small language models, and domain specialization, we add up to 20 community-generated variants to the Hugging Face collection each month. Are there specific criteria for app scenarios? We have a model for that!

evaluation: “There is something like this. metric system For that purpose!”

teathat Azure AI model benchmark It helps you evaluate model performance for the following quality metrics: Foundation, consistency, fluency and similarity. With our curated list of foundational LLM and SLM models, compare your options using model- and dataset-level accuracy scores and choose the right criteria for your scenario. Filter views by tasks or other criteria for more effective comparisons. You can also write your own custom estimator to score models based on specific criteria. Do you want a way to choose the best fit among shortlisted models? There are metrics for this!

adjustment: “There is something like this. machine For that purpose!”

that Azure AI Model Inference API Developers can work with a variety of models and providers using a common syntax and language. this You can easily connect other models or replace them later. This abstraction layer is also extensible to support unique model features. It currently works with a core subset of the model (serverless APIs and managed compute), with more features coming in the future. Want to compare different models using one codebase? There’s a mechanism there!

our The journey from catalog to code to cloudLet’s focus on the search process. today Look back at the evaluation and assignment steps in future posts in this series. The pictorial guide below provides a big picture of model selection and highlights the steps you can use to shortlist your catalog options to get a manageable subset of tiles 3-6. Click here to see a high-resolution (downloadable) version of the image.

Start with having a developer ask the right questions For evaluation purposes, reduce your catalog selection to one to three more manageable options.

1. Day: What is the main inference task in your application? Choosing the right action will greatly reduce your model options, making it the perfect first step in filtering. For example, there are many options for “generating text” or “answering questions,” but fewer options for “generating images.” Starting with these filters will help you focus on the right subset to start with.
2. specialization. Does your application require higher precision for a special domain? Do special models exist or can they be trained using data? The first requires searching by keyword or task classification, and the second checks whether there is a fine-tuned model for the target task or a variant created by the community that meets the requirements.
3. performance. Does your application require real-time response (e.g. mobile or edge devices)? Or is the round-trip latency to the cloud an acceptable balance against other constraints? This effectively allows you to choose between SLM and LLM options.
4. pharmaceutical. Are there any associated cost or resource constraints in developing the solution that will affect your decision? Will serverless API deployment (pay per token) work better than managed computing (pay per VM) for your current needs?

The Azure AI Model Catalog allows you to filter models by a variety of criteria, including collection (provider), license (use), distribution (payment option), operation (classification), and keyword (search). The results are laid out like this: model card Provide additional details such as model weights, relevant samples, and evaluation or dataset (in case of variants) used to fine-tune that model. Deploy the model on the card to get an active endpoint and playground that can be used for code-first and low-code verification, respectively.

Want a more rigorous evaluation? In the next post, we’ll look at using the Azure AI Model Benchmark and the Azure AI Model Inference API with real-world application scenarios. For now, check this out. Microsoft Dynamics blog post Learn how to choose the right model for your app or watch the 8-minute video that walks you through the process linked below.