Baseline Agentic AI Systems Architecture

Agent AI System It is designed to solve complex problems with limited direct human supervision. [1]. These systems consist of multiple conversational agents that converse with each other and can be centrally coordinated or self-organizing in a distributed manner. [1, 2]As enterprises increasingly use multi-agent systems to automate complex processes or solve complex tasks, we want to take a closer look at how these systems might be architected.

These agents have the following features: planIt allows you to predict future states and select optimal actions to achieve specific goals. It also integrates. MemoryIt allows the agent to recall past interactions, experiences, and knowledge. This is important for maintaining continuity of work and improving strategies. Agents can also equipmentRun code, query databases, and interact with other systems, including APIs and external software. [1, 3]. This tool expands your capabilities and allows you to perform a wider range of tasks.

Agents can perform tasks and write and execute code, so they have the potential to execute code that may be malicious or harmful to the host system or other users. [3]Therefore, understanding the architecture of these systems is critical to sandboxing code execution, limiting or denying access to production data and services, and mitigating failures, vulnerabilities, and abuse.

Many components of this proposed architecture are similar to the baseline OpenAI end-to-end chat reference architecture. [4] The main components of Agentic AI Systems are Azure OpenAI, Azure AI Search, and Azure AI Services. We will highlight the main components of this architecture.

• Azure AI Studio [5] A managed cloud service used to train, deploy, automate, and manage machine learning models, including large-scale language models (LLMs), small-scale language models (SLMs), and multimodal models used by agents. The platform provides a comprehensive set of tools and services that facilitate the end-to-end machine learning lifecycle. Key features of Azure AI Studio include:

  • Prompt Flow [6] A development tool designed to simplify the entire development lifecycle of generative AI applications. You can create, test, and deploy prompt flows, which can be used to generate responses or actions based on given prompts. These prompt flows can be deployed to a Machine Learning Workspace or containerized and deployed to Azure Container Apps or Azure Kubernetes Services. [7]AI Studio also lets you develop and deploy these instant flows.

  • Managed Online Endpoints Used to invoke prompt flows for real-time inference from agents and backend services. Provides a scalable, reliable, and secure endpoint to deploy machine learning models and enable real-time decision making and interactions. [7].

  • Azure AI Dependencies Contains essential Azure services and resources that support the functionality of AI Studio and related projects. [8]:

    • Azure Storage Account Stores artifacts of the project, such as prompt flows and assessment data. Primarily used to manage data and model assets in AI Studio.
    • Azure AI SearchA comprehensive cloud search service that supports full-text search, semantic search, vector search, and hybrid search. It provides search capabilities for AI projects and agents and is essential for implementing the Augmented Search Generation (RAG) pattern. This pattern involves extracting relevant queries from a prompt, querying an AI search service, and using the results to generate a response using an LLM or SLM model.
    • Azure Key Vault Used to securely store and manage secrets, keys, and certificates required by agents, AI projects, and backend services.
    • Azure Container Registry Stores and manages container images for agents, backend APIs, orchestrators, and other components. It also stores images generated when using custom runtimes for prompt flows.
    • Azure OpenAI Service Supports natural language processing tasks such as text generation, summarization, and conversation.
    • Azure AI Services Provides APIs for vision, speech, language, and decision making, including custom models.
      • Document Intelligence Extract data from documents and process it intelligently.
      • Azure Voice It converts speech to text, text to speech, and also has translation capabilities.

      • Content AI Safety Ensure AI-generated content is ethical and safe, preventing the creation or spread of harmful or biased material.

  These components and services provided by Azure AI Studio facilitate the development and operation of Agentic AI Systems by enabling seamless integration, deployment, and management of sophisticated AI solutions.