Building an AI Dev Space With a Little Assistance from Aspire

@code {
    public class chatDialog
    {
        public string? systemMessage;
        public string? inputText;
        public string? outputText;
        public int maxTokens = 400;
        public float temperature = 0.7f;
    }

    //This view is hardwired to use the simulator so we can adjust accordingly
    private string oaiEndpoint = string.Empty;
    private string oaiDeploymentName = string.Empty;
    private string oaiKey = string.Empty;
    public static chatDialog dialog = new();

    protected override void OnInitialized()
    {
        oaiEndpoint         = "http://localhost:8000";
        oaiDeploymentName   = Configuration["oaiDeploymentName"] ?? "gpt-4o";
        oaiKey              = Configuration["oaiKey"] ?? string.Empty;

        dialog = new()
        {
            systemMessage = "I am a hiking enthusiast named Forest who helps people discover hikes in their area. If no area is specified, I will default to near Rainier National Park. I will then provide three suggestions for nearby hikes that vary in length. I will also share an interesting fact about the local nature on the hikes when making a recommendation.",
            inputText = "Can you recommend some good hikes in the Redmond area?",
            outputText = string.Empty,
            temperature = 0.7f,
            maxTokens = 400,
        };
    }

    protected async Task chat()
    {     
        AzureOpenAIClient client = new AzureOpenAIClient(new Uri(oaiEndpoint), new System.ClientModel.ApiKeyCredential(oaiKey));

        OpenAI.Chat.ChatClient chatClient = client.GetChatClient(oaiDeploymentName);

        OpenAI.Chat.ChatCompletionOptions chatCompletionOptions = new()
        {
            MaxOutputTokenCount = dialog.maxTokens,
            Temperature = dialog.temperature,
        };

        OpenAI.Chat.ChatCompletion completion = await chatClient.CompleteChatAsync(
        [
            new OpenAI.Chat.SystemChatMessage(dialog.systemMessage),
            new OpenAI.Chat.UserChatMessage(dialog.inputText),
        ],chatCompletionOptions);

        var response = $"Response:\r\n{completion.Content[0].Text} \r\nOutput tokens: {completion.Usage.OutputTokenCount}\r\nTotal tokens: {completion.Usage.TotalTokenCount}";

        dialog.outputText = response;
    }
}
​

All code for this post can be found here.

It seems like you’re hardcoding localhost. It’s a bad idea and kind of non-distributable. I know and that’s on purpose. here. We tested pushing the simulator to Azure Container Registry and pulling it to Azure Container App. That’s not a problem at all, (localhost is also invalid) but that will be I need some additional logic Create deployment script So I chose making things are simpler instead.

The following code adds: simulator instance Web app (to AppHost program.cs😞

builder.AddDockerfile("aoai-simulator-generate", "../AOAI_API_Simulator")
    .WithHttpEndpoint(port: 8000, targetPort:oaiSimulatorPort)
    .WithEnvironment("SIMULATOR_MODE", "generate")
    .WithEnvironment("SIMULATOR_API_KEY", localOaiKey)
    .ExcludeFromManifest();​

The results are ridiculous Practice SDK, perform load testing and Experiment with UI that works. (As with actual AI deployments, creation may be delayed.)

You probably want to look past the fluff and get real reactions. The simulator also has a mod for this. You can insert values ​​for your Azure OpenAI deployment. Then enter ‘recording’ mode. Your question will then be proxied and the response will be saved to disk as a JSON file.

If you add a few lines of code, Aspire can handle this part as well. Create both an Azure OpenAI resource and another instance of the simulator.

var azaoai = builder.AddBicepTemplate(
    name: "AI",
    bicepFile: "../infra/ai.bicep")
    .WithParameter(AzureBicepResource.KnownParameters.KeyVaultName);

var cloudEndpoint   = azaoai.GetOutput("endpoint");
var accountName     = azaoai.GetOutput("accountName");
var cloudKey        = azaoai.GetSecretOutput("accountKey");
var cloudDeployment = "gpt-4o";

builder.AddDockerfile("aoai-simulator-record", "../AOAI_API_Simulator")
    .WithBindMount("recordings", "/app/.recording")
    .WithHttpEndpoint(port: 8001, targetPort: oaiSimulatorPort)
    .WithEnvironment("SIMULATOR_API_KEY", localOaiKey)
    .WithEnvironment("SIMULATOR_MODE", "record")
    .WithEnvironment("AZURE_OPENAI_ENDPOINT", cloudEndpoint)
    .WithEnvironment("AZURE_OPENAI_KEY", cloudKey)
    .WithEnvironment("AZURE_OPENAI_DEPLOYMENT", cloudDeployment)
    .WithEnvironment("AZURE_OPENAI_EMBEDDING_DEPLOYMENT", cloudDeployment)
    .ExcludeFromManifest();​

That will happen if you ask the same question. hopefully Get more meaningful answers.

This also acts as a cache, so repeating the question requires reading from the file. Instead of hitting the endpoint in Azure.

We don’t just record because we can. It also provides a playback mode for recording. So we create another simulator instance pointing to the same directory as the recorder.

builder.AddDockerfile("aoai-simulator-replay", "../AOAI_API_Simulator")
    .WithBindMount("recordings", "/app/.recording")
    .WithHttpEndpoint(port: 8002, targetPort: oaiSimulatorPort)
    .WithEnvironment("SIMULATOR_API_KEY", localOaiKey)
    .WithEnvironment("SIMULATOR_MODE", "replay")

this is not the case It forwards the request to the cloud but only verifies the file locally. This means that asking the same question repeatedly will give you the exact same answer, but the results will be different. action Rather than asking an AI assistant There is an advantage to being More predictable in development and testing. For that matter, you can edit the JSON to change the answer, add a question, and more. If you are asking a new question (No recorded responses) Since we can’t return anything meaningful, we just catch the exception and take a shortcut to insert the placeholder text.