.NET: Improve structured output for chat client agent

[SergeyMenshykh]

This PR adds several overloads of the RunAsync<T> methods to ChatClientAgent, improving the development experience when using the agent to produce structured output from non-streaming APIs.

Code before

// Create the agent options, specifying the response format to use a JSON schema based on the PersonInfo class.
ChatClientAgentOptions agentOptions = new(name: "HelpfulAssistant", instructions: "You are a helpful assistant.")
{
    ChatOptions = new()
    {
        ResponseFormat = ChatResponseFormat.ForJsonSchema<PersonInfo>()
    }
};

// Create the agent using Azure OpenAI.
AIAgent agent = new AzureOpenAIClient(
    new Uri(endpoint),
    new AzureCliCredential())
        .GetChatClient(deploymentName)
        .CreateAIAgent(agentOptions);

// Invoke the agent with some unstructured input, to extract the structured information from.
var response = await agent.RunAsync("Please provide information about John Smith, who is a 35-year-old software engineer.");

// Deserialize the response into the PersonInfo class.
var personInfo = response.Deserialize<PersonInfo>(JsonSerializerOptions.Web);

Console.WriteLine("Assistant Output:");
Console.WriteLine($"Name: {personInfo.Name}");
Console.WriteLine($"Age: {personInfo.Age}");
Console.WriteLine($"Occupation: {personInfo.Occupation}");

Code after

// Create the ChatClientAgent with the specified name and instructions.
ChatCilentAgent agent = new AzureOpenAIClient(
    new Uri(endpoint),
    new AzureCliCredential())
        .GetChatClient(deploymentName)
        .CreateAIAgent(agentOptions);

 // Set PersonInfo as the type parameter of RunAsync method to specify the expected structured output from the agent and invoke the agent with some unstructured input.
 AgentRunResponse<PersonInfo> response = await agent.RunAsync<PersonInfo>("Please provide information about John Smith, who is a 35-year-old software engineer.");

 // Access the structured output via the Result property of the agent response.
 Console.WriteLine("Assistant Output:");
 Console.WriteLine($"Name: {response.Result.Name}");
 Console.WriteLine($"Age: {response.Result.Age}");
 Console.WriteLine($"Occupation: {response.Result.Occupation}");

[stephentoub]

Do you need to duplicate all the code from RunAsync? Seems like the only things that differ are the GetResponseAsync call and the construction of the response at the end, in which case you could share most of it using a delegate or interface and two implementations or something like that?

[stephentoub]

I don't know how useful useJsonSchemaResponseFormat: false is. I guess we can keep exposing it, as it's just mirroring what the underlying API provides, but I don't see folks using false and it's not actually using structured output so its results are questionable.

[SergeyMenshykh]

The AgentRunResponse<T> class can potentially be used by other agents that may support structural output; hence, it should be declared in the Microsoft.Agents.AI.Abstractions project rather than the Microsoft.Agents.AI one.

The reason it's declared in the Microsoft.Agents.AI project is that the class depends on the ChatResponse<T> class from Microsoft.Extension.AI package, and moving the class to Microsoft.Agents.AI.Abstractions project would require adding a dependency on Microsoft.Extension.AI package to the project, which does not seem like the right thing to do.