Keep the original casing of model name for custom endpoint to enable Foundry Local #413
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Summary
Fix #410
This PR contains the following 2 fixes:
OpenAI models require lower case for their names, but the custom endpoint may be case-sensitive, such as the endpoint set up by Foundry Local. So, we should keep the original casing for custom endpoint configurations.
Use Foundry Local with
openai-gptagentUpdate the
openai.agent.jsonfile with the following GPT configuration:{ "GPTs": [ { "Name": "foundry-local", "Description": "A GPT instance using Foundry Local.", "Endpoint": "http://127.0.0.1:56952/v1", "ModelName": "Phi-3.5-mini-instruct-generic-cpu", "Key": "OPENAI_API_KEY" } ] "Active": "foundry-local" }In PowerShell, run the following commands to start up the Foundry service and load the model (use
phi-3.5-minias an example)Be noted that:
foundry service startplus/v1.foundry service pscommand."OPENAI_API_KEY".References
Foundry Local CLI: https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-local/reference/reference-cli
Foundry Local SDK: https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-local/reference/reference-sdk?pivots=programming-language-csharp