How to switch the default LLM to a local one?

[inf17101]

Hi,

I am going through the Getting Started Tutorial. I have to setup OpenAI keys. It is mentioned that the lmos-runtime and the demo agents use OpenAI as LLM backend.

How can I use a local model e.g. with ollama ?

Thank you in advance.

[suyambuganesh]

This example spring boot project uses ollama https://github.com/eclipse-lmos/arc-spring-init

[RobWin]

Yes, you can configure multiple models

https://github.com/eclipse-lmos/arc-spring-init/blob/main/config/application.yml#L14

And then reference the client/model id in your agent

https://github.com/eclipse-lmos/arc-spring-init/blob/main/agents/assistant.agent.kts#L8

and finally you need to add the library

https://github.com/eclipse-lmos/arc-spring-init/blob/main/build.gradle.kts#L57

[RobWin]

@patwlan can you confirm?

[patwlan]

That is correct. Currently, we only support ollama for local models. The Url can also be set, otherwise the default is http://localhost:11434

If there is any other @inf17101 system you would like to use, let us know.