fix(integration tests): Dynamic Ollama model discovery for concurrent multi-model requests#539
Merged
Merged
Conversation
aiosfoundation
approved these changes
Apr 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix: Dynamic Ollama model discovery for concurrent multi-model requests
Problem
When concurrent LLM requests specify different Ollama models (e.g.,
qwen3:1.7bandqwen3:4b), requests for models not pre-registered inconfig.yamlfail with a 500 error:"Selected LLMs are not all available"— even though those models are installed and available on the Ollama server.This caused 2 of 4 concurrent Ollama CI tests to fail (
test_ollama_different_models,test_ollama_some_specified_different_models).Root Cause
LLMAdapter._initialize_llms()only registers models explicitly listed inconfig.yaml. The availability check (check_availability_for_selected_llm_lists) does a strict name-in-list comparison with no fallback to query the Ollama server for models that are installed but not pre-configured.Fix
Added dynamic Ollama model discovery to
LLMAdapterinaios/llm_core/adapter.py:_query_ollama_available_models()— queries the Ollama server's/api/tagsendpoint to get installed models_dynamic_register_ollama_model()— thread-safe, idempotent method that discovers and registers an Ollama model on-demand (updatesllm_configs,llms,available_llm_names, and router config)execute_llm_syscalls()— when a request fails the initial availability check and the unavailable models have an"ollama"backend, attempts dynamic registration before rejectingNon-Ollama backends, already-configured models, and truly unavailable models are completely unaffected.
Files Changed
aios/llm_core/adapter.py— dynamic registration infrastructure + integration intoexecute_llm_syscalls()aios/config/config.yaml.example— addedqwen3:4bto the default Ollama models list (CI pulls both models)requirements.txt— addedhypothesis(test dependency for property-based tests)tests/modules/llm/ollama/test_dynamic_registration_pbt.py— new test file with Hypothesis-based bug condition and preservation tests (runs without Ollama server, fully mocked, compatible with both pytest and unittest runners)Testing
python -m pytestandpython -m unittest(CI uses the latter)test_ollama_same_model,test_ollama_different_models,test_ollama_some_specified_same_model,test_ollama_some_specified_different_models)