-
Notifications
You must be signed in to change notification settings - Fork 243
Gemma3-4b QNN example fixes #2106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Did you test the inference of the output model? I tested https://huggingface.co/google/gemma-3-270m but it failed with onnxruntime-genai when inferencing (this will be fixed soon), but I'm not sure if you got an error as well for Gemma3-4b with qnn. |
We have intelligible output from the text portion of the model within our KPIs but we have not successfully prepared the vision portion of the model or combined the two models together in a pipeline of OGA. Do you have details on what specific portion of the model you had issues with? |
I encountered a bug in model builder when testing gemma-3-270m, but it has now been fixed. Interestingly the 4B model works without issues, maybe there are some difference between their configs. If it is tested, I'm fine. Thanks. |
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
This fixes the issue of text model repeating words in the output.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
apparently I have to make a comment
@jambayk @xiaoyu-work wondering if this could get reviewed when you have a chance |
Describe your changes
Checklist before requesting a review
lintrunner -a