Skip to content

Conversation

qti-kromero
Copy link
Contributor

@qti-kromero qti-kromero commented Aug 18, 2025

Describe your changes

Checklist before requesting a review

  • Add unit tests for this change.
  • Make sure all tests can pass.
  • Update documents if necessary.
  • Lint and apply fixes to your code by running lintrunner -a
  • Is this a user-facing change? If yes, give a description of this change to be included in the release notes.
  • Is this PR including examples changes? If yes, please remember to update example documentation in a follow-up PR.

@xiaoyu-work
Copy link
Contributor

Did you test the inference of the output model? I tested https://huggingface.co/google/gemma-3-270m but it failed with onnxruntime-genai when inferencing (this will be fixed soon), but I'm not sure if you got an error as well for Gemma3-4b with qnn.

@qti-kromero
Copy link
Contributor Author

Did you test the inference of the output model? I tested https://huggingface.co/google/gemma-3-270m but it failed with onnxruntime-genai when inferencing (this will be fixed soon), but I'm not sure if you got an error as well for Gemma3-4b with qnn.

We have intelligible output from the text portion of the model within our KPIs but we have not successfully prepared the vision portion of the model or combined the two models together in a pipeline of OGA. Do you have details on what specific portion of the model you had issues with?

@xiaoyu-work
Copy link
Contributor

Did you test the inference of the output model? I tested https://huggingface.co/google/gemma-3-270m but it failed with onnxruntime-genai when inferencing (this will be fixed soon), but I'm not sure if you got an error as well for Gemma3-4b with qnn.

We have intelligible output from the text portion of the model within our KPIs but we have not successfully prepared the vision portion of the model or combined the two models together in a pipeline of OGA. Do you have details on what specific portion of the model you had issues with?

I encountered a bug in model builder when testing gemma-3-270m, but it has now been fixed. Interestingly the 4B model works without issues, maybe there are some difference between their configs. If it is tested, I'm fine. Thanks.

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

prudhvi-qti and others added 3 commits September 5, 2025 02:53
Copy link
Contributor Author

@qti-kromero qti-kromero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apparently I have to make a comment

@qti-kromero qti-kromero changed the title Add Gemma3-4b QNN example Gemma3-4b QNN example fixes Sep 18, 2025
@qti-kromero qti-kromero marked this pull request as ready for review September 18, 2025 18:15
@qti-kromero
Copy link
Contributor Author

@jambayk @xiaoyu-work wondering if this could get reviewed when you have a chance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants