Auto-generated alt-text in images using OpenRouter#128
Conversation
|
I've done something similar before. Android actually provides this capability; we can use Android's built-in LLM to generate ALT Text for images. However, only some phones support this. My phone, although on the list of supported devices, doesn't actually support it, so this feature is unstable, which is why I haven't merged it into the main branch. But your solution seems better; it works on any device. Thanks for the contribution. |
|
Hi! Yes, and also should work on iOS. I don't have a Mac/iphone to test the build so I left a stub in there and split the iOS image resizing work into a separate branch. Also this works with openAI or any service which uses open AI API spec. |
08e6c0b to
77fc396
Compare
|
Hi @0xZhangKe , I've rebased off of your latest main the PRs I had with conflicts. Also, to fix the CI builds failing, see the fix: KonradIT@7f31769 |
This PR is inspired by this post: https://bsky.app/profile/ed3d.net/post/3mkvxlhyot22i
and by Ave's blog post: https://ave.zone/blog/playing-with-vlm-generated-alt-text
A new button shows up on the alt-text dialog to send the image to an LLM which returns the alt text.
The model prompt, model choice and max token size were taken from Ave's blog post as-is. I trust them in their analysis.
The iOS part is stubbed-out. WIP work is here, untested: https://github.com/KonradIT/Fread/tree/ai-autogenerate-alt-text-ios
UI changes:
A new config page was added to the settings to configure LLM autogenerated alt-text:
After this is filled out properly with your LLM provider of choice (using OpenAI-compatible API requests, openrouter is just what I chose)