Icon fetching reliability

[gunn]

Is this a fine place to post an issue?

This project is working excellently for the most part. There are a few domains with favicons which besticon doesn't find though.

storage.googleapis.com fails despite it redirecting to a png from https://storage.googleapis.com/favicon.ico - perhaps this is because storage.googleapis.com responds with a HTTP 400? same story with lh3.googleusercontent.com

There are some other domains that fail because of the casing of the link tag attributes:

<link rel="Shortcut Icon" href="...

I don't know if the parsing library supports it, but if it does then a fix would be to use case-insensitive query selectors:

"link[rel='shortcut icon' i]"

See https://css4-selectors.com/selector/css4/attribute-case-sensitivity/

[gurgeous]

Thanks gunn. I think #49 fixes the Shortcut Icon issue. If I end up doing more PRs I'll take a peek at the other two.

[gunn]

Nice!

[gurgeous]

I can confirm that both https://storage.googleapis.com and https://lh3.googleusercontent.com fail because we are unable to fetch the HTML for those domains. We stop processing at that point and return an error. The favicons are fine, though:

https://storage.googleapis.com/favicon.ico
https://lh3.googleusercontent.com/favicon.ico

I'm hoping we can fix this soon. Thanks for the bug report!

[gurgeous]

@mat what do you think about changing fetchIcons to recover from HTTP errors? The easiest fix would be to have fetchIcons populate with default paths if fetchHTML fails. Happy to give it a shot.

[mat]

This should solve it yes, makes total sense to not rely on the fetchHTML at all 👍