Cannot use ImageContent.from_url: 400 Bad Request

[ZeJ0hn]

Describe the bug
Using ImageContent.from_url, since version 2.18.0, we are unable to fetch images because of key 'user-agent' which appears twice in the HTTPX headers. The URL used is a SAS URL of a storage account blob.
If we deep dive in the code, we can see that there are this 2 values :
'User-Agent': 'haystack/LinkContentFetcher/2.20.0'
And
'user-agent': 'python-httpx/0.28.1'

Error message
400 Bad Request

Expected behavior
A status code 200

To Reproduce

• generate a SAS URL from image stored on Azure Storage Account
• try to fetch the Image using ImageContent.from_url

System:

• Haystack version (commit or version number): 2.20.0

[anakin87]

Hey @ZeJ0hn, thanks for the report.

Having a public URL example would help to debug the issue...

[ZeJ0hn]

Hi @anakin87 ,

Here a public image : https://sawgcommon.blob.core.windows.net/assets/welcome.jpg?sp=r&st=2025-11-20T13:51:22Z&se=2026-02-01T22:06:22Z&spr=https&sv=2024-11-04&sr=b&sig=3SrTEg4yIKPlxkCQlXGRJSkhP%2FJ%2FucfsZ7fLq2ZLFik%3D

(Link is valid until 01/02/2026)

Here, the python sample uses to test:

import httpx

if __name__ == "__main__":
    url = 'https://sawgcommon.blob.core.windows.net/assets/welcome.jpg?sp=r&st=2025-11-20T13:51:22Z&se=2026-02-01T22:06:22Z&spr=https&sv=2024-11-04&sr=b&sig=3SrTEg4yIKPlxkCQlXGRJSkhP%2FJ%2FucfsZ7fLq2ZLFik%3D'
    r = httpx.get(url, headers={'Accept-Language': 'en-US,en;q=0.9,it;q=0.8,es;q=0.7', 'User-Agent': 'haystack/LinkContentFetcher/2.20.0', 'accept': '*/*', 'accept-encoding': 'gzip, deflate', 'connection': 'keep-alive', 'referer': 'https://www.google.com/', 'user-agent': 'python-httpx/0.28.1'})

    print(r.status_code)

If I remove one of the two user-agent, it works 😄

[anakin87]

Thanks, the example helps!

[anakin87]

Might be related: #9760

[ZeJ0hn]

That exactly that 😄

Default headers from httpx have been added (dict(self._client.headers)) but keys are in lowercase. And haystack and custom headers are in another case, so you can have the same key twice.

[anakin87]

Feel free to create a PR if the root cause of this issue is clear. I haven't yet investigated.

[vblagoje]

Fixed with #10111 Closing, thank you @ZeJ0hn