What is the best way to send image to agent via API?

[hoanghonn]

I have an external system that will handle user's query and image upload. I will need to pass this image to the agent service (which is a microservice) via API endpoint. What is the best way to do this?

What I am trying right now:

1. Upload the image to GCS.
2. Using callbacks to download GCS and load to artifacts.

TIA 🙇🏼‍♂️

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: Your current approach of uploading the image to Google Cloud Storage (GCS) and passing the GCS URI to the agent is a robust and recommended method, especially for agents deployed on Google's managed Agent Engine service. [2] An alternative is to send the image data directly within your API request (e.g., as a base64-encoded string) and process it from bytes on the server.

---

Hello! That's an excellent question. Let's break down the best practices for sending images to an ADK agent.

Option 1: GCS URI (Your Current Approach)

Your method of uploading the image to GCS and then passing the URI to the agent is the officially recommended and supported method for agents deployed on Agent Engine. [2] It's a very common and scalable pattern.

How it works:

You would use the types.Part.from_uri function to create an image part from the GCS link. [2]

from google.genai import types

# The GCS URI you receive in your API call
gcs_uri = "gs://your-bucket-name/path/to/image.jpg"

image_part = types.Part.from_uri(
    file_uri=gcs_uri,
    mime_type="image/jpeg", # Or the appropriate mime type
)

# Combine with a text prompt
text_part = types.Part.from_text("What is in this image?")

# Send to the agent
# response = agent.query(message=[text_part, image_part])

Advantages:

• Decoupling: It separates the concern of file storage from your agent's logic.
• Performance: It keeps the request payload to your agent small and avoids potential timeouts or size limits associated with sending large files over HTTP.
• Managed Services: It integrates perfectly with services like Agent Engine. [2]

Option 2: Direct Image Data in API Request

Alternatively, you can send the image data directly to your agent service. A common way to do this is to base64-encode the image on the client side, send it as a JSON string, and then decode it on the server.

How it works:
Your agent service would receive the encoded string, decode it back into bytes, and then use types.Part.from_bytes. [7]

import base64
from google.genai import types

# In your API endpoint, assuming you get a base64 string
base64_image_string = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/wcAAwAB/epv2AAAAABJRU5ErkJggg=="
image_data = base64.b64decode(base64_image_string)

image_part = types.Part.from_bytes(
    data=image_data,
    mime_type="image/png",
)

# Combine with a text prompt and send to the agent
# ...

Advantages:

• Simplicity: It can simplify the client-side logic since it's a single API call to your service without a separate upload step.
• Self-Contained: The agent request contains all the necessary data, which can be useful for logging or debugging.

Conclusion

Both methods are valid. Your current approach with GCS is considered a best practice for scalability and is the standard for managed services like Agent Engine. [2] The direct data method is a good alternative for simpler use cases or when you want to avoid a dependency on GCS for every request. Given you are already implementing the GCS flow, you are on the right track with a robust solution.

[hoanghonn]

What if multiple agents are using the same image? Can I save that into artifacts to reduce access query?