Not seeing create evaluation set in Eval

[VandanaJn]

I am on windows machine. I have created an agent using python files. It works as expected from cli as well as adk web but does not show create evaluation set in eval tab. If I create a new agent using adk web, which generates yml files, create evaluation set shows for that agent. Appreciate any guidance. I am using google-adk==1.19.0 and google-adk[eval]

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: To enable the "Create evaluation set" button, your Python-based agent needs to be organized in a specific directory structure that the ADK web UI expects for saving evaluation files.

---

Hello! Thank you for your question.

The behavior you're observing is likely due to how the ADK web UI discovers agents and manages their associated files, including evaluation sets.

When you create an agent using the web UI, it automatically generates a directory for that agent, which is where it stores the configuration (root_agent.yaml) and any evaluation sets you create. The "Eval" tab functionality depends on this directory structure to know where to save new evaluation files (.evalset.json). [2]

For an agent defined in Python files, you need to manually create a similar structure. The ADK expects to find each agent within its own sub-directory inside a main "agents" folder.

To resolve this, please try the following steps:

1. Create a main directory for your agents if you don't have one already (e.g., my_agents).

2. Create a sub-directory inside it for your Python-based agent (e.g., my_agents/my_python_agent).

3. Move your Python files (the one defining your root_agent) into this new sub-directory.

4. Add an __init__.py file inside the sub-directory (my_agents/my_python_agent/__init__.py). This file can be empty. This tells Python to treat the directory as a package, which helps the ADK with discovery. [1]

5. Run the ADK web UI from your terminal, pointing it to the main agents directory:

   adk web my_agents

By organizing your project this way, the ADK web UI will recognize my_python_agent as a distinct application and will have a designated directory (my_agents/my_python_agent/) to save evaluation files. This should enable the "Create evaluation set" button in the Eval tab.

For more details on how evaluations work in the web UI, you can refer to the official documentation. [2]

Let us know if that solves the issue!

---

References
[1] AGENTS - AGENTS.html
[2] How to run Evaluation with the ADK - index.html

[VandanaJn]

It worked! Thank you