Since the Curriculum Agent repeatedly receives the same task-generation prompt and is initialized from the base model, I am curious about how question diversity evolves during the early stages of Curriculum Agent training.
My understanding is that the repetition penalty discourages similar tasks within a generated batch by clustering questions according to BLEU similarity and assigning a larger penalty to questions in larger clusters. However, especially at the beginning of training, the Curriculum Agent has not yet been significantly updated by RL. Therefore, I would expect repeated sampling from the same prompt to still produce a substantial number of similar or near-duplicate questions.
Could you please clarify:
How severe is question repetition at the beginning of Curriculum Agent training?
What is the typical proportion of generated questions that fall into the same or highly similar clusters?
Do you have any statistics showing how this proportion changes as Curriculum Agent training progresses?
What are the batch size and BLEU similarity threshold used when computing the repetition penalty?
Since RL fine-tuning changes the policy gradually, do many repetitive samples initially receive low rewards before the Curriculum Agent learns to generate a more diverse task distribution?
I am particularly interested in understanding the dynamics of the repetition penalty itself, rather than only its downstream effect shown in the ablation study.
Thank you for releasing the work and code!
Since the Curriculum Agent repeatedly receives the same task-generation prompt and is initialized from the base model, I am curious about how question diversity evolves during the early stages of Curriculum Agent training.
My understanding is that the repetition penalty discourages similar tasks within a generated batch by clustering questions according to BLEU similarity and assigning a larger penalty to questions in larger clusters. However, especially at the beginning of training, the Curriculum Agent has not yet been significantly updated by RL. Therefore, I would expect repeated sampling from the same prompt to still produce a substantial number of similar or near-duplicate questions.
Could you please clarify:
How severe is question repetition at the beginning of Curriculum Agent training?
What is the typical proportion of generated questions that fall into the same or highly similar clusters?
Do you have any statistics showing how this proportion changes as Curriculum Agent training progresses?
What are the batch size and BLEU similarity threshold used when computing the repetition penalty?
Since RL fine-tuning changes the policy gradually, do many repetitive samples initially receive low rewards before the Curriculum Agent learns to generate a more diverse task distribution?
I am particularly interested in understanding the dynamics of the repetition penalty itself, rather than only its downstream effect shown in the ablation study.
Thank you for releasing the work and code!