Constraints on evaluation and reinforcement

[ABCDabcde1098234]

Your work is so excellent! But I have a question. The names of the constraints in your review are different from those in the reinforcement settings. Can all the constraints in your reinforcement settings include those in your review? There are all 54 constraints, but their names are different. For example, count: word_comunt_range is the constraint you evaluated, but the strengthened constraint has been changed to length_craints: numbered instead.

[ValentinaPy]

Hi,

can you tell me what constraint names you are comparing? Could you link to the two files?

[ABCDabcde1098234]

For example, https://github.com/allenai/IFBench/blob/main/instructions.py This constraint's wordCountRangeChecker, but https://github.com/allenai/open-instruct/blob/main/open_instruct/IFEvalG/instructions.py There's nothing in this one

[ValentinaPy]

The instructions are different because we are adhering to a train-test split. The instructions in IFBench are TEST instructions (not for training) and the instructions in IFEvalG are training instructions! IFBench is a benchmark, not intended for training. The IFTrain constraints are in IFEvalG.