Skip to content

Add additional checks to EvalEvalBot #141

@nelaturuharsha

Description

@nelaturuharsha

Add following features to the EvalEvalBot:

  • Enforcing score to be between max and min specified. In case min_score, max_score and score_type are not present - the user should be warned.
  • Enforcing counts to be integers
  • For a given benchmark, the min_score, max_score and score_type should be the same across all occurrences.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions