querier_format.md

LLMProblemInput Class Specification

Fields

• problem_id (str): A string identifier for the problem.
• prompt_id (str): A string identifier for the prompt.
• prompt (str): The text of the prompt.
• sample_inputs_outputs (List[TestCase]): A list of TestCase objects representing sample inputs and expected outputs for the problem.
• input_code (str): The initial code provided for the problem, if any.
• function_prototype (FunctionPrototype): A FunctionPrototype object representing the function prototype for the problem.

Methods

__init__(self, data: Dict[str, Any]) -> None

Constructor method which initializes an LLMProblemInput instance from a dictionary.

from_json(cls, json_data: Dict[str, Any]) -> 'LLMProblemInput'

Class method which creates an LLMProblemInput instance from a dictionary.

to_json(self) -> Dict[str, Any]

Method which serializes the LLMProblemInput instance to a JSON-serializable dictionary.

__str__(self) -> str

Method which returns a string representation of the LLMProblemInput instance.

LLMSolution Class Specification

Fields

• problem_identifier (str): A string identifier for the problem.
• model_identifier (str): A string identifier for the model.
• prompt_identifier (str): A string identifier for the prompt.
• solution_code (str): The solution code generated by the model.
• feedback (Optional[dict]): Optional feedback information.

Methods

__init__(self, problem_identifier: str, model_identifier: str, prompt_identifier: str, solution_code: str, feedback: Optional[dict] = None) -> None

Constructor method which initializes an LLMSolution instance with specified values.

from_json(cls, data: Dict[str, Any]) -> 'LLMSolution'

Class method which creates an LLMSolution instance from a dictionary.

to_json(self) -> Dict[str, Any]

Method which serializes the LLMSolution instance to a JSON-serializable dictionary.

__str__(self) -> str

Method which returns a string representation of the LLMSolution instance.

JSON Format

When serialized, the JSON for LLMSolution should adhere to the following format:

{
	"problem_identifier": "<string>",
	"model_identifier": "<string>",
	"prompt_identifier": "<string>",
	"solution_code": "<string>",
	"feedback": "<string>"
}

• problem_identifier: (String) A unique identifier for the problem.
• model_identifier: (String) A unique identifier for the model.
• prompt_identifier: (String) A unique identifier for the prompt.
• solution_code: (String) The solution code generated by the model.
• feedback: (String) Optional feedback information.