[WIP]Jessli/temp check main - check for tests

[YoYoJa]

Description

Please add an informative description that covers that changes made by the pull request and link all relevant issues.

If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.

All SDK Contribution checklist:

• The pull request does not introduce [breaking changes]
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

[Copilot]

Pull Request Overview

This pull request introduces support for Azure Monitor OpenTelemetry logging and adds token count tracking throughout the evaluation system. The changes enhance evaluation results with detailed token usage information and enable sending evaluation events to Application Insights. The PR appears to be a work-in-progress (WIP) based on the title.

Key Changes:

• Added comprehensive token count tracking (input/output/total tokens) across evaluators and graders
• Implemented Application Insights logging via OpenTelemetry with distributed tracing support
• Modified Prompty response format to return dictionaries containing both LLM output and metadata instead of raw outputs
• Added new data structures for AOAI evaluation results conversion and summary statistics

Reviewed Changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
setup.py	Added opentelemetry dependencies as optional extras
test_evaluate.py	Added comprehensive test for AOAI evaluation results conversion
test_built_in_evaluator.py	Updated mock functions to return dictionaries and adjusted assertions for new token count fields
test_prompty_async.py	Updated tests to handle new dictionary response format from Prompty
test_mass_evaluate.py	Updated assertions to reflect increased key counts from new token metrics
_prompty.py	Modified to return dict with llm_output and metadata
_utils.py	Modified format_llm_response to return comprehensive metadata dictionary
_base_prompty_eval.py	Updated to extract llm_output from new dictionary response format
_relevance.py	Updated to extract llm_output from new dictionary response format
_evaluate.py	Added AOAI results conversion, App Insights logging, and token count aggregation exclusion
_run_submitter.py	Added error handling for batch results
rai_service.py	Added token count fields to RAI service responses
Grader files	Added _type class variable for proper type identification
_constants.py	Added evaluator-to-metric mappings and OpenTelemetry event name constant

[Copilot]

The condition len(inputs) > 0 is redundant. In Python, empty dictionaries are falsy, so if inputs already checks for non-empty dictionaries. Consider simplifying to if inputs:.

Suggested change

	if inputs and len(inputs) > 0:
	if inputs:

[Copilot]

The variable sample_input is used without being initialized when the if inputs and len(inputs) > 0 condition is false. This will cause an UnboundLocalError when the function tries to use sample_input in the return statement. Initialize sample_input = "" before the conditional block.

[Copilot]

Incomplete comment text. The word 'Re' appears to be a fragment. Should likely be 'passed' or similar based on context.

Suggested change

	# Ensure the absolute path is Re to pf.run, as relative path doesn't work with
	# Ensure the absolute path is passed to pf.run, as relative path doesn't work with

[Copilot]

Comparing boolean to True using == True is redundant. Simplify to if copy_label is not None and isinstance(copy_label, bool) and copy_label:.

Suggested change

	if copy_label is not None and isinstance(copy_label, bool) and copy_label == True:
	if copy_label is not None and isinstance(copy_label, bool) and copy_label:

[Copilot]

[nitpick] Complex multi-line conditional is difficult to read. Consider extracting into a helper function or assigning intermediate boolean variables with descriptive names to improve readability.

Suggested change

	if (isinstance(result_item, dict) and "sample" in result_item and result_item["sample"]
	and result_item["metric"] not in dup_usage_list):
	is_dict = isinstance(result_item, dict)
	has_sample = "sample" in result_item
	sample_is_truthy = result_item.get("sample") if has_sample else False
	metric_not_in_dup_usage_list = result_item.get("metric") not in dup_usage_list
	if is_dict and has_sample and sample_is_truthy and metric_not_in_dup_usage_list: