Adding Estimate NPU Latency pass and unit test

[alinah-amd]

Describe your changes

Overview

Added the EstimateNPULatency pass under olive/onnx/vitis_ai/estimate_latency.py
EstimateNPULatency makes use of the NPU Perf Estimator tool to predict computational performance of workloads given a set of parameters.

This is an analysis pass and does not transform the graph at all. Used for performance analysis only.

Installation

To install (if not installed through requirements.txt), run the following:
pip install [placeholder for wheel]

Confirm python version installed is >= 3.10 for compatibility.

If perf estimator package is not installed, the following warning will show and the pass will simply be bypassed:

Usage

Inputs
EstimateNPULatency takes in both the model in the form of an OnnxModelHandler object and a list of optional parameters in the form of a dict of PassConfigParams (consistent with all other passes).

Optional Parameters
To pass in optional parameters, list parameter name and parameter value as key-value pairs in the json file. See example:

1. target_device: Target device type. This is used to provide default config specific to that device type. Currently, only Strix is supported but will modify to include other devices in future.

• Type: str
• Default: stx
• Allowed Values: ["stx"]

Adding Pass to Config File
Should ideally be run as the last pass and listed last in the <model>.json file. For example:

Output
Generates a concise_summary directory within the run directory with the following files:

{model_name}_concise_summary.txt will display the following info on roofline latency, total compute ops, and what conclusion can be drawn on performance bottleneck (whether it is DDR Bandwidth bound or Compute bound):

{model_name}_concise_summary.csv will display the same info but specific to per op. Ops are listed in descending order of latency:

Known Passing Tests

Resnet w/ Perf Estimator
Refer to Olive Recipes Repo

MobileNet w/ Perf Estimator
Refer to Olive Recipes Repo

Unit Test

• Run python -m pytest test/unit_test/passes/vitis_ai/test_estimate_latency.py
• This unit test runs a dummy model with the estimate latency pass (calls it directly instead of end to end flow) and asserts that a concise_summary directory with dummy csv and txt files are generated

Checklist before requesting a review

• Add unit tests for this change.
• Make sure all tests can pass.
• Update documents if necessary.
• Lint and apply fixes to your code by running lintrunner -a
• Is this a user-facing change? If yes, give a description of this change to be included in the release notes.

(Optional) Issue link

[alinah-amd]

@microsoft-github-policy-service agree company="Microsoft"

@microsoft-github-policy-service agree company="AMD"

[jambayk]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[jambayk]

I think instead of raising a warning, it might be better to fail with import a helpful import error which tells what package to install. since olive caches runs, to rerun the pass with the dependency installed, they would have to clean the cache or delete the cached run that skipped the estimation.

[jambayk]

could you add a "module_dependencies" option like under the autoawqquantizer pass for the package required to run this estimation?

[jambayk]

please also add the required dependency to the requirements-test.txt under test