stable-2023-09-19

New models enabled (e2e tested via portable lib):

• Emformer RNN-T Transcriber, Predictor, Joiner (as three modules)

Quantization:

• Enabled quantization for incpetion_v4 and deeplab_v3 in examples with XNNPACKQuantizer

API changes:

• Runtime API
  • Many runtime APIs changed to improve ergonomics and to better match the style guide. Most of these changes are non-breaking (unless indicated as breaking), since the old APIs are available but marked as deprecated. We recommend that users migrate off of the deprecated APIs before the next release.
    • For an example of how these API changes affected common use cases, see the edits made to examples/executor_runner/executor_runner.cpp under the "Files changed" tab of stable-2023-09-12...78f884f
  • Breaking behavioral change: MethodMeta
    • MethodMeta::num_non_const_buffers and MethodMeta::non_const_buffer_size no longer require adjusting by 1 to skip over the reserved zero index. This will require that users of MethodMeta remove adjustments while counting and iterating over non-const buffers.
    • Details about the change, including migration to adapt to the new behavior: 5762802
    • Also note that these methods have been renamed to num_memory_planned_buffers and memory_planned_buffer_size (see note below)
    • Note that the deprecated Program::num_non_const_buffers and Program::get_non_const_buffer_size methods did not change behavior re: skipping index zero. But they are deprecated, and will be removed in a future release, so we recommend that users migrate to the MethodMeta API and behavior.
  • MethodMeta method names changed from non_const to memory_planned
    • MethodMeta::num_non_const_buffers() is now MethodMeta::num_memory_planned_buffers()
    • MethodMeta::non_const_buffer_size(N) is now MethodMeta::memory_planned_buffer_size(N)
    • Changed in 6944c45
    • The old names are available but deprecated, and will be removed in a future release
  • Breaking code-compatibility change: Method's constructor and init() method are now private
    • Users should not have used these methods; Method instances should only be created by Program::load_method()
    • Changed in 4f3e5e6
  • MemoryManager constructor no longer requires const_allocator or kernel_temporary_allocator
    • A new constructor lets users avoid creating zero-sized allocators that they don't use
    • It also renames the parameters for the remaining allocators to make their uses more clear
    • Changed in 6944c45
    • Example migration to the new constructor: fedc04c
    • The old constructor is available but deprecated, and will be removed in a future release
  • Breaking code-compatibility change: MemoryManager is now final and cannot be subclassed
    • Changed in 6944c45
  • HierarchicalAllocator's constructor now takes an array of Span<uint8_t> instead of an array of MemoryAllocator
    • Changed in 58c8c92
    • Example migration to the new API: 0bce2cb
    • The old constructor is still available but deprecated, and will be removed in a future release
  • Breaking code-compatibility change: HierarchicalAllocator is now final and cannot be subclassed
    • Changed in 58c8c92
  • Program::Load() renamed to Program::load()
    • Changed in 8a5f3e8
    • The old name is still available but deprecated, and will be removed in a future release
  • FileDataLoader::From() renamed to FileDataLoader::from()
    • Changed in e2dd0be
    • The old name is still available but deprecated, and will be removed in a future release
  • MmapDataLoader::From() renamed to MmapDataLoader::from()
    • Changed in 395e51a
    • The old name is still available but deprecated, and will be removed in a future release
• Delegate API
  • File rename: runtime/backend/backend_registry.cpp -> runtime/backend/interface.cpp
  • Partition API update: Partitioner.partition function takes ExportedProgram instead of torch.nn.GraphModule. With this change we access the parameters and buffer in partition function.
    • How to rebase: access graphmodule by exported_program.graph_module
• SDK
  • BundledProgram updates APIs to enable user bundling test cases on specific method by using method name instead of method id in the past

    • AOT: class BundledConfig (method_names: List[str], inputs: List[List[Any]], expected_outputs: List[List[Any]]). method_names is the new added attribute.

    • Runtime: Replace the original method_idx with method_name

      • API for load bundled test input to ET program:
        __ET_NODISCARD Error LoadBundledInput( Method& method, serialized_bundled_program* bundled_program_ptr, MemoryAllocator* memory_allocator, const char* method_name, size_t testset_idx);
      • API for verify result with bundled expected output:
        __ET_NODISCARD Error VerifyResultWithBundledExpectedOutput( Method& method, serialized_bundled_program* bundled_program_ptr, MemoryAllocator* memory_allocator, const char* method_name, size_t testset_idx, double rtol = 1e-5, double atol = 1e-8);

    • Details and examples can be found https://github.com/pytorch/executorch/blob/stable/docs/website/docs/tutorials/bundled_program.md

Bug Fixes:

• When exporting with enable_aot=True, all constant tensors will be lifted as inputs to the graph (in addition to the parameters and buffers).
• Kwargs are now consistently placed in the call_spec of the exported program.