[regression PR #232] dynamic-shape compile fails: undefined symbol hipdnn_ep_get_host_scratch_base

[fhanuman]

Summary

origin/main (currently 670b96d, the merge of PR #232 "Add Memory Manager subsystem for GPU memory allocation") cannot compile any dynamic-shape model. The compiler emits a call to the runtime function hipdnn_ep_get_host_scratch_base, but PR #232 deleted that function from the runtime without updating the compiler — ld.lld rejects the JIT-link and the session aborts.

Fixed-shape compiles are unaffected because hip-materialize-host-scalars only emits the offending op on graphs with dynamic dimensions — also why GitHub Actions gpu-test (only runs *seq128.onnx fixed-shape models) doesn't catch it.

Where

Compiler side — still references the deleted symbol (these are correct as-is; the bug is on the runtime side):

• lib/Dialect/Transforms/MaterializeHostScalars.cpp — --hip-materialize-host-scalars pass creates hip.get_host_scratch ops on dynamic-shape graphs
• include/hip/Dialect/IR/HipOps.td#L78 — Hip_GetHostScratchOp op definition
• lib/Conversion/HipToLLVM/MemoryLowering.cpp#L150-L171 — emits the llvm.call @hipdnn_ep_get_host_scratch_base(state, size)
• lib/Conversion/HipToLLVM/HipToLLVMUtils.h#L41-L42 — kHipGetHostScratch = "hipdnn_ep_get_host_scratch_base"

Runtime side — symbol deleted by PR #232 (commit 6dc93b4):

• lib/Runtime/hipdnn_ep_runtime.h — declaration deleted
• lib/Runtime/hipdnn_ep_runtime_state.cpp — definition deleted
• lib/Runtime/runtime_state_internal.h — host_scratch_base / host_scratch_size fields deleted

Repro

Tested on Linux gfx1151 (Strix Halo). Dynamic-shape Llama-3.1-8B model is staged at xconucstrhalo20:/scratch/fhanuman/reporMemoryManagerIssue (amd/Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml with genai_config_MorphiZenEP.json as genai_config.json).

git fetch origin main && git checkout -b verify-mm-regression origin/main   # 670b96d
BUILD_OGA=1 ./docker/run.sh build

docker run --rm \
    --mount type=bind,source=/scratch/fhanuman,target=/scratch/fhanuman,bind-propagation=shared \
    -w /scratch/fhanuman/RCOmEP/install/lib \
    --user $(id -u):$(id -g) --group-add $(stat -c '%g' /dev/kfd) \
    --device=/dev/kfd --device=/dev/dri/renderD128 \
    -e THEROCK_DIST=/scratch/fhanuman/RCOmEP/therock-dist \
    -e LD_LIBRARY_PATH=/scratch/fhanuman/RCOmEP/install/lib:/scratch/fhanuman/RCOmEP/therock-dist/lib \
    --entrypoint bash hipdnn-ep-build:llvm22-noble \
    -c '../bin/model_benchmark -i /scratch/fhanuman/reporMemoryManagerIssue -l 128 -g 128 -r 3 -w 1 -b 1 -v'

Fails in ~2 s with:

[ConvertOnnxToHipPass] streaming: 743 file-ref + 2 splat -> per-entry descriptors
ld.lld: error: undefined symbol: hipdnn_ep_get_host_scratch_base
>>> referenced by LLVMDialectModule
>>>               /tmp/morphizen_mlir_<N>_<ts>_0.obj:(.text+0xd8a)
clang++: error: linker command failed with exit code 1
[E:onnxruntime:, MlirCompiler.cpp:122] Compilation failed: Failed to link DLL
[E:onnxruntime:, inference_session.cc:2564 Initialize] This session contains
  graph nodes that are assigned to the default CPU EP, but fallback to CPU EP
  has been explicitly disabled by the user.

Why CI did not catch this

• .github/workflows/windows-build.yml gpu-test runs *seq128.onnx fixed-shape models only. Fixed shapes don't run hip-materialize-host-scalars, so the symbol is never referenced and the link succeeds.
• The dynamic-shape path is only exercised in OGA Jenkins pipelines, where this failure was reported but dismissed as flaky.

Follow-up

An independent second regression in PR #232 surfaces once this link error is unblocked: dynamic-shape inference runs ~250 iterations and then emits ~4000 wrap_group_query_attention: gqa_forward failed (rc=-1) errors followed by Memory access fault by GPU node-1 at tear-down. Will file separately.