bug(runtime): Add support for MPICH and IntelMPI implementations in MPI plugin

[yush-1018]

What happened?

Description:

Summary

The trainer.kubeflow.org/v1alpha1 API definition supports OpenMPI, Intel, and MPICH as valid values for MPIImplementation. However, the MPI plugin (pkg/runtime/framework/plugins/mpi/mpi.go) currently only supports OpenMPI. Specifying MPICH or Intel causes the controller to throw an unhandled runtime error.

Steps to Reproduce

1. Create a TrainingRuntime or TrainJob manifest with mpiImplementation set to MPICH or Intel:

   apiVersion: trainer.kubeflow.org/v1alpha1
   kind: TrainJob
   spec:
     trainer:
       resourcesPerNode:
         limits:
           cpu: "2"
     runtimeRef:
       name: mpi-runtime

2. Apply the manifest to the cluster.
3. Observe the controller reconciliation logs.

Expected Behavior

The controller should accept MPICH and Intel implementations, correctly mount the hostfile, inject the appropriate environment variables, and reconcile the job without errors.

Actual Behavior

The controller fails at runtime with the error:
MPI implementation for MPICH doesn't supported
Line 52 in pkg/runtime/framework/plugins/mpi/mpi.go explicitly marks this unaddressed TODO:
// TODO : Support MPICH and IntelMPI implementations.

Proposed Fix

1. Extend EnforceMLPolicy() in pkg/runtime/framework/plugins/mpi/mpi.go to handle MPIImplementationMPICH and MPIImplementationIntel.
2. Update buildHostFileConfigMap() to render hostfiles for MPICH and Intel MPI.
3. Add unit test coverage in pkg/runtime/framework/plugins/mpi/mpi_test.go.

Environment & Affected Files

• Kubeflow Trainer version: v2 (master branch)
• pkg/runtime/framework/plugins/mpi/mpi.go
• pkg/runtime/framework/plugins/mpi/mpi_test.go
  7:44 PM

What did you expect to happen?

When setting mpiImplementation: MPICH or mpiImplementation: Intel in a TrainingRuntime or TrainJob, the controller should accept these valid API enum values, inject the appropriate hostfile and environment variables for MPICH or Intel MPI, and successfully reconcile the job without errors.

Environment

Kubernetes version:

$ kubectl version

Kubeflow Trainer version:

$ kubectl get pods -n kubeflow-system -l app.kubernetes.io/name=kubeflow-trainer -o jsonpath="{.items[*].spec.containers[*].image}"

Kubeflow Python SDK version:

$ pip show kubeflow

Impacted by this bug?

Give it a 👍 We prioritize the issues with most 👍

[github-actions]

🎉 Welcome to the Kubeflow Trainer! 🎉

Thanks for opening your first issue! We're happy to have you as part of our community 🚀

Here's what happens next:

• Our team will review your issue soon! cc @kubeflow/kubeflow-trainer-team
• If you'd like to contribute to this issue, check out our Contributing Guide for repo-specific guidelines and the Kubeflow Contributor Guide for general community standards.

Join the community:

• Slack: Join our #kubeflow-trainer Slack channel.
• Meetings: Attend the Kubeflow AutoML and Training Working Group bi-weekly meetings.

Feel free to ask questions in the comments if you need any help or clarification!
Thanks again for contributing to Kubeflow! 🙏