Run test_dtensor with torchrun

[kshitij12345]

Context:
When running the test_dtensor.py like we do now (pytest thunder/tests/distributed/test_dtensor.py), we see Error from segmentation group 1: The singleton Communicator isn't available. for #2503

Changes:

• Move all the tests under class DTensorTest(DistributedParallelTestCase) to free functions with test prefix and run the test with torchrun --nproc-per-node 2 --no-python pytest thunder/tests/distributed/test_dtensor.py (This is required to correctly run the nvFuser tests)

cc @Borda

[wujingyue]

LGTM, but can you clarify the context? What makes it hard for nvFuser DTensor tests to work with DistributedParallelTestCase?

[kshitij12345]

but can you clarify the context?

(Have also added this to PR description)
When running the test_dtensor.py like we do now (pytest thunder/tests/distributed/test_dtensor.py), we see Error from segmentation group 1: The singleton Communicator isn't available. for #2503

What makes it hard for nvFuser DTensor tests to work with DistributedParallelTestCase?

I was seeing hang-up in some tests when DistributedParallelTestCase is used with torchrun --nproc-per-node 2 --no-python pytest thunder/tests/distributed/test_dtensor.py. Haven't found the exact cause for the same.

[KaelanDt]

Thank you @kshitij12345

[crcrpar]

I was seeing hang-up in some tests when DistributedParallelTestCase is used with torchrun --nproc-per-node 2 --no-python pytest thunder/tests/distributed/test_dtensor.py. Haven't found the exact cause for the same.

we can just run DistributedParallelTestCase-based tests with pytest and I think it's the way to run such tests. PyTorch distributed elastic launch command is not needed. (I know you have a problem with pytest and dtensor test)