[compile] Enable sequence parallelism matching w/o custom ops enabled

[angelayi]

Purpose

Based on #24604, modified sequence-parallelism pass to do custom op matching w/o needing to enable the custom op

Test Plan

pytest -sv tests/compile/test_sequence_parallelism.py

Performance numbers

I did some benchmarking with the command on H100 w/o flashinfer

VLLM_DISABLE_COMPILE_CACHE=1 VLLM_USE_STANDALONE_COMPILE=1 VLLM_LOGGING_LEVEL=DEBUG vllm bench latency --model=nvidia/Llama-3.3-70B-Instruct-FP8 --output-len 1 --input-len 8192 --batch-size 1 --tensor-parallel-size 8 --load-format dummy --num_iters_warmup 5 --num_iters 15 -O '{"level": 3, "use_inductor_graph_partition": false, "splitting_ops":[], "cudagraph_mode": "FULL", }' --no-enable-prefix-caching

while varying

• "pass_config": {"enable_async_tp": true, "enable_sequence_parallelism": true} vs. "pass_config": {"enable_async_tp": false, "enable_sequence_parallelism": false}
• "custom_ops":["+quant_fp8", "+rms_norm"] vs. "custom_ops":[]

[ProExpertProg]

Thanks for taking this on! Could you just add me as a co-author on one of the commits?

[ProExpertProg]

Does this work? I thought that during tracing the pattern matching tracer will think that args is a single parameter

[angelayi]

yes! updated the test to assert the number of all_reduce/all_gather ops in the graph!