Perftest: Add support for Send with Immediate verb #345

Yue-ByteDance · 2025-09-02T04:10:20Z

Currently, perftest supports RDMA Write / Write-with-Immediate / Send /
Read and Atomic operations, but lacks Send-with-Immediate support. This
commit adds Send-with-Immediate functionality, allowing users to test it
via ib_send_bw with --send_with_imm flag.

Additionally, to verify the completion logic for operations with
immediate data is correct, completion events are now validated to ensure
the IBV_WC_WITH_IMM flag is set.

mrgolin · 2025-09-02T06:46:30Z

Generally looks good to me but I think we should add a check in run_iter_bw* and run_iter_lat that completion really has imm data. Something like:

if (user_param->use_send_with_imm && !(wc[i].wc_flags & IBV_WC_WITH_IMM)) {
        // error
        ...

Also can you please describe your changes in the commit message?

Yue-ByteDance · 2025-09-03T01:47:00Z

Generally looks good to me but I think we should add a check in run_iter_bw* and run_iter_lat that completion really has imm data. Something like:
if (user_param->use_send_with_imm && !(wc[i].wc_flags & IBV_WC_WITH_IMM)) {
        // error
        ...
Also can you please describe your changes in the commit message?

No problem, I will push a new version later

Yue-ByteDance · 2025-09-30T10:29:56Z

@mrgolin I've updated the code and the commit message with your suggestion, could you please take a look at the new version? Thanks!

mrgolin · 2025-10-19T13:08:49Z

src/perftest_resources.c

 		ctx_post_send_work_request_func_pointer(ctx, user_param);
 	#endif

+	if (user_param->verb == SEND_IMM || user_param->verb == WRITE_IMM) {


This should be always true.

Good catch, fixed, thanks a lot!

mrgolin · 2025-10-19T13:37:59Z

src/perftest_resources.c

 		ctx_post_send_work_request_func_pointer(ctx, user_param);
 	#endif

+	if (user_param->verb == SEND_IMM || user_param->verb == WRITE_IMM) {


We shouldn't get here with user_param->verb == WRITE_IMM.

Fixed, thanks!

src/perftest_resources.c

NitzanLavy · 2025-10-20T06:29:50Z

Our team would like to leverage Perftest test suite to test send scenarios while using the immediate field.
We were working on an implementation (#357) while being unaware that this PR is in progress. As this PR is already in advanced stages and is quite extensive and comprehensive, I will abandon our PR and will follow this instead. Thanks @Yue-ByteDance.
Code looks good to me, left a few comments.

src/perftest_parameters.c

Currently, perftest supports RDMA Write / Write-with-Immediate / Send / Read and Atomic operations, but lacks Send-with-Immediate support. This commit adds Send-with-Immediate functionality, allowing users to test it via ib_send_bw with --send_with_imm flag. Additionally, to verify the completion logic for operations with immediate data is correct, completion events are now validated to ensure the IBV_WC_WITH_IMM flag is set. Signed-off-by: Zelong Yue <[email protected]>

Yue-ByteDance · 2025-10-23T09:17:15Z

Our team would like to leverage Perftest test suite to test send scenarios while using the immediate field. We were working on an implementation (#357) while being unaware that this PR is in progress. As this PR is already in advanced stages and is quite extensive and comprehensive, I will abandon our PR and will follow this instead. Thanks @Yue-ByteDance. Code looks good to me, left a few comments.

Thanks! I've noticed that your PR was checking against imm data correctness, but mine is not, should I add this to my code? Do you have any suggestion?

NitzanLavy · 2025-10-26T12:56:42Z

Our team would like to leverage Perftest test suite to test send scenarios while using the immediate field. We were working on an implementation (#357) while being unaware that this PR is in progress. As this PR is already in advanced stages and is quite extensive and comprehensive, I will abandon our PR and will follow this instead. Thanks @Yue-ByteDance. Code looks good to me, left a few comments.

Thanks! I've noticed that your PR was checking against imm data correctness, but mine is not, should I add this to my code? Do you have any suggestion?

Yes I noticed that too. It could be nice to have it, but since it is a hardcoded constant I'm not sure it gives so much added value in comparison to validating the imm flags (which you are doing in your PR).
You can also generate a new value each run and assign all the WQEs imm field that value, but than you will have communicate it to the other side out-of-band so the receiver will know what to expect. Perhaps this is an exaggeration.

Yue-ByteDance · 2025-10-30T04:33:46Z

Our team would like to leverage Perftest test suite to test send scenarios while using the immediate field. We were working on an implementation (#357) while being unaware that this PR is in progress. As this PR is already in advanced stages and is quite extensive and comprehensive, I will abandon our PR and will follow this instead. Thanks @Yue-ByteDance. Code looks good to me, left a few comments.

Thanks! I've noticed that your PR was checking against imm data correctness, but mine is not, should I add this to my code? Do you have any suggestion?

Yes I noticed that too. It could be nice to have it, but since it is a hardcoded constant I'm not sure it gives so much added value in comparison to validating the imm flags (which you are doing in your PR). You can also generate a new value each run and assign all the WQEs imm field that value, but than you will have communicate it to the other side out-of-band so the receiver will know what to expect. Perhaps this is an exaggeration.

OK, I'll skip implementing imm data checking in this PR. I think perftest is mainly aimed at providing the best performance measurements for every opcode, rather than checking protocol correctness. Maybe we could add an option for that in the future, so it won't affect performance by default.

Yue-ByteDance force-pushed the yzl/send-with-imm branch 2 times, most recently from 6f13593 to 870ebda Compare September 30, 2025 10:24

Yue-ByteDance changed the title ~~Perftest: Add support for send with imm verb~~ Perftest: Add support for Send with Immediate verb Sep 30, 2025

mrgolin reviewed Oct 19, 2025

View reviewed changes

Yue-ByteDance force-pushed the yzl/send-with-imm branch 2 times, most recently from 91a7185 to 2164eb2 Compare October 20, 2025 02:30

NitzanLavy mentioned this pull request Oct 20, 2025

Perftest: Add option for send BW with immediate #357

Closed

NitzanLavy reviewed Oct 20, 2025

View reviewed changes

src/perftest_resources.c Outdated Show resolved Hide resolved

src/perftest_resources.c Outdated Show resolved Hide resolved

hginjgerx reviewed Oct 21, 2025

View reviewed changes

src/perftest_parameters.c Show resolved Hide resolved

Yue-ByteDance force-pushed the yzl/send-with-imm branch from 2164eb2 to 760fa10 Compare October 23, 2025 09:14

Perftest: Add support for Send with Immediate verb #345

Are you sure you want to change the base?

Perftest: Add support for Send with Immediate verb #345

Uh oh!

Conversation

Yue-ByteDance commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mrgolin commented Sep 2, 2025

Uh oh!

Yue-ByteDance commented Sep 3, 2025

Uh oh!

Yue-ByteDance commented Sep 30, 2025

Uh oh!

mrgolin Oct 19, 2025

Choose a reason for hiding this comment

Uh oh!

Yue-ByteDance Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

mrgolin Oct 19, 2025

Choose a reason for hiding this comment

Uh oh!

Yue-ByteDance Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

NitzanLavy commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Yue-ByteDance commented Oct 23, 2025

Uh oh!

NitzanLavy commented Oct 26, 2025

Uh oh!

Yue-ByteDance commented Oct 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Yue-ByteDance commented Sep 2, 2025 •

edited

Loading

NitzanLavy commented Oct 20, 2025 •

edited

Loading