[Question]: Why not use QP created by ibverbs instead in IBGDA

[gangxie112]

Question

Hello,

Any consideration about this? Recently, we found the behavior is quite different when error handling with devx QP. And devx QPs are not as stable as those created via ibverbs. So, we are considering change the qp in IBGDA.

Thanks

[schwitanski]

Hi,

You are asking why in we are not using ibv_create_qp and others provided by ibverbs, right? In IBGDA, we need to have control over placement of the WQE buffers, CQ buffers, DBR etc. because we need to place them in GPU memory. This is not possible by using the high-level ibv_* functions so we use mlx5dv + DEVX functionality here.