Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add RDMA support for Intel IPU E2000 (GEN 3) #1486

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

tatyana-en
Copy link
Contributor

[RFC Internal PATCH 0/9] Add RDMA support for Intel IPU E2000 (GEN 3)

This RFC patch series introduces the user space RDMA RoCEv2 support
for the Intel Infrastructure Processing Unit (IPU) E2000 line of
products, referred to as GEN 3 in the irdma provider.

The RFC patch series for the kernel GEN 3 irdma driver is at https://www.spinics.net/lists/linux-rdma/msg124994.html

Key highlights of this series as it pertains to GEN 3 device include:

  • Enable 64-byte Completion Queue Entry (CQE) support
  • CQE Error and Flush Handling
  • Push Page Support
  • Atomic Operations Support
  • Shared Receive Queue Support
  • New CQE Timestamp Capability

Faisal Latif (2):
providers/irdma: Add Atomic Operations support
providers/irdma: Add SRQ support

Jay Bhat (1):
providers/irdma: Add Push Mode for GEN 3

Mustafa Ismail (2):
providers/irdma: Enable 64-byte CQE support for GEN 3
providers/irdma: Set ibv_wc opcodes in GEN 3 RQ completions

Shiraz Saleem (4):
providers/irdma: Flush changes for GEN 3
providers/irdma: Set GEN 3 flag in ucontext allocation request
providers/irdma: Add CQE timestamp capability for GEN 3
providers/irdma: Add Type2 MW support

providers/irdma/abi.h | 2 +
providers/irdma/defs.h | 16 +-
providers/irdma/irdma.h | 9 +-
providers/irdma/osdep.h | 1 +
providers/irdma/uk.c | 471 +++++++++++++++++++++++++++++++++++----
providers/irdma/umain.c | 12 +
providers/irdma/umain.h | 26 ++-
providers/irdma/user.h | 274 ++++++++++++++++++++++-
providers/irdma/uverbs.c | 409 ++++++++++++++++++++++++++++-----
9 files changed, 1098 insertions(+), 122 deletions(-)

--
2.37.3

https://lore.kernel.org/r/all/20240724233917.704-1-tatyana.e.nikolova@intel.com/T/#t

mustafakismail and others added 10 commits July 26, 2024 09:58
Enable 64-byte CQE support if the feature flag IRDMA_FEATURE_64_BYTE_CQE is
set. Adjust the CQ size calculations in create_cq() and resize_cq()

Signed-off-by: Mustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Modify cq_poll_compl() to poll flushed CQEs for GEN 3

Add IBV_WC_RETRY_EXC_ERR and FLUSH_RNR_RETRY_EXC_ERR to the returned
ibv_wc_status errors

Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Add set_ib_wc_op_rq_gen_3() to set the ibv_wc opcodes for GEN 3 RQ
completions

Signed-off-by: Mustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Set WQE_FORMAT_V2 flag in ucontext allocation request. This prevents
driver from binding GEN3 devices to older user-space providers

Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Modify irdma_qp_get_next_send_wqe() to make a padding of NOPs as needed
for 64 byte aligned Push WQEs.

Add IRDMA_SUPPORT_MAX_HW_PUSH_LEN to comp_mask to impose a limit of the
Push WQE size, based on the supported message length.

Signed-off-by: Jay Bhat <jay.bhat@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Add atomic operations support for GEN 3 devices, namely Compare and Swap
and Fetch and Add.

Signed-off-by: Faisal Latif <faisal.latif@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Implement verb API and UAPI changes to support SRQ functionality in GEN 3
devices.

Provide function ae_to_qp_err_code() to translate AE codes into QP errors.
The function can be leveraged for flushed CQEs and error CQEs associated
with an SRQ.

Signed-off-by: Faisal Latif <faisal.latif@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Remove deprecated timestamp functionality for GEN 2.

Enable FEATURE_CQE_TIMESTAMPING for GEN 3 devices. The Host Channel
Adapter (HCA) core clock frequency in the timestamp info is retrieved
from the CQE.

Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Enable Type2 MW for GEN3 devices.

Type2 MW is disabled for GEN2/1 devices via the capability flag in
query_device.

Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Update kernel headers to patch
("[RFC PATCH 24/25] RDMA/irdma: Add Push Page Support for GEN3") at
https://www.spinics.net/lists/linux-rdma/msg125016.html

Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants