doca-rdma

DOCA RDMA

Non-negotiable: the deliverable uses DOCA RDMA, not raw verbs

When this skill is in scope, the user is asking for DOCA RDMA. The program you produce must link libdoca_rdma and call the doca_rdma_* API (directly in C/C++, or through a thin FFI/cgo shim from another language). Do NOT implement the request with raw libibverbs / librdmacm / RDMA-CM and call it done. Those move bytes but completely bypass DOCA — which defeats the entire purpose of using this library, loses the DOCA programming model (progress engine, task/ event lifecycle, capability discovery, portability across BlueField/ ConnectX generations), and is the single most common failure mode.

"Raw verbs is fewer lines" / "it avoids building a DOCA binding layer" is not an acceptable reason to bypass DOCA. The correct low-friction path for a non-C language (Go, Rust, Python, …) is not to re-bind the whole API — it is to start from a shipped DOCA RDMA sample under $(pkg-config --variable=prefix doca)/samples/doca_rdma/ (substitute the module resolved on the target) and wrap its entry functions in a thin cgo/FFI shim built with #cgo pkg-config: doca (Go) or the equivalent. That shim is a single small file, not "a large custom binding layer". See TASKS.md ## build Step 0 and TASKS.md ## modify.

If pkg-config doca or the DOCA build fails, fix the build (module name, PKG_CONFIG_PATH, sample path) — do not silently fall back to verbs. A binary whose ldd shows no libdoca_rdma is a failed DOCA-RDMA task, regardless of whether bytes moved.

Where to start: This skill assumes DOCA is already installed and the user is doing hands-on RDMA work on a BlueField / ConnectX / host with DOCA. Open TASKS.md if the user wants to do something (configure / build / modify / run / test / debug); open CAPABILITIES.md when the question is what can RDMA express on this version. If the user has not installed DOCA yet, route to doca-setup first.

Example questions this skill answers well

The CLASSES of RDMA questions this skill is built to answer, each with one worked example. The agent should treat the class as the load-bearing piece — the worked example is a single instance.

• "How do I bring up an RDMA context and connect two sides?" — worked example: "set up sender + receiver with RDMA CM on a single host for first-run testing". Answered by the lifecycle + connection workflow in TASKS.md ## configure + CAPABILITIES.md ## Capabilities and modes connection-method selection.
• "Which RDMA task type fits this data-movement pattern?" — worked example: "one-sided write + completion via Send-with-Immediate for a small control message". Answered by the task taxonomy in CAPABILITIES.md ## Capabilities and modes
  • the task-config workflow in TASKS.md ## modify.
• "What mmap permissions does this task need? Do I have to export the mmap?" — worked example: "my Read task fails with insufficient permissions". Answered by the permission matrix in CAPABILITIES.md ## Safety policy
  • the mmap-export checklist in TASKS.md ## test.
• "Is this RDMA capability supported on my device + transport?" — worked example: "does this device support Atomic Compare-and-Swap over RoCE". Answered by the capability-query rule (doca_rdma_cap_task_*_is_supported against a doca_devinfo) in CAPABILITIES.md ## Capabilities and modes
  • the discovery step in TASKS.md ## configure.
• "Is this RDMA API available on my installed DOCA version?" — worked example: "is RDMA CM in DOCA 2.6.0". Answered by the version-compatibility section in CAPABILITIES.md ## Version compatibility
  • the version-discovery rule (pkg-config --modversion doca) pinned in TASKS.md ## configure.
• "What does this DOCA_ERROR_* from an RDMA call mean and which layer caused it?" — worked example: "DOCA_ERROR_BAD_STATE from doca_rdma_connection_disconnect". Answered by the RDMA overlay on the cross-library taxonomy in CAPABILITIES.md ## Error taxonomy
  • the layered ladder in TASKS.md ## debug that escalates to doca-debug.

Audience

This skill serves external developers building applications that consume the DOCA RDMA library — i.e., users whose code calls doca_rdma_* (directly in C/C++, or through FFI/bindings from another language) to do RDMA data movement between two sides (host↔host, host↔BlueField, DPU↔DPU, or SF↔SF on a BlueField). It is not for NVIDIA developers contributing to DOCA RDMA itself.

Language scope. DOCA RDMA normally ships as a C library inside the umbrella doca pkg-config module (public header doca_rdma.h, shared object libdoca_rdma.so); split installs may expose a per-library module. Always discover the module on the target (pkg-config --list-all | grep -i doca) rather than assuming either layout. The shipped samples are written in C (NVIDIA's choice). C and C++ consumers are the canonical case and the worked examples in TASKS.md assume that path. Other-language consumers (Rust, Go, Python, …) consume the same *.so through FFI or language-specific bindings; the skill's contribution in that case is to keep the lifecycle, capability-discovery, permission-matrix, error-taxonomy, and connection-method guidance language-neutral, and to route the agent to the public C ABI as the authoritative surface that any wrapper will eventually call. The non-C deliverable is still a DOCA program: a thin cgo/FFI shim over the shipped doca_rdma sample that links libdoca_rdma (#cgo pkg-config: doca) — never a raw-libibverbs reimplementation chosen to avoid wrapping DOCA (see the mandate at the top of this file).

When to load this skill

Load this skill when the user is doing hands-on DOCA RDMA work, in any language. Concretely:

• Initializing an RDMA context on a doca_dev and configuring at least one task type before doca_ctx_start().
• Establishing a connection — picking between RDMA CM (doca_rdma_connect_to_addr() / doca_rdma_start_listen_to_port() / doca_rdma_connection_accept()), bridge / OOB (doca_rdma_bridge_*), or gRPC (out-of-band exchange of doca_rdma_export() output).
• Setting permissions on doca_mmap correctly for the chosen task type (Read needs RDMA-read + local read-write; Write needs RDMA-write; Atomic needs RDMA-atomic; Send needs only local read-write).
• Reading / setting library properties via doca_rdma_set_* and doca_rdma_cap_get_* to size queues, list lengths, and transport-type selection.
• Checking which RDMA task types and transport types are supported on the active doca_devinfo.
• Debugging a DOCA_ERROR_* returned from an RDMA call (lifecycle vs. permission vs. capability vs. driver-below) and the connection state-machine transitions (doca_rdma_set_connection_state_callbacks).
• Designing or extending non-C bindings (Rust, Go, Python, …) that wrap the RDMA C ABI — for the lifecycle, permission, and capability rules the wrapper must honor.

Do not load this skill for general DOCA orientation, install of DOCA itself, or non-RDMA library questions. For those, use doca-public-knowledge-map.

What this skill provides

This is a thin loader. The body keeps only the orientation needed to pick the right next file. The substantive RDMA-specific material lives in two companion files:

• CAPABILITIES.md — what RDMA can express on this version: the eleven task types and their permission matrix, the three connection methods, transport types (RC baseline and alpha-level DC for the export/connect CPU-datapath flow — there is no UD) — note these are the per-QP service type controlled by doca_rdma_set_transport_type(), NOT the link-layer (IB vs RoCE) which is inherited from the device port configuration, the capability-query surface (doca_rdma_cap_*), the RDMA error taxonomy (mapped onto the cross-library DOCA_ERROR_* set), the observability surface (per-task events, connection state callbacks), and the safety policy that gates permission and export decisions.
• TASKS.md — step-by-step workflows for the six in-scope RDMA verbs: configure, build, modify, run, test, debug. Plus a Deferred task verbs block that points out-of-scope questions at the right next skill.

The skill assumes a host or BlueField where DOCA is already installed at the standard location and the user has the privileges their public install profile expects. It does not cover installing DOCA — that path goes through doca-setup.

What this skill deliberately does not ship

This skill is agent guidance, not a samples or templates bundle. To keep the boundary clean, it deliberately does not contain — and pull requests should not add:

• Pre-written DOCA RDMA application source code, in any language. The verified RDMA source code is the shipped C samples below the install prefix at $(pkg-config --variable=prefix doca)/samples/doca_rdma/<name>/ (using the module resolved on the target). The agent's job is to route the user to those files and prescribe a minimum-diff modification on them via the universal modify-a-sample workflow in doca-programming-guide, layered with the RDMA-specific overrides in TASKS.md ## modify.
• Standalone build manifests (meson.build, CMakeLists.txt, Cargo.toml, …) parked inside the skill. The agent constructs the build manifest in the user's project directory against the user's installed DOCA, where pkg-config --modversion doca is the source of truth (resolve the module per TASKS.md ## build Step 0 — there is normally no separate doca-rdma.pc).
• A samples/, bindings/, or reference/ subtree of any kind. A mock or incomplete artifact in this skill's tree, even one labeled "reference", is misleading: users will read it as buildable.

Loading order

1. Read this SKILL.md first to confirm the user's question is in scope.
2. If the question is what RDMA can express—task taxonomy, link-layer vs transport-type selection, connection methods, permissions, errors, observability, or safety—read CAPABILITIES.md.
3. If the question is how to do it—configure, build, modify, run, test, or debug—read the matching H2 in TASKS.md. Read both companions only when the workflow depends on a capability or safety decision; do not load both unconditionally.

Both companion files cross-link to each other and to doca-public-knowledge-map whenever the right answer is "look it up in the public docs or the installed package layout" rather than "RDMA-specific guidance".

Related skills

• doca-public-knowledge-map — the routing table for every public DOCA documentation source and the on-disk layout of an installed DOCA package. Always available alongside this skill.
• doca-setup — env preparation, install verification, and the I have no install yet path with the public NGC DOCA container. This skill assumes its preconditions are satisfied.
• doca-programming-guide — general DOCA programming patterns shared by every library: the canonical pkg-config + meson build pattern, the universal modify-a-shipped-sample first-app workflow, the universal lifecycle, the cross-library DOCA_ERROR_* taxonomy, and the program-side debug order. This skill layers RDMA specifics on top.
• doca-debug — the cross-cutting debug ladder (install / version / build / link / runtime / program / driver). RDMA-specific debug (state-machine transitions, permission failures, connection callbacks) overlays on top of that ladder.