Evaluating 800Gb/s InfiniBand for AI Inference

An 800Gb/s interface belongs in a system plan. Before choosing it, establish which part of the inference workload waits for communication and whether the host can use the proposed connection effectively.

Start with a production-platform reference

CoreWeave’s March 2026 HGX B300 release names ConnectX-8, Quantum-X800 and BlueField-3. It reports a 1.95x collective-bandwidth result on DeepSeek-R1 versus its H200 platform. That is a whole-system comparison with multiple changes, not the measured gain from replacing a network card. CoreWeave’s B300 release

Use the report to identify a relevant architecture, then build a test around your own model and service objective.

Determine what needs to improve

For an inference service, separate time to first token, inter-token latency, end-to-end request latency and throughput at the agreed concurrency. Capture input and output lengths, precision, model version, parallelism settings and batching policy. A higher aggregate number can conceal a worse result for an interactive user.

Run a communication profile before changing hardware. Identify whether the limiting path is GPU memory, GPU scale-up communication, host I/O, storage, or the inter-node fabric. The result should explain why more scale-out bandwidth is relevant to this workload.

Qualify the endpoint and the switch together

The bill of materials should specify the complete adapter order code, server slot and power arrangement, operating protocol, switch platform, cable endpoints and supported software release. Confirm whether the offered assembly is the intended card form factor or a system-specific module.

For an InfiniBand proposal, ask the supplier to demonstrate link negotiation and collective tests on the proposed combination. A familiar connector or a matching nominal rate is not sufficient evidence of compatibility.

Build an acceptance run that can be repeated

Use identical models and serving parameters for the baseline and candidate system. Record warm-up policy and test duration. Compare performance at the target latency, then repeat with competing traffic and a controlled failure. Keep changes to compute and networking visible so the buyer can understand what the comparison actually measures.

The purchasing decision should include operational fit: monitoring, replacement procedures, driver support and expansion plans. A technically faster configuration can still be the wrong choice if the site cannot maintain it.

Product-scope boundary

The CoreWeave source identifies families, not the order codes in another supplier’s catalog. It does not prove deployment of a particular Q3400 or Q3450 switch variant. Family-level application evidence can support this selection discussion; exact hardware qualification and commercial availability must be verified separately.

Sources checked: 10 October 2026. This article discusses third-party evidence or an explicitly labeled engineering scenario; it does not establish a reseller’s delivery history, current stock or exact-SKU deployment.