Cluster Requirements

These requirements define the minimum a supplier-brought GPU cluster must meet to engage with the SF Compute Marketplace.

Every requirement is tagged with a Tier:

• Core — a hard requirement the cluster must meet.
• Nice-to-have — scored; a miss may be accepted as a tradeoff.

Requirements not met on a particular cluster are reviewed manually before acceptance. Acceptance is verified by SF Compute's validation suite (§10), which the cluster must pass. Contracts reference this document by version.

1. GPU server#

H100 / H200 HGX#

B200 HGX#

B300 HGX#

The BOM must state the chassis manufacturer, not just the GPU SKU. Every chassis must be an NVIDIA HGX reference platform and support bulk (non-GUI, non-USB) BIOS/firmware updates (§9). Off-list chassis — including the "upon review" entries — are accepted only after manual validation (§10) and a firmware-manageability review (§9).

Dell model numbers ending in "L" (e.g. XE9680L, XE9780L) are Dell's direct-liquid-cooled variants, listed for BOM accuracy — not a cooling mandate (see the Cooling row below).

Server cooling, network, storage, OS#

All NICs / HCAs must be SR-IOV-capable for VM passthrough (see §9).

2. Cluster networking#

A separate storage network is not required — network-attached storage rides the in-band / north-south fabric. Storage sizing lives in §4.

Fabric QoS is not a gate: intra-cluster ECN/PFC tuning for RoCE is a per-deployment conversation, not a listing requirement.

3. Management compute#

These nodes run SF Compute's in-DC control plane (k8s control-plane + workers, UFM proxy, storage services); quorum services run on an odd subset. State both the node count and per-node RAM — undersized control nodes (e.g. 16 GB) do not qualify.

4. Network-attached storage#

State the NAS solution and its usable (not raw) capacity: high-performance filesystems (VAST/WEKA) only fill to ~80–90% before performance degrades, and VAST's minimum deployment is ~1 PB. Throughput is recorded against per-GPU reference tiers, not held as an SLA.

5. Access & permissions#

Why we ask for BMC access#

SF Compute is first-line support for nodes sold on the Marketplace and needs BMC access for timely service and security at scale. Specifically, it is used for:

1. Console output — view console output, especially during boot, when the network is unavailable, devices fail to enumerate, or the host OS fails to load.
2. Programmatic rebooting — reboot a node via IPMI without a support ticket.
3. Host OS re-imaging — re-image nodes to keep them current and to reset between tenants.
4. Security verification — confirm BIOS configurations are secure and the latest firmware is installed.
5. Performance optimization — tune BIOS parameters for optimal performance.

To keep this access responsible, we recommend the provider place SF Compute on a separate management VLAN, issue separate credentials, secure the BMC behind a VPN, and allowlist the IPs permitted onto the management network. In return, SF Compute will notify before persistent BIOS changes, request approval before flashing firmware/BIOS, and provide an export of the original BIOS settings so they can be restored.

6. Outside connectivity#

7. Data center & environment#

Facility detail (power density, cooling, certifications, physical security) lives in the Colo Requirements spec; this is the cluster-side summary.

8. Support#

9. Machine state & provisioning configuration#

These settings let SF Compute provision, virtualize, and operate the cluster on bare metal. A node can pass every §10 performance test and still be unusable if these are not met.

Exact BIOS values, version floors, and provisioning steps are validated internally and aren't part of this spec — the rows below state the capability required, not the recipe.

10. Performance, burn-in & acceptance#

Suppliers must provide proof of a successful burn-in, or allow time for SF Compute to run one. SF Compute validates before delivery and does not rely on a vendor's attestations alone. Burn-in detects hardware faults (bad GPUs, marginal optics, weak DIMMs, miscabled rails) while remediation is still the supplier's responsibility, and establishes a performance baseline.

Sev-1 criteria are Core (a node/cluster must pass them). Sev-2 criteria are scored (Nice-to-have) — a miss is accepted only with a documented disposition.

Methodology#

Burn-in runs in two tiers, with continuous monitoring across both (XID events, PCIe AER events, IB counter deltas, thermal/power throttling, node liveness) that can fail either tier independently. Cluster-wide burn-in does not begin until per-host burn-in is clean. The cluster-wide tier is skipped for single-node / VM instances (no shared fabric to validate).

Per-host acceptance#

Tooling: DCGM, gpu-burn, stressapptest, stress-ng, memtester, fio.

Cluster-wide acceptance#

Tooling: nccl-tests, ClusterKit / perftest (ib_write_bw, ib_read_bw). Soak: NCCL all-reduce loop (default) or Megatron-LM / HPL LINPACK, 24 h – 5 days (default 48 h).

GPU count and SKU, host RAM, NVMe, NIC count/speed/model, and firmware must match the PO and BOM (mismatches are Sev-1). The firmware baseline (BIOS, BMC, GPU VBIOS) is pinned and recorded per node as an acceptance artifact.

Per-node reference tests#

A node that passes all of the following is considered working. The NCCL and DCGM checks are identical across SKUs; the NVLink and InfiniBand bandwidth targets vary by SKU (see the reference table below).

• Single-node NCCL — on every node:

  NCCL_DEBUG=INFO python3 -m torch.distributed.run --standalone --nproc_per_node=8 nccl-test.py

• Multi-node NCCL — concurrently across nodes (rendezvous on node 0):

  NCCL_DEBUG=INFO python3 -m torch.distributed.run --nproc_per_node 8 --nnodes NUM_NODES \
    --rdzv-backend c10d --rdzv-endpoint HOST-0:8888 nccl-test.py

• NVLink P2P bandwidth — per-SKU target below, via p2pBandwidthLatencyTest (P2P enabled).
• InfiniBand inter-node bandwidth — per-SKU target below, via ib_write_bw -d mlx5_0 --use_cuda 0 -a --report_gbits.
• DCGM — sudo dcgmi diag -r 4 passes.

Line rates are firm. Measured NVLink P2P and real IB throughput are references (fleet-confirmed), not hard SLAs.

Deliverables & sign-off#

On completion the customer receives: a per-node / per-GPU hardware inventory matched against the PO; a tooling manifest (image digests, driver/CUDA/OFED versions, pinned firmware baseline); and a signed acceptance certificate referencing this document by version. Acceptance is achieved when all Sev-1 (Core) criteria pass on all nodes, all Sev-2 findings have a documented disposition (remediated, waived with reason, or scheduled), and the customer countersigns. A Sev-1 failure triggers remediation and a re-run of the affected tier; remediation is the supplier's responsibility during burn-in.

11. Documentation#

12. Customer experience & accepted contracts#

An Accepted Contract requires that the GPU node is paid for, provisioned for customer access, and confirmed fully operational and usable — per our Marketplace Terms of Service.

Appendix — Hardware specification questionnaire#

Suppliers provide the following alongside the BOM during onboarding. SF Compute validates the responses — this is an intake, not a self-certification.

• General — BOM; rack elevation; network diagram.
• Hardware — access to non-GPU nodes (and whether VMs, with specs); shared storage filesystem (type, quota, access method); tested virtualization (and GPU passthrough); per-node dedicated public IP (static/dynamic, or how inbound is handled).
• Ingress / egress & networking — datacenter/location; total cluster bandwidth; number of uplinks; bandwidth shared vs. dedicated; measured speed to S3 us-west-1 and nearest S3 zone; public IP per machine.
• Access & OOB — direct OOB/BMC access; BMC portal reboots / console / BIOS / reimaging.
• Responsibilities — who configures InfiniBand partitioning / UFM / SM (and whether SM virtualization can be enabled); who configures Ethernet partitioning/isolation (and method); who configures BIOS; who handles OS install, driver install, and firmware versioning.