Each node is a dual-socket 1U server carrying two 96-core EPYC 9654 processors and 768 GB of DDR5-4800 across all 24 memory channels, with a single NDR InfiniBand port. The nodes are identical on purpose. A job that runs on one runs on all of them, and a failed node swaps out without touching the image. Most simulation code waits on memory bandwidth, not clock speed. Twelve channels per socket give roughly 920 GB/s per node, which keeps stencil and sparse-matrix work fed where a desktop would starve it. Nodes boot stateless over the network, so the whole pool holds one identical image and grows by racking more of the same.

Spread one MPI job over 64 nodes and it's only as fast as the slowest message between them. The fabric is NDR InfiniBand, 400 Gb/s per port on NVIDIA Quantum-2 switches, wired as a non-blocking fat-tree so every node can talk to every other node at full rate at the same time. For tightly coupled work, latency matters more than raw bandwidth. End-to-end MPI latency lands near 1.1 microseconds, and RDMA moves data straight between node memories without waking the CPU. When a link gets congested, adaptive routing steers around it instead of stalling on it. Storage traffic rides the same fabric on its own partition, so a heavy write never starves the message passing.

A thousand cores writing checkpoints at once will flatten an ordinary file server. Storage here is a Lustre parallel filesystem: metadata on one tier, data striped across many NVMe object storage targets, all on the same InfiniBand fabric. A single large file is split across targets, so many nodes read and write it in parallel. That holds up past 40 GB/s read and 25 GB/s write, sustained. Every node sees one POSIX namespace at the same path, so job scripts never stage or copy data between local disks. Stripe count and size are tunable per directory. A handful of enormous files and millions of tiny ones can each be laid out the way they read best.

A cluster only stays trustworthy if every node is provably identical. Nodes network-boot a single Warewulf image held on redundant head nodes, so a reboot returns a node to a known-good state and a new node inherits it with no hand configuration. Slurm runs the queue from the head nodes, with a standby that takes over if the primary fails. Software reaches users through Spack-built stacks exposed as Lmod modules, so you load a named compiler and MPI version instead of fighting library paths. Everything a user sees, from the login shell to the module tree, is the same on the first node and the five hundredth.