How do servers work in terms of CPUs and motherboards?

Let's assume we're running Windows Server 2016 Data Center edition, which supports up 64 individual CPUs.

Obviously there are no motherboards, consumer or server grade, that have anywhere near 64 CPU sockets.

So how does that work? Do servers link motherboards together? Would all of the 64 CPUs run as a single operating system on WS2016DC? Would hyper-threading across all of [potentially] thousands of threads be possible if applications were designed for it? Would that single operating system have access to all of the systems RAM (as much as 24TB) across all of those motherboards at once?

Thanks!

2 Answers

Relevance
  • YKhan
    Lv 7
    2 years ago
    Favorite Answer

    Yeah, I don't think there's much more than 4-socket server motherboards available from anyone. But they can link several motherboards together over a high-speed link (usually Infiniband, or something proprietary) inside a cabinet. You can link 16 boards with 4 sockets on them together, and you got your 64 sockets right there; or 32 boards with 2 sockets.

    However, no matter how fast the interboard connects are, they no where nearly as fast as the intraboard connects. Those in turn are nowhere near as fast as intraprocessor connects. So Windows Server sets them up in a NUMA kernel (Non-Uniform Memory Access), where there is a hierarchy of speed, and Windows tries to keep as much related stuff as close as possible to the fastest paths. So it will attempt to keep as much stuff operating on the same core. If it can't keep them on the same core, then it will attempt to keep them on the same processor in an adjacent core. If it can't be kept on the same processor, then it will attempt to put it on an adjacent processor on the same board. And finally, if all else fails, it will try to put things on a different board's processor.

    Comparison of Standard and Datacenter editions of Windows Server 2016 – Microsoft 365 Powered Devices Australia

    https://blogs.technet.microsoft.com/ausoemteam/201...

    • YKhan
      Lv 7
      2 years agoReport

      Each copy of the OS probably communicates with each other to maintain licensing restrictions.

  • 2 years ago

    Each core in a multi-core CPU is functionally a separate CPU; eight single-core devices or one eight-core device both count as "Eight CPUs" in overall function.

    As an example, this board will take four, 16 core CPUs making a 64 core system on a single board.

    https://www.supermicro.com/Aplus/motherboard/Opter...

    In terms of server operating system terms, windows is a toy...

    Serious systems generally use a Unix or Linux type operating system - eg RedHat is a common one.

    The Cray Titan XK7 is a good example of a large scale system; that has 299008 CPU cores and runs a version of Linux.

    (18,688 16 core AMD Opterons, the same as could fit the board above).

    https://www.cray.com/blog/titan-supercomputer-more...

Still have questions? Get your answers by asking now.