The “Ice Lake SP” CPUs are the 3rd generation of Intel’s Xeon Scalable Processor family. This generation brings new features, increased performance, and new server/workstation platforms. The Xeon ‘Ice Lake SP’ CPUs cannot be installed into previous-generation systems. Those considering a new deployment are encouraged to review with one of our experts.

Highlights of the features in Xeon Scalable Processor Family “Ice Lake SP” CPUs include:

• Up to 40 processor cores per socket (with options for 8-, 12-, 16-, 18-, 20-, 24-, 26-, 28-, 32-, 36-, and 38-cores)
• Up to 38% higher per-core performance through micro-architecture improvements (at same clock speed vs “Cascade Lake SP”)
• Significant memory performance & capacity increases:
  • Eight-channel memory controller on each CPU (up from six)
  • Support for DDR4 memory speeds up to 3200MHz (up from 2933MHz)
  • Large-memory capacity with Intel Optane Persistent Memory
  • All CPU models support up to 6TB per socket (combined system memory and Optane persistent memory)
• Increased link speed between CPU sockets: 11.2GT/s UPI links (up from 10.4GT/s)
• I/O Performance Improvements – more than twice the throughput of “Cascade Lake SP”:
  • PCI-Express generation 4.0 doubles the throughput of each PCI-E lane (compared to gen 3.0)
  • Support for 64 PCI-E lanes per CPU socket (up from 48 lanes)
• Continued high performance with the AVX-512 instruction capabilities of the previous generation:
  • AVX-512 instructions (up to 16 double-precision FLOPS per cycle per AVX-512 FMA unit)
  • Two AVX-512 FMA units per CPU core (available in all Ice Lake-SP CPU SKUs)
• Continued support for deep learning inference with AVX-512 VNNI instruction:
  • Intel Deep Learning Boost (VNNI) provides significant, more efficient deep learning inference acceleration
  • Combines three AVX-512 instructions (VPMADDUBSW, VPMADDWD, VPADDD) into a single VPDPBUSD operation
• Improvements to Intel Speed Select processor configurability:
  • Performance Profiles: certain processors support three distinct core count/clock speed operating points
  • Base Frequency: specific CPU cores are given higher base clock speeds; the remaining cores run at lower speeds
  • Turbo Frequency: specific CPU cores are given higher turbo-boost speeds; the remaining cores run at lower speeds
  • Core Power: each CPU core is prioritized; when surplus frequency is available, it is given to high-priority cores
• Integrated hardware-based security improvements and total memory encryption

With a product this complex, it’s very difficult to cover every aspect of the design. Here, we concentrate primarily on the performance of the processors for HPC & AI applications.

Continued Specialization of Xeon CPU SKUs

Those already familiar with Intel Xeon will see this processor family is divided into familiar tiers: Silver, Gold, and Platinum. The Silver and Gold models are in the price/performance range familiar to HPC/AI teams. Platinum models are in a higher price range. The low-end Bronze tier present in previous generations has been dropped.

Further, Intel continues to add new specialized CPU models that are optimized for particular workloads and environments. Many of these specialized SKUs are not relevant to readers here, but we summarize them briefly:

• N: network function virtualization (NFV) optimized
• P: virtualization-optimized (with a focus on clock frequency)
• S: max SGX enclave size
• T: designed for higher-temperature environments (NEBS)
• V: virtualization-optimized (with focus on high-density/low-power)

Targeting specific workloads and environments provides the best performance and efficiency for those use cases. However, using these CPUs for other workloads may reduce performance, as the CPU clock frequencies and Turbo Boost speeds are guaranteed only for those specific workloads. Running other workloads on these optimized CPUs will likely lead to CPU throttling, which would be undesirable. Considering these limitations, the above workload-optimized models will not be included in our review.

Four Xeon CPU specializations relevant to HPC & AI use cases

There are several specialized Xeon CPU options which are relevant to high performance computationally-intensive workloads. Each capability is summarized below and included in our analysis.

• Liquid-cooled – Xeon 8368Q CPU: optimized for liquid-cooled deployment, this CPU SKU offers high core counts along with higher CPU clock frequencies. The high clock frequencies are made possible only through the more effective cooling provided by liquid-cooled datacenters.
• Media, AI, and HPC – Xeon 8352M CPU: optimized for AVX-heavy vector instruction workloads as found in media processing, AI, and HPC; this CPU SKU offers improved performance per watt.
• Performance Profiles – Y: a set of CPU SKUs with support for Intel Speed Select Technology – Performance Profiles. These CPUs are indicated with a Y suffix in the model name (e.g., Xeon 8352Y) and provide flexibility for those with mixed workloads. Each CPU supports three different operating profiles with separate CPU core count, base clock and turbo boost frequencies, as well as operating wattages (TDP). In other words, each CPU could be thought of as three different CPUs. Administrators switch between profiles via system BIOS, or through Operating Systems with support for this capability (Intel SST-PP). Note that several of the other specialized CPU SKUs also support multiple Performance Profiles (e.g., Xeon 8352M).
• Single Socket – U: single-socket optimized. The CPUs designed for a single socket are indicated with a U suffix in the model name (e.g., Xeon 6312U). These CPUs are more cost-effective. However, they do not include UPI links and thus can only be installed in systems with a single processor.

Summary of Xeon “Ice Lake-SP” CPU tiers

With the Bronze CPU tier no longer present, all models in this CPU family are well-suited to HPC and AI (though some will offer more performance than others). Before diving into the details, we provide a high-level summary of this Xeon processor family:

• Intel Xeon Silver – suitable for entry-level HPC
  The Xeon Silver 4300-series CPU models provide higher core counts and increased memory throughput compared to previous generations. However, their performance is limited compared to Gold and Platinum (particularly on Core Count, Clock Speed, Memory Performance, and UPI speed).
• Intel Xeon Gold – recommended for most HPC workloads
  Xeon Gold 5300- and 6300-series CPUs provide the best balance of performance and price. In particular, the 6300-series models should be preferred over the 5300-series models, because the 6300-series CPUs offer improved Clock Speeds and Memory Performance.
• Intel Xeon Platinum – only for specific HPC workloads
  Although 8300-series models provide the highest performance, their higher price makes them suitable only for particular workloads which require their specific capabilities (e.g., highest core count, large L3 cache).

Xeon “Ice Lake SP” Computational Performance

With this new family of Xeon processors, Intel once again delivers unprecedented performance. Nearly every model provides over 1 TFLOPS (one teraflop of double-precision 64-bit performance per second), many models exceed 2 TFLOPS, and a few touch 3 TFLOPS. These performance levels are achieved through high core counts and AVX-512 instructions with FMA (as in the first and second Xeon Scalable generations). The plots in the tabs below compare the performance ranges for these new CPUs:
[tabby title=”AVX-512 Instruction Performance”]

[tabby title=”AVX2 Instruction Performance”]

[tabbyending]

In the charts above, the shaded/colored bars indicate the expected performance range for each CPU model. The performance is a range rather than a specific value, because CPU clock frequencies scale up and down on a second-by-second basis. The precise achieved performance depends upon a variety of factors including temperature, power envelope, type of cooling technology, the load on each CPU core, and the type(s) of CPU instructions being issued to each core.

The first tab shows performance when using Intel’s AVX-512 instructions with FMA. Note that only a small set of codes will be capable of issuing exclusively AVX-512 FMA instructions (e.g., HPL LINPACK). Most applications issue a mix of instructions and will achieve lower than peak FLOPS. Further, applications which have not been re-compiled with an appropriate compiler will not include AVX-512 instructions and thus achieve lower performance. Computational applications which do not utilize AVX-512 instructions will most likely utilize AVX2 instructions (as shown in the second tab with AVX2 Instruction performance.

Intel Xeon “Ice Lake SP” Price Ranges

The pricing of the 3rd-generation Xeon Processor Scalable Family spans a wide range, so budget must be kept in mind when selecting options. It would be frustrating to plan on 38-core processors when the budget cannot support a price of more than $10,000 per CPU. The plot below compares the prices of the Xeon “Cascade Lake SP” processors:

As shown in the above plot, the CPUs in this article have been sorted by tier and by price. Most HPC users are expected to select CPU models from the Gold Xeon 6300-series. These models provide close to peak performance for a price around $3,000 per processor. Certain specialized applications will leverage the Platinum Xeon 8300-series

To ease comparisons, all of the plots in this article are ordered to match the above plot. Keep this pricing in mind as you review this article and plan your system architecture.

Recommended Xeon CPU Models for HPC & AI/Deep Learning

As stated at the top, most of this new CPU family offers excellent performance. However, it is common for HPC sites to set a minimum floor on CPU clock speeds (usually around 2.5GHz), with the intent that no workload suffers too low of a performance. While there are users who would demand even higher clock speeds, experience shows that most groups settle on a minimum clock speed in the 2.5GHz to 2.6GHz range. With that in mind, the comparisons below highlight only those CPU models which offer 2.5+GHz performance.

[tabby title=”2.5+GHz Core Counts”]

[tabby title=”AVX-512 Performance”]

[tabby title=”AVX2 Performance”]

[tabby title=”2.5+GHz Cost-Effectiveness”]

[tabbyending]

The post Detailed Specifications of the “Ice Lake SP” Intel Xeon Processor Scalable Family CPUs appeared first on Microway.

An extra depth chassis is not always required, though.Workarounds can be implemented in order to fit an extra depth chassis into a regular depth cabinet, but an uncommon cabinet configuration would be required.

Common Extra Depth Systems

Some common extra depth systems are described in Table 1.All of the systems described in the table are GPU systems, having socketed or PCIe Tesla GPUs.

Table 1. Common extra depth systems

Commonly used extra depth cabinets are described in Table 2, ranging in height from 42U to 48U. Extra depth cabinets are the easiest solution for rackmounting extra depth systems.There are workarounds, however, which can be implemented for instances where a customer already has a regular depth cabinet on-site, and would prefer to use the existing cabinet, due possibly, to scarcity of unused floor space.

Table 2. Common extra depth cabinets

Workarounds Solutions

Workaround solutions for mounting extra depth systems into regular depth cabinets require extra considerations and conditions.

Workaround Solution #1: Remove any vertical PDUs, and replace them with horizontal PDUs

An extra depth system would be obstructed by full height vertical, “zero U”, PDUs, as it is slid toward the back of the cabinet, preventing it from sliding fully into the cabinet.All full height vertical PDUs must be removed from the cabinet, and replaced with horizontal PDUs.Because extra depth systems usually have a secondary system board, with socketed GPUs, they are usually power-dense.The NVIDIA DGX-1 GPU-accelerated system for deep learning, for example, requires 3.5kW of power, at peak workload.Tri-phase power is recommended, whenever possible, for power-dense GPU systems.With some power-dense configurations, it will not be possible to meet peak power requirements with single-phase power.

Along with their unusually high power density, extra depth systems will require high airflow.Using the NVIDIA DGX-1 again as an example, four chassis fans will each produce a maximum of 340 CFM of air flow, for a total of 1,360 CFM.For groups of systems having high air flow requirements, the cabinet doors must be perforated.

Selecting a tri-phase horizontal PDU can be a challenge, since they do not present as many outlets as vertically mounted PDUs, and will sometimes present an outlet type which is not compatible with the inlet type on the system(s).If the entire cabinet will not be needed for mounting extra depth systems, then using a half height PDU, possibly in addition to a horizontal PDU, may be a good choice.

APC currently offers only one tri-phase horizontal PDU. Geist offers a wide variety of tri-phase horizontal PDU types.These can be searched using the Geist PDU finder. Geist does not offer a tri-phase PDU, for use with a 208V, 20Amp source.However, it offers a variety of horizontal, tri-phase PDUs, for lines carrying 30Amps, or more.Like Geist, Server Technology offers a range of tri-phase, horizontal PDUs.They offer a tri-phase PDU model which can be used with a 208V, 20Amp power source, compatible with the NEMA L21-20P plug type.

Workaround Solution #2: Replace any full height vertical PDUs with half height PDUs

Half height PDUs are typically used for shorter cabinets.But they can also be used in regular height cabinets, to allow for installation of one or more extra depth system chassis.If the power receptacle is below the floor, then it will be easier to mount the PDU under the extra depth system(s), with the plug pointed downward.If the power receptacle is on the ceiling, then it will be easier to mount the PDU above the extra depth system(s), with the plug pointed upward.

Half height vertical PDUs should only be used if the entire cabinet will not be needed for mounting extra depth systems.This is because half height vertical PDUs will still prevent installation of extra depth systems into approximately half of the cabinet’s rackspace.Some half height PDUs can be mounted on the exterior of the cabinet frame, so that they will not obstruct extra depth systems from being rackmounted.

Figure 1. Half height vertical PDU (shown horizontally)

Workaround Solution #3: Use an Extra Wide, Regular Depth Cabinet

If there is an extra wide cabinet onsite, it could possibly be used to install an extra depth system.Extra wide cabinets provide sufficient width such that vertical PDUs, mounted at the sides, will not obstruct an extra depth chassis from sliding all the way to the back of the cabinet.Removal of cabinet rear doors may still be required, however, depending on the depth of the system and cabinet.If right angle power connectors are used, then, in some cases, removal of rear doors will not be required (e.g., DGX-1 in the AR3100 cabinet).Using right angle power connectors for connecting to the PDU itself may also allow more horizontal clearance for extra depth systems.In cases where the horizontal clearance is a bit narrow, an extra depth system could be positioned vertically into another rack position, so that it will not have to squeeze between plugs connected into vertical PDUs.Positioning the system vertically to correspond with the height of a meter LCD panel (on a metered PDU), for example, or between power banks (on most PDUs), would allow for more horizontal clearance, since plugs will not be encroaching upon horizontal clearance at these vertical positions.

Table 3. Some Common extra wide cabinets

Some PDU types are deeper than others, meaning the plugs will likely encroach further into the horizontal clearance, since the outlets on the PDU will be at a greater distance from the side of the cabinet.Some Raritan PDUs, for example, have more depth than some APC PDUs.

Workaround Solution #4: Mount Systems at Height Corresponding to Space between PDU Power Banks

As mentioned with workaround #3, it maybe possible to mount an extra depth system at a height so that it will not run into plugs connected to PDUs. This is possible only if the PDUs are sufficiently shallow and if the system is mounted at a height corresponding to the space between PDU power banks, where no plugs protrude.

For example, an IBM Power9 system (33.3″ depth) will still fit into an APC 3100 regular depth cabinet, with four vertical AP7541 PDUs installed at the back of the cabinet, as long as it is installed at a height between the PDU power banks.

The post Solutions for Rackmounting Extra Depth Systems appeared first on Microway.

Acoustic Considerations – Fan Noise

Microway’s server systems provide intelligent cooling fans which ramp up and down with system load. However, most systems are very high-density with more than 1,000 Watts consumed per rack unit. Cooling such systems requires high-speed fans which generate significant noise. Most cluster systems will generate 60 dB to 80 dB. This noise level may be subject to safety regulations (e.g., OSHA) and will be audible from other offices in the building. It is recommended that your cluster be placed in a location designed for servers.

If noise is a concern, many of Microway’s workstations – most notably the WhisperStation – are designed to be very quiet and comfortable for an office environment. Quiet HPC Clusters built from Microway WhisperStations are available. If you have any concerns, discuss these matters with your salesperson. In general, if it’s not a WhisperStation you should consider whether the system is appropriate for a lab or office environment.

Power

Microway servers feature auto-switching power supplies which accept both 120V and 208V power, so there are a variety of options when powering multiple computers. The most common electrical circuits are pictured below. Your salesperson can provide a customized recommendation including the rackmount cabinet, power distribution units and optional UPS backup power.

In general, 208V is recommended over 120V. Using 208V allows more systems to run on a single circuit, and each system runs several percent more efficiently.

120V 20A NEMA 5-20 and L5-20 Electrical Outlets

After a 20% de-rating for safety, these circuits supply up to 1,920 Watts of power.

120V 30A NEMA L5-30 Electrical Outlets

After a 20% de-rating for safety, these circuits supply up to 2,880 Watts of power.

208V 20A NEMA L6-20 Electrical Outlets

After a 20% de-rating for safety, these circuits supply up to 3,328 Watts of power.

208V 30A NEMA L6-30 Electrical Outlets

After a 20% de-rating for safety, these circuits supply up to 4,992 Watts of power.

3-Phase 208V NEMA L21-20 Electrical Outlets

After safety de-rating, these circuits supply up to 5,700 Watts of power.

3-Phase 208V CS 8365 Electrical Outlets

After safety de-rating, these circuits supply up to 14,400 Watts of power. Depending upon which PDU is selected, the actual load on each PDU may need to be less – 10kW, 12.6kW or 14.4kW.

Before assuming a particular circuit layout will be sufficient for your new equipment, review:

• Is other equipment already connected to the electrical circuit? It is common for multiple outlets to connect to the same circuit, so don’t assume that an empty outlet means power is available. You will need to perform an inventory of the power required by your existing equipment.
• Particularly for smaller circuits: will you be able to evenly split your servers across multiple circuits? Three 1,000W systems will not successfully connect to two 1,500W circuits. Determine how much electricians will charge for installation of multiple circuits, because it is likely that a single large circuit will be a better choice.

Cooling

Groups of computers typically require special cooling arrangements – your building air-conditioning is designed for offices and will not be able to keep up with the load of compute servers. Note that some facilities reduce or shut down air-conditioning during holidays and weekends. Systems will overheat if they are run in a closed room without sufficient cooling.

A server room or datacenter is the best location for your systems. Microway servers and clusters are designed for installation in industry-standard rackmount cabinets, so you should have no concern when using your own cabinets. We can also provide cabinets with network and power cables pre-wired.

Many facilities specify a maximum power load per rackmount cabinet. You may be restricted to only 7kW or 10kW per cabinet, which would result in racks which are only half-filled. To be certain, present the cooling requirements (provided by your salesperson) to your facilities manager. Your facilities personnel will have details on the cooling load limitations of the datacenter/server room.

The post Power and Cooling (Multiple-Computer Installations) appeared first on Microway.

Acoustic Considerations – Fan Noise

Most single computers can be powered and cooled within an office environment. However, some systems contain high-speed fans which generate loud noise. In some cases, an office worker may decide this is acceptable. In others, the noise may be subject to safety regulations (e.g., OSHA) and will be audible from other offices in the building.

Many of Microway’s workstations – most notably the WhisperStation – are designed to be very quiet and comfortable for an office environment. Other systems contain so many high-power components that they simply cannot be quietly cooled.

In contrast with other vendors, Microway does not re-brand servers as workstations. We will provide an honest assessment of system noise. If you have any concerns, discuss these matters with your salesperson. In general, if it’s not a WhisperStation you should consider whether the system is appropriate for an office environment.

Power

Most single Microway computers can be powered from a standard wall outlet. For groups of computers, there are better options than those listed below – see Power and Cooling for Multiple Computers.

Home / Small Office Electrical Outlets

These power outlets provide 15 Amps at 120V and are designated NEMA 5-15. After a 20% de-rating for safety, the NEMA 5-15 can supply up to 1440 Watts of power.

There are several cases when 15A 120V outlets are not sufficient:

• Other equipment is already connected to the electrical circuit. It is common for multiple outlets to connect to the same circuit, so don’t assume that an empty outlet means power is available. You will need to perform an inventory of the power required by your existing equipment.
• You would like to provide a backup power solution (UPS) and the system consumes more than 1000 Watts.
• Servers with more than four GPU accelerators or coprocessors will consume more than 1440 Watts.
• Your system is a Twin or Blade server which actually contains multiple computers. These systems almost always consume more than 1440 Watts.

Commercial Electrical Outlets

These power outlets provide 20 Amps at 120V and are designated NEMA 5-20. After a 20% de-rating for safety, the NEMA 5-20 can supply up to 1920 Watts of power.

This type of outlet is common in office buildings, medical centers and universities, but is not universal. Double-check to be certain your area actually has this type of power circuit. Also verify that it is not designated for special purposes, such as emergency power.

Although this outlet should be sufficient for almost all systems, consider:

• Other equipment may already be connected to the electrical circuit. It is common for multiple outlets to connect to the same circuit, so don’t assume that an empty outlet means power is available. You will need to perform an inventory of the power required by your existing equipment.
• Servers with more than four GPU accelerators or coprocessors may consume more than 1920 Watts.
• If this is a Blade chassis with multiple computers, it will consume more than 1920 Watts.

Cooling

Systems in this category can typically be run without special cooling arrangements – the building air-conditioning will be able to keep up with the load.

Note that some facilities reduce or shut down air-conditioning during holidays and weekends. Many systems will overheat if they are run in a closed room without sufficient cooling. You may need to make special arrangements.

To be certain, present the cooling requirements (provided by your salesperson) to your facilities manager. Your facilities personnel should be able to determine if issues will occur.

The post Power and Cooling (Single-Computer Installations) appeared first on Microway.