Benchmark developed by Mike Heroux gains adherents

The fourth semi-annual supercomputer ratings determined by a software program designed and developed by Mike Heroux, in collaboration with Jack Dongarra and Piotr Luszczek from the University of Tennessee, continues to gain traction in the high-performance computing community. The new ratings were released Nov. 18 at SC15, one of the world’s premier supercomputing meetings.

The emerging tool, termed the High Performance Conjugate Gradients (HPCG) benchmark, was executed on more than 60 supercomputer entries for the current round, up from the initial 15 competitors 18 months ago.

“HPCG is designed to complement the traditional High Performance Linpack (HPL) benchmark used as the official metric for ranking the top 500 systems,” says Mike. “Currently, the pair of numbers provided by HPCG and HPL act as bookends on the performance spectrum of a given system.”

While HPL tests supercomputer speed in solving relatively straightforward problems, HPCG’s more complex criteria test characteristics such as high-performance interconnects, memory systems, and fine-grain cooperative threading that are important to a broad set of applications.

A lot of work ahead

“Many benchmarks have been proposed as complements or even replacements for Linpack,” says Mike. “We have had more success than previous efforts. But there is still a lot of work to keep the effort going.”

The current list contains the same entries as many of the top 50 systems from Linpack’s TOP500 but exhibits a significant shuffling of HPL rankings, indicating that HPCG features are stressing different system characteristics.

Asked what causes supercomputer manufacturers and users to gravitate toward HPCG as a useful test, Mike says, “Only time will tell. All major vendor computing companies have invested heavily in optimizing our benchmark. All participating system owners have dedicated machine time to make runs. These investments are the strongest confirmation that we have developed something useful.”

As a case in point, the new Los Alamos/Sandia supercomputer Trinity, listed sixth fastest in the current Linpack ratings, was bumped up to fourth by HPCG.

Says Mike, “I believe this indicates that HPCG is a better representative for workloads that NNSA cares about. It also confirms the NNSA choice to build a system — Trinity — that is more about balance than getting the best HPL result.”