NAS Parallel Benchmarks (NPB) are a set of
benchmark
Benchmark may refer to:
Business and economics
* Benchmarking, evaluating performance within organizations
* Benchmark price
* Benchmark (crude oil), oil-specific practices
Science and technology
* Benchmark (surveying), a point of known elevati ...
s targeting performance evaluation of highly
parallel
Parallel is a geometric term of location which may refer to:
Computing
* Parallel algorithm
* Parallel computing
* Parallel metaheuristic
* Parallel (software), a UNIX utility for running programs in parallel
* Parallel Sysplex, a cluster of IBM ...
supercomputer
A supercomputer is a computer with a high level of performance as compared to a general-purpose computer. The performance of a supercomputer is commonly measured in floating-point operations per second ( FLOPS) instead of million instructions ...
s. They are developed and maintained by the
NASA
The National Aeronautics and Space Administration (NASA ) is an independent agency of the US federal government responsible for the civil space program, aeronautics research, and space research.
NASA was established in 1958, succeeding t ...
Advanced Supercomputing (NAS) Division (formerly the NASA Numerical Aerodynamic Simulation Program) based at the
NASA Ames Research Center
The Ames Research Center (ARC), also known as NASA Ames, is a major NASA research center at Moffett Federal Airfield in California's Silicon Valley. It was founded in 1939 as the second National Advisory Committee for Aeronautics (NACA) laborat ...
. NAS solicits performance results for NPB from all sources.
History
Motivation
Traditional benchmarks that existed before NPB, such as the
Livermore loops, the
LINPACK Benchmark
The LINPACK Benchmarks are a measure of a system's Floating-point arithmetic, floating-point computing power. Introduced by Jack Dongarra, they measure how fast a computer solves a dense ''n'' by ''n'' system of linear equations ''Ax'' =&nbs ...
and th
NAS Kernel Benchmark Program were usually specialized for vector computers. They generally suffered from inadequacies including parallelism-impeding tuning restrictions and insufficient problem sizes, which rendered them inappropriate for highly parallel systems. Equally unsuitable were full-scale application benchmarks due to high porting cost and unavailability of automatic software parallelization tools.
As a result, NPB were developed in 1991
and released in 1992
to address the ensuing lack of benchmarks applicable to highly parallel machines.
NPB 1
The first specification of NPB recognized that the benchmarks should feature
* new parallel-aware algorithmic and software methods,
* genericness and architecture neutrality,
* easy verifiability of correctness of results and performance figures,
* capability of accommodating new systems with increased power,
* and ready distributability.
In the light of these guidelines, it was deemed the only viable approach to use a collection of "paper-and-pencil" benchmarks that specified a set of problems only algorithmically and left most implementation details to the implementer's discretion under certain necessary limits.
NPB 1 defined eight benchmarks, each in two problem sizes dubbed ''Class A'' and ''Class B''. Sample codes written in
Fortran 77 were supplied. They used a small problem size ''Class S'' and were not intended for benchmarking purposes.
NPB 2
Since its release, NPB 1 displayed two major weaknesses. Firstly, due to its "paper-and-pencil" specification, computer vendors usually highly tuned their implementations so that their performance became difficult for scientific programmers to attain. Secondly, many of these implementation were proprietary and not publicly available, effectively concealing their optimizing techniques. Secondly, problem sizes of NPB 1 lagged behind the development of supercomputers as the latter continued to evolve.
NPB 2, released in 1996,
came with source code implementations for five out of eight benchmarks defined in NPB 1 to supplement but not replace NPB 1. It extended the benchmarks with an up-to-date problem size ''Class C''. It also amended the rules for submitting benchmarking results. The new rules included explicit requests for output files as well as modified source files and build scripts to ensure public availability of the modifications and reproducibility of the results.
NPB 2.2 contained implementations of two more benchmarks.
NPB 2.3 of 1997 was the first complete implementation in
MPI.
It shipped with serial versions of the benchmarks consistent with the parallel versions and defined a problem size ''Class W'' for small-memory systems.
NPB 2.4 of 2002 offered a new MPI implementation and introduced another still larger problem size ''Class D''.
It also augmented one benchmark with
I/O-intensive subtypes.
NPB 3
NPB 3 retained the MPI implementation from NPB 2 and came in more flavors, namely
OpenMP
OpenMP (Open Multi-Processing) is an application programming interface (API) that supports multi-platform shared-memory multiprocessing programming in C, C++, and Fortran, on many platforms, instruction-set architectures and operating syste ...
,
Java
Java (; id, Jawa, ; jv, ꦗꦮ; su, ) is one of the Greater Sunda Islands in Indonesia. It is bordered by the Indian Ocean to the south and the Java Sea to the north. With a population of 151.6 million people, Java is the world's List ...
and
High Performance Fortran High Performance Fortran (HPF) is an extension of Fortran 90 with constructs that support parallel computing, published by the ''High Performance Fortran Forum'' (HPFF). The HPFF was convened and chaired by Ken Kennedy of Rice University. The fi ...
.
These new parallel implementations were derived from the serial codes in NPB 2.3 with additional optimizations.
NPB 3.1 and NPB 3.2 added three more benchmarks,
which, however, were not available across all implementations; NPB 3.3 introduced a ''Class E'' problem size.
Based on the single-zone NPB 3, a set of multi-zone benchmarks taking advantage of the MPI/OpenMP hybrid programming model were released under the name NPB-Multi-Zone (NPB-MZ) for "testing the effectiveness of multi-level and hybrid parallelization paradigms and tools".
The benchmarks
As of NPB 3.3, eleven benchmarks are defined as summarized in the following table.
References
{{Reflist
External links
NAS Parallel Benchmarks Changes(official website)
Supercomputer benchmarks