The Standard Performance Evaluation Corp. (SPEC) is offering rewards of up to $9,000 and a free benchmark license for application code and datasets accepted under its benchmark search program. Applications that make it through the entire search program will be incorporated into the next major version of the SPEC CPU suite.
SPEC CPU benchmarks provide performance metrics for comparing systems on a known compute-intensive workload, with emphasis on the system's processor(s), memory hierarchy, and compiler(s). SPEC is expecting the next CPU benchmark suite to continue this trend. A good benchmark candidate would embody the following characteristics:
- Based on a representative application used in the real world.
- Compute bound, or can have its compute bound portion excerpted.
- Portable or can be ported to multiple hardware architectures and operating systems with reasonable effort.
- Representative of the state of the art for the given field.
- Capable of solving problems of varying sizes.
- Follow a reasonably consistent code path.
- Real, parallelised applications
SPEC seeks benchmarks that are derived from real applications, but it is not necessary for an application developer to contribute an entire commercial codebase. A more reasonable submission might be the compute-intensive portion of such an application, without its user interface. SPEC is particularly interested in applications that take advantage of multi-core processors and parallel, multi-threaded computing.
Current SPEC CPU 2017 benchmarks range from AI to 3D rendering and ray tracing, to atmospheric, ocean and explosion modelling, to name a few.
Open for submissions
Proposals for new SPEC CPU benchmarks will be accepted beginning February 25, 2020. The submission period will be one year, but submitters are encouraged to enter as early as possible to provide adequate time for proposal review.
For more information, including a list of application areas and languages in which code is being sought and an entry form for the search program, visit www.spec.org/cpuv8.
SPEC is a non-profit organisation that establishes, maintains and endorses standardised benchmarks and tools to evaluate performance and energy consumption for the newest generation of computing systems. Its membership comprises more than 120 computer hardware and software vendors, educational institutions, research organisations, and government agencies worldwide.
