Researchers at Forschungszentrum Jülich and the Karlsruhe Institute of Technology (KIT) have demonstrated the AI performance of Europe’s fastest supercomputer using the MLPerf benchmark. The results were presented at the world's largest supercomputing conference, SC21, which is taking place in the US city of St Louis, Missouri.
The researchers wanted to know how Europe's fastest supercomputer performs when it comes to training neural networks. This computing power can then be harnessed to accelerate research for a wide number of scientific applications including the development of language models and weather forecasts with AI as well as the resolution of unknown RNA structures.
Director of the Jülich Supercomputing Centre professor Thomas Lippert comments: ‘Computers the size of JUWELS enable Europe to keep up with the USA and China. With the benchmark test, we have shown that our machine can also unfold its potential in practice and contribute to keeping on the ball when it comes to AI.’
Researchers and engineers from Baidu and Google as well as from Harvard, Stanford and Berkeley created the MLPerf benchmark a few years ago to measure the performance of computing systems in applications based on AI. The MLPerf HPC benchmark specifically aims to determine the performance of classical supercomputers when training neural networks. Traditionally, such computers are used for simulation and modelling.
The success of artificial intelligence is also evident in research. In the past, supercomputers were largely used for simulation and modelling. Today, they are increasingly applied to train large, computationally extremely intensive AI models. The demand for computation for this has increased 300,000-fold in the last 6 years. The construction of the fastest and most efficient supercomputer in Europe takes this development into account. The booster module of the Jülich supercomputer JUWELS has more than 3700 NVIDIA graphics processors of the latest generation, which are specially designed for machine learning.
In addition to the JUWELS booster, the "Karlsruhe High Performance Computer", or HoreKa for short, was also involved in the test. With almost 700 of these high-performance chips, the supercomputer at KIT represents the backbone of high-performance computing for German universities.
The researchers optimised the execution of the benchmark code within Helmholtz AI - a research cooperation of the Helmholtz Association on artificial intelligence – to get the most out of both supercomputers. The still relatively new MLPerf HCP benchmark can be used to determine how powerful supercomputers are when training neural networks.
‘With the MLPerf HPC Benchmark, we were able to use over 3000 GPUs simultaneously to accelerate machine learning. In total, more than 100 quadrillion computing operations per second were executed,' explains Jülich AI expert Dr Stefan Kesselheim. It took just two minutes to train a large climate model. On the predecessor of the JUWELS supercomputer, the task would have taken several hours.
Jülich computing power in demand for European AI language model
The JUWELS booster is to be used, for example, to train a large AI language model that can compete with the US language model GPT-3. The GPT-3 model, developed by OpenAI and commercialised by Microsoft, caused a sensation when it was introduced in 2020 because it is capable of independently formulating texts that sound human. Forschungszentrum Jülich is cooperating with BMW, WDR and Fraunhofer IAS on the development of its European counterpart, Open-GPT-X. In June of this year, Open-GPT-X was accepted by the German Federal Ministry of Economics and Technology as part of the European Gaia-X data infrastructure.
Researchers at Forschungszentrum Jülich are also using the JUWELS Booster to develop weather forecasts with AI. Modern deep learning techniques are used to improve medium- and short-term forecasts. Another example of application is the prediction of protein structures, especially RNA structures, many of which are still unknown. The strategic importance of this was recently brought to light by the development of Biontech's Covid-19 vaccine, which is based on such an RNA.
The Helmholtz Artificial Intelligence Cooperation Unit, or Helmholtz AI for short, promotes inter-centre and interdisciplinary research on applied artificial intelligence in the Helmholtz Association. Within the framework of Helmholtz AI, researchers from Forschungszentrum Jülich and the Karlsruhe Institute of Technology (KIT) have jointly determined how two top German computers perform in the computationally intensive training of algorithms from the field of artificial intelligence. In addition to the booster module of the Jülich supercomputer JUWELS, the Karlsruhe high-performance computer - HoreKa for short - was also subjected to a performance test.
