The Barcelona Supercomputing Center (BSC) and the private sector programme at the National Center for Supercomputing Applications (NCSA) have collaborated to scale BSC’s Alya multi-physics code to 100,000 cores of NCSA’s Blue Waters supercomputer.
While the majority of multi-physics codes are not prepared for large-scale simulations in supercomputers, with efficiency dropping as problems are scaled to a wider number of processors, Alya achieved more than 85 per cent parallel efficiency running on Blue Waters. The code will be used to simulate complex engineering problems such as airflow in the human body, contraction of the heart or combustion in a kiln furnace.
‘These unprecedented results contradict the common belief that engineering simulation codes do not scale efficiently in large supercomputers, opening a new wide horizon of potential applications in the industrial realm,’ said Seid Koric, the senior technical lead for industrial projects with NCSA’s private sector programme. According to Koric, ‘it would take 17.4 years for a serial code to do what Alya on 100,000 cores of Blue Waters can do in less than two hours'.
Alya, a multi-physics parallel code, has been under development at BSC since 2004 and is used to simulate complex engineering problems. Blue Waters is based on a CrayXE6 Opteron blade server nodes and XK6 mixed CPU-GPU nodes, all linked together in a single network using the Gemini XE interconnect created by Cray. The system provides more than 1.5 petabytes of memory, 25 petabytes of disk storage and 500 petabytes of tape storage.
Staff from BSC and NCSA’s private sector programme collaborated to push the scaling of Alya on Blue Waters. Koric worked closing with BSC staff, especially Mariano Vázquez, Guillaume Houzeaux and Antoni Artigues, over a couple of months to port Alya to Blue Waters; the code had never previously been run on a Cray platform of such size, so they had to overcome some porting and optimisation challenges before the code could run efficiently.
Three cases covering a wide range of engineering simulation needs were chosen to run on Blue Waters: airflow in the human respiratory system, coupled electrophysiology and mechanical contraction in the heart, and combustion in the gaseous phase of a kiln furnace, which is used in the cement industry. All were simulated on Blue Waters at unprecedented levels of scalability.
NCSA and BSC plan to continue collaborating to test and improve Alya and to use it to solve large multi-physics problems for NCSA’s industrial partners.
‘Thanks to the high parallel efficiency attained in codes like Alya, exascale supercomputers will allow engineers and scientists to keep dreaming in projects that today are hard to imagine. Alya is a clear example of BSC’s efforts to scale complex codes and make these dreams come true,’ said Mateo Valero, director at Barcelona Supercomputing Center.
