Case Study: Alya Red: HPC-based cardiac computational modelling
Alya Red is Barcelona Supercomputing Center’s (BSC) flagship project in biomechanics. Taking its name from Alya System, BSC’s main simulation tool, it is fully developed in-house, from the numerical methods up to the parallel implementation, including mesh-generation and visualisation. A computational mechanics code for multiphysics, Alya System is designed to run in large-scale supercomputing facilities and is capable of solving different physics problems in a coupled way, each one with its own modelling characteristics, such as fluids, solids, electrical activity, species concentration and free surface.
The Alya Red project focused on the development of a cardiac computational model at organ level, in order to provide medical scientists with aids to understanding and diagnosing these biological systems. The collaboration involves medical doctors from several hospitals, image researchers, bioengineers and computational scientists. Mariano Vázquez, high-performance computational mechanics team leader, BSC-CNS (Barcelona Supercomputing Center – Centro Nacional de Supercomputación), explains that as an organ, the heart works at a level of three different physical problems: the electrical activation, the muscular deformation and the blood. Each aspect is complicated and time consuming in terms of computational resources, he says, but the multiphysics simulations solve this combined problem.
Concentrating specifically on the computational mechanics, there has been a natural progression in his group’s research. ‘We began by focusing on the electrical aspect and once we had that working the medical doctors questioned why we weren’t also focusing on the mechanical part of the heart. Then, as we worked in the basics of computational mechanics, we were asked to simulate the solid mechanics, and then the blood flow,’ says Vázquez. He adds that large-scale HPC resources are a necessity at this level, as the degree of accuracy and anatomical detail that can be obtained from clinical imagery is very high.
In a published case study, Vázquez confirms that the project team is capable of simulating two ventricles models coming from real geometries (dogs, rabbits and humans) for the electro-mechanical coupling, in meshes, of tens of millions of elements. This high resolution is determined by experimental data acquisition – as happens with HPC software and hardware, medical data acquisition is getting better at a faster pace than simulation software. For the coupled electromechanical problem, Alya has attained more than 90 per cent linear scalability with up to 2,000 cores. This efficiency allows the programming of complex electrophysiology, coupling and material models, which their developers usually can test only in toy problems.
Vázquez adds: ‘This is a very rewarding area and we hope to open the collaboration up to other aspects of medical research. Doctors must formulate the problems, but it’s important to remember how important the people who develop the simulation tools and the clinical image processing are. Our main collaborators come from the Hospital Sant Pau, the Universitat of Lleida and the Computer Vision Center of the UAB, all in Spain.’ He concludes by stating that, moving forward, simulation is going to increasingly be called upon as a medical science tool and these computational hearts are going to have many different applications.
Case Study: Finite-element models from CT scans of artificial knee joints
The University of Applied Sciences in Amberg-Weiden is a leading German university in the field of product development, rapid prototyping, simulation and materials science. Led by Professor Franz Magerl, a team was tasked with providing recommendations regarding the structural stability and performance of artificial knee joints. This was to be achieved by creating accurate finite-element models of the polymer joints, using computer tomography (CT) scans.
Professor Magerl explains: ‘After we received the CT data, we used it to produce voxel (volumetric pixel) data which we then transferred to STL, a format that represents 3D surface geometry. This format is used with HyperWorks for surface meshing for the purposes of finite-element calculations, and based on this we were then able to create a volume mesh. We had been concerned about the structural integrity of the knee joints, but the software made calculations regarding easy-to-verify critical stresses and we were able to determine that the joints were in fact sound.’
Due to the size and the level of detail in a CT model, in the past it has been very difficult to create high-quality FE meshes based on CT data. In order to obtain accurate results from the simulation, the team need a good FE mesh. ‘With HyperMorph, a morphing tool within HyperMesh, engineers can change the shape of FE meshes without sacrificing the element quality and thus explore many new designs in a very short time,’ says Stuart Sampson, director of modelling and visualisation at Altair Engineering. He adds that shape optimisation picks up the shapes created by the morphing technology to find their best combination for a structure based on OptiStruct’s optimisation algorithms.
According to a case study published by Altair Engineering, the CT scan of the knee joint, available as triangulated surfaces data, was imported into HyperMesh. The STL mesh cannot be used for simulations because of poor element quality, however HyperMesh includes a function that re-meshes existing meshes, such as a STL model, without underlying analytical geometry information (e.g. IGES data) to improve mesh quality.
The software created surface approximations in the background and generated a shell mesh based on that approximation. As a result, small geometric details inside the artificial knee joint, such as a bubble inclusion, could be captured with a high-fidelity mesh. In a second step, a tetrahedral-based model was created from the shell mesh and boundary conditions were applied. As Professor Magerl confirmed, the simulation revealed non-critical stresses in the area of the bubble, so there was no need to improve the manufacturing process.
Looking at the broader landscape of biomedical modelling, Stuart Sampson adds that, as computers and solvers are getting more powerful, engineers are considering an increasing number of details in the modelling process. A pre-processor needs to be able to generate these models, which can have many million elements, while efficient definition of boundary conditions and final model setup are also important aspects.
‘Solvers need to be capable to handle these large and complex models, which not only have many elements, but also show highly nonlinear geometric and material behaviour. Last but not least, a strong pre-processor needs to handle the large result files efficiently and fast,’ continues Samson. ‘The requirements keep changing due to the increasing complexity of the models. Where, years back, more assumptions and abstractions had to be made, biomedical models now are more detailed and more features are taken into account (e.g. muscles and sinews).’
He believes that the performance of software to handle large and complex models should play a big role. This counts, he says, for all stages of model handling, such as preparation, solving and post-processing. For the biomedical field geometric accuracy and mesh quality are very important, as is the value of a software solution. ‘Virtual traumatology – including the design, analysis and validation of a “virtual human body” and of the evaluation of possible injuries – is certainly one of the areas where we have seen the greatest number of applications over the last years, and we see this trend continuing in the future,’ Samson comments. ‘The reason is obviously the possibility to simulate with a great level of accuracy events otherwise not reproducible in a “testing” environment. Moreover, the definition of detailed models of the human body opens new research fields for biomedical purposes, ranging from implant designs to virtual surgery simulation.’
