In his second report from the PRACEdays14 conference, held in Barcelona from the 20-22 May, Robert Roe reports how computer science is being used to understand the human respiratory system -- providing surgeons with new tools to improve surgery and treatment.
It can be a symptom of hay fever. It is often a sign of infectious disease – sometimes the common cold; sometimes a more serious condition. In class-conscious Britain, it can even be a patronising indicator of social superiority. But the humble ‘sniff’ is also a formidable scientific problem that no one yet understands.
Now, researchers from the Barcelona Supercomputing Centre, Spain and Imperial College London, UK in collaboration with St Marys Hospital Paddington, UK and the Royal Brompton Hospital, UK have been using computed tomography, and powerful computer simulations, to generate a model of the human respiratory system – and, as a by-product, understand precisely what happens when we start sniffing.
The project made use of supercomputing resources provided through the Partnership for Advanced Computing in Europe (Prace) and Hadrien Calmet, a researcher at the Barcelona Supercomputing Centre (BSC) set out the details of the project at the PRACEdays14 meeting, held in Barcelona from 20 to 22 May. He explained that there is very little in the scientific literature about this kind of simulation, so it is a key area of research where new insights can be made.
‘The flow is really is complex and the geometry is also complex, so it is really an application that is well suited for HPC,’ said Calmet. Not only will this help scientists to develop a more robust understanding of the respiratory system, it could also be used to personalise medicine and surgical procedures -- tailored to the specific needs of a patient.
Calmet explained that the final goal with this project is to ‘create a kind of perfect simulation of the human respiratory system, with a very fine mesh, because we have very powerful tools.’
The researchers have been using HPC resources at the BSC and Fermi in Italy to produce complex simulations generated using direct numerical simulation. ‘It will be like a virtual human, allowing us to recreate the respiratory system and use as a tool for the surgeon - so they can understand the flows and the complex geometry,’ said Calmet.
Direct numerical simulation (DNS) in the human nose-throat is a very difficult challenge, due to the complex geometry found in the respiratory system. Between the nasal cavity and the pharynx, and again between the pharynx and larynx, there are smaller passageways which cause pressure changes and turbulence as air passes through the system.
To produce data on the geometry of the respiratory system, the team used X-ray computed tomography (X-ray CT) to produce tomographic images (virtual 'slices') of the respiratory system. The researchers then processed the ‘slices’ to generate a three-dimensional image of the inside, from a large series of two-dimensional radiographic images.
Once the geometry of the system had been established, work could begin on the simulation, which even in the initial stages was very complex. The team used Alya to run the simulations. Alya is the BSC in-house HPC-based multi-physics simulation code. It is designed from scratch to run efficiently in parallel supercomputers, solving coupled problems.
Calmet and his colleagues generated three meshes over the course of the project, with the first mesh comprising 8 million elements. This allowed the team to approximate the system and prove the validity of the methodology. The second mesh was made up of 44 million elements, and was used to analyse the turbulence in the airways in more detail, and to ensure sufficient resolution had been obtained during the simulation using the first mesh.
Due to the lighter data analysis, the less complex mesh is normally used to describe the flow of air within the respiratory system. The final mesh consisted of 350 million elements: this was used to simulate the turbulence of the flow within the respiratory system.
Due to the complex flow patterns found in the human respiratory system, the large airways are each treated as a separate section, so that the flow analysis can be completed separately. This enabled the researchers to develop an understanding of how the flow changes as it reaches different airways that vary in shape and volume. This meant that a large number of turbulence statistics must be computed, so that the main flow features for each region can be mapped separately.
In a paper entitled ‘Alya: Towards Exascale for Engineering Simulation Codes’ that Calmet co-authored, a methodolgy for combining meshes, like those for each segment of the respiratory system, is described in detail. The paper states ‘Different meshes are generated independently and are glued together using some extension elements to connect them. The resulting global mesh is non-conforming and consists of connected overlapping meshes.’ It goes on to state ‘The most cited gluing method is probably the Chimera method, used for overset grids, where patch meshes are superimposed onto a background mesh.’
Segmentation of the airways was performed using the Amira package (TGS Europe) and required some manual intervention, particularly in the nasal airways. There, the fine bone structure challenges the resolution typical of data acquired under routine clinical protocols, but the fidelity of the reconstructed data was carefully checked by ENT surgeons.
Translation of the coarse segmentation into a smooth surface was performed using in-house, curvature-adapted smoothing procedures. Mesh generation was accomplished in stages, using the Gambit and TGrid packages (Ansys). This work was done in collaboration with Doorly and Bates from Imperial College (UK). The solution of this problem involves the solution of the incompressible Navier-Stokes equations. The time discretisation is based on a second order BFD scheme and the linearisation is carried out using the Picard method.
The simulations took place on Fermi, the new supercomputing system in CINECA, Italy and took 20 million core hours. Fermi is a Tier-0 supercomputer systems infrastructure, based on IBM BG/Q architecture.
In reply to a question after his talk on how he could improve the simulation, Calmet said: ‘The acquisition of the data is the CT, the computed tomography. This means that we don’t simulate the tissues, we only simulate the conduit. Now we want to do a new simulation using MRI. To make it a better simulation would be to model the fluids and the tissues -- of course this is so complex that it is a lot of work.’ However, even at this stage, the research has already produced results that can be used to improve medical knowledge of the respiratory system.