Far more than petaflops
Thomas Lippert, director of the Jülich Supercomputing Centre at Forschungszentrum Jülich explains how supercomputers of highest scalability and scientific users can be brought closer together by means of a new support instrument: the simulation laboratory
May 2009 marked a milestone for the Forschungszentrum Jülich and European supercomputing: Jugene, the first supercomputer in Europe with a computing power of more than a petaflop per second, was inaugurated in Jülich. The Jülich IBM Blue Gene/P system consists of about 72,000 PowerPC 450 cores at 850MHz and a memory size of 144 terabytes. Jugene is the highly-scalable pillar in the Jülich heterogeneous supercomputing concept. The other one is Juropa, a best of breed cluster computer integrated by the French HPC company Bull, and is used for general-purpose HPC applications. With 2,048 compute nodes based on the Intel Xeon Processor X5570 (Nehalem-EP), Juropa provides more than 200TFlop/s. The Jülich heterogeneous supercomputing concept is adapted to serve the broad portfolio of Jülich’s applications in a most efficient manner. A selected number of very large, time-intensive and highly scalable simulations are carried out on Jugene, while a large group of smaller simulation projects with high demands on memory and flexible communication network are run on Juropa.
In recent years, Europe has recognised supercomputing as a key enabler for research, both in science and industry. With the Partnership for Advanced Computing in Europe (PRACE), Europe is about to create a supercomputing infrastructure with up to five leading centres at the highest performance level. The Forschungszentrum Jülich acts as the coordinating institute for PRACE. As the complexities concerning the highly scalable hardware are rising exponentially, we have to meet increasing challenges for modelling, algorithm research, software technology and visualisation technologies. The most demanding task, however, is to help the user to keep pace with these rapid developments.
Against this background, the Jülich Supercomputer Centre is devoting itself to designing and providing computing power at the highest level and, at the same time, to developing algorithms, models, tools, and methods supporting users in a variety of fields of computational science and engineering. Jülich has recognised that project-oriented support models are not sufficient anymore. In order to most strongly integrate our support activities within the research communities, Jülich has implemented an advanced user support model called the simulation laboratory.
A simulation laboratory is a targeted research and support organisation for a specific scientific community. It collaborates with this community and supports the users in performing their simulations on supercomputers. The lab itself is seen as part of the community and consists of an interdisciplinary core group located at the supercomputer centre and a number of associated scientists from outside. In Jülich, we are currently implementing four simulation laboratories for the fields of plasma physics, computational biology, molecular systems, and Earth and environment. The core group of the lab will finally comprise at least four staff members and a few PhD students, i.e., a senior scientist recognised and involved in the specific research field, two scientists working on methodological, mathematical and software-related questions, and a technician with skills in software implementation, optimisation and data management. There will also be PhD students working on specific research and development tasks.
The core group is tightly connected to the community by associated scientists, who participate actively in the work of the laboratory. This setup enables the simulation lab to fulfil its objectives, participating actively in research, developing the software and guaranteeing its sustained provision, adapting and optimising software on high-end systems, carrying out training and education on high-performance systems, and organising and triggering the exchange of knowledge and expertise in the community.
This new approach to user support will help tackle grand challenges and complex questions in science, which is the main goal of Forschungszentrum Jülich. Calculations performed at Jülich Supercomputing Centre recently led to two important examples of scientific breakthroughs, throwing light on both the origin of life and the origin of mass.
High pressure, high temperatures and sulphur-containing minerals may have played an important part in the origin of life, i.e. the building of protein molecules outside of cells. Experiments under these conditions are very complex and costly. Theoretical chemists from Ruhr-Universität Bochum, led by Dominik Marx, used the Jülich supercomputers to identify a possible reaction path, where sulphur-containing minerals play the role of a catalyst.
Most of the mass of the visible universe stems from nuclei, i.e. protons and neutrons, which themselves are made of three quarks. But the model of interactions in the nuclei, quantum chromodynamics, is far too complex to carry out the necessary calculations by hand. With the supercomputer Jugene, a German-Hungarian team led by Zoltan Fodor calculated the mass of low-lying hadron particles and verified all mass values to lay within the range of the experimental predictions. This result was recognised as verification of quantum chromodynamics and an explanation of the origin of mass by means of supercomputing. It found its entry into the top 10 of the 2008 scientific breakthroughs of the journal Science. Both examples show the immense power of simulations and demonstrate today’s impact of supercomputing.
Such findings show that supercomputing is far more than petaflop/s. I am deeply convinced that unimagined scientific discoveries through simulation can be expected in the future.