The Grid Computer and Clusters team at the Barcelona Supercomputing Centre has released a set of tools designed to help developers run applications efficiently on distributed systems such as clusters, grids, and clouds.
This announcement coincides with the first anniversary of the ‘Supercomputing Technology Center’ -- whose main aim is the execution of research projects related to hardware and software technologies in high-performance computing.
The centre was created through a partnership between the Barcelona Supercomputing Centre (BSC) and IBM and is valued at 6 million euros.
Earlier collaboration between the two dates back to 2005 and the supercomputer MareNostrum. Thanks to the joint work carried out at the time, MareNostrum ranked first in the Top500 ranking of European supercomputers several times, and achieved fourth position worldwide.
The creation of the MareNostrum, with its 48,896 cores, required the team at the BSC to establish a skill set around parallelisation of code to run on massively parallel HPC resources. One of the products of this work has been the creation of Comps, a task based programming model known for notably improving the performance of large scale applications by automatically parallelising their execution.
The new release includes PyCOMPSs, a new binding for Python which provides support to large number of scientific disciplines. Version 1.1.2, of the COMPSs programming environment is already available in three main programming languages: Java, C/C++ and Python.
The software has been available to the Spanish Supercomputing Network and users of the MareNostrum supercomputer and the for a number of years. This has led to its adoption in several research projects such as OPTIMIS, VENUS-C, EUBrazilOpenBio and EGI.
In these projects COMPSs has been applied to use cases provided by different communities across diverse disciplines as biomedicine, engineering, biodiversity, chemistry, astrophysics and earth sciences. Currently it’s also under extension and adoption in applications in the projects ASCETIC, EUROSERVER, EUBrazil CloudConnect, transPLANT, the BSC Severo Ochoa program and the Human Brain Project flagship.
While COMPSs native programming language is Java, a binding for C/C++ was developed in 2010 to give service to existing applications. Now, the new release includes support to Python (PyCOMPSs) expanding the areas of use of COMPSs.
The success of the software has meant more than 500 downloads last year and is used by around 20 groups in real applications. COMPSs has recently attracted interest from areas such as genomics and biodiversity, where specific courses and dissemination actions have been performed.
In the last year the team’s efforts have been focusing on emerging virtualisation technologies, adopted by cloud environments. In such systems, COMPSs provides scalability and elasticity features by dynamically adapting the number of resources to the actual workload. COMPSs is designed to be interoperable with both public and private cloud providers like Amazon EC2, OpenNebula, BSC EMOTIVE Cloud and with OCCI compliant offerings.
The current release includes the following features; PyCOMPSs, a new binding for Python applications ;a new tracing system using the Extrae tool; and Integrated Development Environment (IDE) for COMPSs applications that help in the development of the applications and in its deployment in the distributed environment. This component is available as an Eclipse plugin published in the Eclipse marketplace.
