John D’Ambrosia evaluates the influence of interconnects on the Top500 list of supercomputers
In 1993 when the Top500 list premiered, the maximum noted performance was listed at 59.7 GigaFlops for the CM-5/1024 System at Los Alamos National Laboratory in the United States. The latest Top500 list, released in June of this year, indicated that the ‘top dog’ was the Tianhe-2 (MilkyWay-2) System at the National University of Defense Technology in China, with a whopping 33,862,700 GigaFlops. This represents an increase in processing power of 567,214 times the CM-5/1024 of 1993 – impressive to say the least. And if you were to look at the time in between these two points, you would find that the number one system in each report followed a fairly predictable exponential growth curve.
However, the list is called the Top500 for a reason: the top 500 supercomputers are tracked, not just the leading system. While there is clearly a lot of notoriety with being first, to understand the impact that a given technology is having on the high-performance computing arena, it is important to consider the Top500 in its entirety. This is particularly true when considering the impact of Ethernet, and I found the June 2013 results to be fascinating.
If we were to look at the number of systems that are based on an Ethernet or InfiniBand interconnects, we would see that these two technologies comprise 84 per cent, or 420, of the top 500 machines. The share between the two is very close with Ethernet, based on Gigabit Ethernet and 10 Gigabit Ethernet, making up 43 per cent, and InfiniBand making up 41 per cent. Further insight can be gained by looking at those entries making their debut in the Top500 list. While InfiniBand had an impressive 52 new entries in 2013 that made up 10 per cent of the total number of supercomputers, Ethernet-based supercomputers comprised a total of 121 new systems, or 24 per cent, in the 2013 Top500 list, with 75 of those new systems based on Gigabit Ethernet and 46 based on 10 Gigabit Ethernet.
This is still fairly macro in perspective, however, and does not necessarily help provide sufficient insight into the role that Ethernet is playing in the high-performance computing market. The Top500 list provides further detail that proves useful; in particular the ‘segment’ classification of the supercomputer.
10 Gigabit Ethernet
The industry segment makes up 54 per cent of the total list, with Ethernet accounting for 78 per cent of the total entries. Combining academic with research yields 40 per cent, and InfiniBand-based solutions dominate these two segments with 70 per cent and 63 per cent respectively, as well as 21 per cent in the industry segment. It is very interesting that these two groupings, which comprise 94 per cent of the Top500, have such diversity. The research and academic segments, which are the basis of 26 of the top 30 positions, lean towards proprietary or InfiniBand interconnects, while the industry segment, which makes up 54 per cent of the Top 500, leans towards Ethernet.
Given the commoditisation of Gigabit Ethernet, it is interesting to note the large number of supercomputers based on a Gigabit Ethernet interconnect that are still on the Top500 list. Furthermore, with 10 Gigabit Ethernet-based systems only showing up on the list since 2011, and 61 per cent of those showing up in the June 2013 listing, Ethernet’s future in the industry segment of high-performance computing seems promising.
As noted earlier, the performance of the top system for each list follows a predictable exponential growth curve. In the near future supercomputers capable of more than 100,000,000 GigaFlops will be a reality. Ethernet’s success is tied to its recognition of its need to evolve in order to address market demands. Currently, only Gigabit Ethernet and 10 Gigabit Ethernet -based supercomputers are showing up on the Top500. With the standardisation of 40 Gigabit Ethernet and 100 Gigabit Ethernet completed in June of 2010, and 400 Gigabit Ethernet currently being explored, Ethernet will continue to be a viable technology for supercomputing far into the future.
John D’Ambrosia, chairman, Ethernet Alliance, and chief Ethernet evangelist, CTO office, Dell