<div class='slidealt'>Experience kvm <a title='virtualization for embedded heterogeneous arm core platforms' href='/en/products'>virtualization extensions</a></div> <div class='slidealt'>Benefit from custom <a title='kvm on arm services full virtualization' href='/en/services'>virtualization services</a></div> <div class='slidealt'>KVM on ARMv7 and ARMv8 <a title='kvm-on-arm open source smu extensions' href='/en/solutions/guides/vfio-on-arm/'>IOMMU full virtualization</a></div> <div class='slidealt'>Virtualization research projects <a title='ARM multicore kvm open source' href='/en/research'>in cloud and embedded systems</a></div> <div class='slidealt'>Virtualization solutions for heterogeneous <a title='ARMv7-ARMv8 virtualization open source solutions' href='/en/solutions'>ARM multicore systems</a></div>

Virtual Open Systems Scientific Publications

Towards a highly energy-efficient and highly integrated compute node for the Exascale revolution: the ExaNoDe approach


Euromicro Conference on Digital System Design (DSD-2017), Vienna, Austria.

Euromicro Conference on Digital System Design - DSD-2017

Contributed slides presentation

The slides presented at this conference are made publicly available.


HPC, exascale, compute node, HPC low power, heterogeneous computing, virtualization, parallel programming models.


A.Rigo, C.Pinto, K.Pouget, D.Raho (Virtual Open Systems) & ExaNoDe Project partners.


This work was supported by the ExaNoDe project that has received funding from the European Union's Horizon 2020 research and innovation program under the Grant Agreement N° 671578. The work presented in this paper reflects only authors' view and the European Commission is not responsible for any use that may be made of the information it contains.


Power consumption and high compute density are the key factors to be considered when building a compute node for the upcoming Exascale revolution. Current architectural design and manufacturing technologies are not able to provide the requested level of density and power efficiency to realize an operational Exascale machine. A disruptive change in the hardware design and integration process is needed in order to cope with the requirements of this forthcoming computing target. This paper presents the ExaNoDe H2020 research project aiming to design a highly energy efficient and highly integrated heterogeneous compute node targeting Exascale level computing, mixing low-power processors, heterogeneous co-processors and using advanced hardware integration technologies with the novel UNIMEM Global Address Space memory system.


This paper presented the H2020 ExaNoDe project, a European funded project aimed at paving the way towards the definition of a compute element that supports the UNIMEM system architecture for an Exascale level computing system. In ExaNoDe the main focus is on the compute element, as the heart of a computing system. Thanks to 3D integration and Silicon Interposer technologies the project will create a disruption in how to design highly modular, yet power efficient computing elements. Such novelty will not only make it possible to reach the energy efficiency requested to make an Exascale system affordable, but will also open the door to actual industrial exploitation thanks to reduced time to market and easier integration of hardware components. ExaNoDe will build a compute element prototype based on ARMv8 processors, FPGAs to accelerate software and a memory hierarchy and locality architecture integrated using the aforementioned 3D technologies. In addition, the prototype will be complemented by a full software stack including: firmware and operating system support, virtualization for resilience and sharing of resources, and parallel programming models to actually span computation over the available computing resources.

ExaNoDe multi-chip-module integration concept


The global race for building ever larger and more powerful supercomputers has over the years led to the creation of a plethora of different architectures, and driven various "revolutions". This race has been recorded over the years in the Top500 Supercomputers list, where the most powerful supercomputers built so far are ranked based on peak floating point (FP) performance. At the present moment the highest FP performance registered in the list is from yet another new architecture delivering 100 Petaflops (10exp(15) FP operations per second). However, even such performance will soon not meet the needs of increasingly complex applications and the research community is already investigating the development of Exascale computing systems. The target is to build supercomputers capable to reach a peak-performance in the order of the Exaflop (10exp(18) FP operations per second). This trend is a clear priority worldwide, with programs in Europe, USA, China and Japan. However, the previous considerations are based on results of simple benchmarks, designed to measure the peak theoretical performance of a computing system with effective real-world performance at less than 5% of peak. The real questions are: why do we actually need Exascale? How can more cost effective performance be realized? What are the concrete benefits of Exascale computing? There are several answers to support Exascale computing, but it is worth noting that advances in the understanding of some scientific problems are often coupled with advances in computing capabilities. Higher computing capabilities, beyond current state-of-the-art Petascale systems, will help scientists in designing models and running simulations of much more complex problems that could help improve our every-day life (e.g. computational biology, climate control, research on energy, etc.). In addition such huge computing power will have a strong impact on industry as well (e.g., oil exploration and production, aerospace, pharmaceutical, etc).

Despite the strong motivations for Exascale and the commitment of various government agencies, achieving such a goal is not a matter of just adding a number of cabinets to an existing supercomputer in order to increase its horse-power. To reach the Exascale level, computing systems will need a complete overhaul on how to deal with the challenges in the areas of cost, power, compute efficiency, resiliency and fault tolerance, data movement across the network and, last but not least, programming models. At the heart of this process we can find the compute element which is the fundamental building block of a computing system, and should be considered as the starting point of any research activity towards Exascale. The European H2020 ExaNoDe research project investigates, develops, integrates and validates the building blocks for a compute element that enables Exascale levels of computation. The principle of the project is to apply novel 3D integration and hardware design technologies, mixed with virtualization of resources and the UNIMEM memory system to deliver a prototype-level system demonstrating that those technologies are promising candidates towards the definition of a compute node for the Exascale computing. The ExaNoDe project idea is built around the following design goals:

  • Affordability: Ensure the solution is commercially viable and competitive in both its performance and its cost of ownership.
  • Design Efficiency: Using power-efficient compute elements and System design principles to ensure minimum duplication and abstractions within the infrastructure, to avoid unnecessary power consumption and latency overheads.
  • Operational Efficiency: power consumption proportional to activities making actual progress.
  • Everything Close: Leverage physical distance and data locality to design for minimum resistance and capacitance, so as to deliver the lowest power overhead associated with the required data connectivity.

ExaNoDe is also closely collaborating with the ExaNeSt and ECOSCALE H2020 projects. ExaNeSt is investigating how storage, interconnections and cooling systems will have to evolve towards Exascale. ECOSCALE, instead, aims to provide a holistic approach for a novel heterogeneous energy-efficient hierarchical architecture, a hybrid MPI+OpenCL programming environment and a runtime system for exascale machines. The combination of these three projects aims at covering the whole picture of an Exascale HPC machine.

Access the full content of this publication

Login or register to access full information