In-Situ Big Data Analysis for Flow and Climate Simulations
- Reference number
- BD15-0082
- Start and end dates
- 170101-230228
- Amount granted
- 25 154 405 SEK
- Administrative organization
- KTH - Royal Institute of Technology
- Research area
- Computational Sciences and Applied Mathematics
Summary
The goal of this project is to break Big Data barriers in numerical simulations to enable next-level flow and climate research. The sheer size and complexity of the simulation data overwhelms current storage systems and post-processing analysis tools, which leads to underutilized data and slows progress in flow and climate research. We will develop, implement and apply novel feature-based data analysis methods to be run alongside the simulation (in-situ), which condense the main aspects of the data by several orders of magnitude. This is accompanied by sophisticated task scheduling to exploit the heterogeneous hardware of modern supercomputers, and interactive visualization of the analysis results. For the first time, this will enable an interactive zoom into the smallest spatial and temporal scales of the largest flow and climate simulations. We apply our methods to target high-impact research questions. For the flow case, we address turbulent separation and wake-wake interactions; the former can directly lead to improved performance of vehicles and planes, the latter is crucial for the design of wind parks. For the climate case, we study the spatio-temporal development of extreme storms to assess the hazard posed to Sweden and other regions in Europe. Dissemination of our results is ensured by implementing our novel methods in a widely spread, open source in-situ library, and incorporating this library into three simulation packages actively used in Sweden.
Popular science description
The transport of people and goods is drastically increasing on a global scale. This requires large amounts of energy, typically in the form of fuel or electricity. However, when a car drives on the street, a train runs between two cities, or a plane brings tourists to other continents, then roughly half of the required energy gets lost at a place that is unsuspicious to the eye. It is the few centimeters around the surface of the vehicle. The process is called "turbulence". It is one of the most complicated processes in nature and researchers all over the world work on understanding and controlling it. Sweden's automotive and aerospace industry (Volvo, Saab, and others) depend on this knowledge to create internationally competitive products. Turbulence is often visible to us when we look at the sky and see a storm coming up. Wind storms are driven by turbulence in the atmosphere. These turbulent processes are much larger than the turbulence around a car, but they follow the same principles. Climate researchers are interested in understanding these processes to accurately predict the future developments due to climate change and to warn the population about extreme storms. Sweden's forest and insurance industry depend on accurate storm warnings. The key to understanding turbulence is numerical simulation. It allows us to compute these physical processes in high detail. It is an irreplaceable discipline in both academic research and industrial development. However, numerical simulations produce so much data that it is impossible to even save it to a disk. Flow and climate researchers suffer from this situation as they cannot progress as much as they would like to. This project develops new technologies to automatically analyze the simulation data while it is being computed on the supercomputer. Hence, it does not need to be saved, but just the result of the analysis needs to be saved and this is much smaller. To achieve this goal, an interdisciplinary team of researchers from KTH Royal Institute of Technology, Linköping University and Stockholm University will adapt and expand recent mathematical methods that have an unprecedented ability to analyze even the most complex data. The researchers plan to apply their work together with partners from Sweden's industry as well as Sweden's meteorological services to put this powerful technology to use.