Accelerating PFLOTRAN-OGS on GPUs using PETSc

Abstract

With the evident effects of rapid climate change and society´s continuing dependence on fossil fuels, efficient modelling of reservoir behaviour for emerging CO2 storage projects is in high demand. Due to the models’ physical complexity and the non-linear nature of CO2 storage processes, the computational demands are so significant that physical aspects, such as the dissolution effect, are ignored due to computational limits. State-of-the-art flow simulation codes use highly parallel hardware and programming models to handle large-domain simulations. At the time of this work, complex flow simulators had yet to fully investigate the benefits and impact of utilising accelerators like GPUs. This study explores the performance of accelerating the production code PFLOTRAN-OGS, developed by OpenGoSim, using GPUs through PETSc’s recently built-in accelerated solvers. Our accelerated simulation is run on two test cases: GW1, a CO2 storage case from OpenGoSim, and SPE1, a simple Black Oil benchmark from the Society of Petroleum Engineers. The preliminary benchmark indicates that a GPU-accelerated solver with a CPU-based framework gives an overall slower simulation. However, our profiling verifies that most of the time was spent on transferring matrices back and forth between the CPU and GPU, while the solver steps have significant speedup on the GPU. Our results thus show that the CUDA-accelerated PETSc FGMRES solver will be faster than its CPU counterpart once the complete code is moved to the GPU.