SELF-Fluids

About SELF

SELF is a Fortran-based library that provides data-structure and type-bound routines for implementing Spectral Element Methods in one, two, and three spatial dimensions. Routines are currently available for collocation and discontinuous galerkin formulations on unstructured meshes with isoparametric elements.

The SELF API is designed based on the assumption that SEM developers and researchers need to be able to implement derivatives in 1-D and divergence, gradient, and curl in 2-D and 3-D on scalar, vector, and tensor functions using spectral collocation, continuous galerkin, and discontinuous galerkin spectral element methods. Additionally, as we enter the exascale era, we are currently faced with a zoo of compute hardware that is available. Because of this, SELF routines provide support for GPU acceleration through AMD's HIP and support for multi-core, multi-node, and multi-GPU platforms with MPI.

Modeling Fluids

SELF-Fluids leverages SELF to solve the Compressible Navier-Stokes equations on multi-GPU architectures.

The Fluid class employs the Nodal Discontinuous Galerkin Spectral Element method for the spatial discretization. The inter-element fluxes are numerically estimated using the upwind Lax-Friedrich's approximate Riemman solver. Currently, the model is forward-stepped using Williamson's 3rd Order Low-Storage Runge-Kutta time integrator. There are plans to develop and support additional explicit and implicit time-stepping schemes.

SELF-Fluids currently exists on the stale/self-fluids branch of the SELF repository. SELF is actively being refactored to support GPU platforms from multiple hardware vendors. We expect SELF-Fluids to be fully functional on Multi-GPU platforms by August 2021.

Recent Updates

SELF runs on AMD GPUs

April 16, 2021

Tasks Completed

HIPFort Migration Complete

February 11, 2021

Tasks Completed

  • SELF has been isolated into it's own repository at https://github.com/FluidNumerics/SELF

  • Serial CPU only and single GPU routines have been implemented and tested.

  • CI infrastructure is in place for automatically testing serial and single GPU tests.

  • Docker containers are available on Google Cloud Platform and Dockerhub

Next Steps

  • Multi-GPU ( MPI + HIPFort ) and MPI-Only Support

  • CGNS/HOPr File IO Support

  • Implement Demo Applications

    • Compressible 2-D Navier Stokes

    • Shallow Water Equations

Refactoring and HIP-Fortran Migration

May 1, 2020

In Q2-Q3 of this year, we are working on the following tasks on feature branches of the self-fluids repository

  • Transition CUDA-Fortran to HIP-Fortran

  • Switch namelist input for JSON input

  • Transition model output towards CGNS format extensions or HOPr format to enhance data-sharing and collaboration.

  • Enhance automated testing and CI capabilities

You can track our progress on these tasks through the SELF-Fluids Github site

Roadmap

The roadmap below shows the features we plan on implementing with a timeline for the expected completion.

February 2021

  • New SELF API Specification

    • Routines defined by the cross product of
      [ interpolation, differentiation, divergence, gradient, curl ] X [scalar, vector, tensor] X [1D, 2D,3D] X [Strong Form, DG Weak Form, CG Weak Form ] x [Serial, Single GPU (HIP), MPI-only, MPI+HIP (multi-gpu) ]

  • All routines for Serial CPU and Single GPU implemented

April 2021

  • MPI-only support for all routines

  • HOPr file IO support

  • VTK file IO support

  • SpecMesh file IO support

June 2021

  • MPI+HIP support for all routines

  • OBJ file IO support

  • CGNS (with higher order extension) file IO support

August 2021

  • Nektar++ file IO support

  • Example Applications : SELF-Fluids (Compressible Navier-Stokes)

  • Board of Directors formation campaign

    • Formally establish board of direction & operating policies

    • Formalize Specifications & Contributing guidelines