Spectral Element Methods for Compressible Fluid Flow

Github Repository

SELF is an object-oriented Fortran library that support the implementation of Spectral Element Methods for solving partial differential equations.

The SELF API is designed based on the assumption that SEM developers and researchers need to be able to implement derivatives in 1-D and divergence, gradient, and curl in 2-D and 3-D on scalar, vector, and tensor functions using spectral collocation, continuous galerkin, and discontinuous galerkin spectral element methods. Additionally, as we enter the exascale era, we are currently faced with a zoo of compute hardware that is available. Because of this, SELF routines provide support for GPU acceleration through AMD's HIP and support for multi-core, multi-node, and multi-GPU platforms with MPI.

Recent Activity

  • 2/1/2021 - SELF_Memory module and SELF_Lagrange class is complete with serial and GPU kernels passing tests!

  • 4/1/2021 - SELF_Mesh and SELF_Geometry modules are complete with serial and GPU kernels passing tests!

  • 6/1/2021 - SELF_MPI module is drafted ! Build tests are passing; no run tests configured yet !

  • 7/1/2021 - Clean SELF API implementation is drafted for SELF_MappedData and SELF_DG. HIP/HIPFort Kernels have been tested and are passing for grid interpolation, boundary interpolation, divergence, and gradient methods !

  • 12/1/2021 - SELF is nearly complete in its transition to HIPFort. Advection2D and Advection3D models are passing serial CPU, single GPU, and MPI (cpu-only) tests!

Fast Equilibration of Ocean Tracers Software (FEOTS)

Github repository

This project began during my PostDoc at Los Alamos National Laboratory. General Circulation Models (GCMs) are used to simulate the world's oceans and are expensive to run at eddy-resolving resolutions. Modeling the distribution of biogeochemical tracers in the oceans can help illuminate the impacts climate change has on ocean acidification, phytoplankton population distribution, and chemical ratios in ocean sediments. However, studying these impacts requires tracer distributions have reached a quasi-steady state; currently, estimates for the equilibration time scale is O( 1000 years ). Forward integration of a full GCM is far too costly to equilibrate ocean tracers.

FEOTS is a tool that assists in diagnosing GCM velocity and diffusion fields (transport operators) through the use of impulse fields and their response to advection and diffusion operators. These fields can be used to reconstruct transport operators that can be used to drive lightweight passive tracer simulations in an "offline mode". In addition to the lighter weight of this post-processing tool, posing the equilibration problem as a nonlinear minimization problem can reduce the number of iterations to reach an equilibrated tracer field

Gulf Stream Separation in General Circulation Models

Follow our current work on this project

Climate models currently struggle with the more traditional, coarse ( O(100 km) ) representation of the ocean. In these coarse ocean simulations, western boundary currents are notoriously difficult to model accurately. The modeled Gulf Stream is typically seen exhibiting a mean pathway that is north of observations, and is linked to a warm sea-surface temperature bias in the Mid-Atlantic Bight. Although increased resolution ( O(10 km) ) improves the modeled Gulf Stream position, there is no clean recipe for obtaining the proper pathway.

The 70 year history of literature on the Gulf Stream separation suggests that we have not reached a resolution on the dynamics that control the current’s pathway just south of the Mid-Atlantic Bight. Without a concrete knowledge on the separation dynamics, we cannot provide a clean recipe for accurately modeling the Gulf Stream at increased resolutions. Further, any reliable parameterization that yields a realistic Gulf Stream path must express the proper physics of separation.

The goal of this project was to determine what controls the Gulf Stream separation. To do so, I examined the results of a model intercomparison study and a set of numerical regional terraforming experiments. It is argued that the separation is governed by local dynamics that are most sensitive to the steepening of the continental shelf, consistent with the topographic wave arrest hypothesis of Stern (1998). A linear extension of Stern’s theory is provided, which illustrates that wave arrest is possible for a continuously stratified fluid.


J. Schoonover, W.K. Dewar, N. Wienders, and B. Deremble. Local Sensitivities of the Gulf Stream Separation. J. Phys. Oceanogr., 47:353–373, 2017

J. Schoonover et al. North Atlantic Barotropic Vorticity Balances and the Gulf Stream Separation in Numerical Models. J. Phys. Oceanogr., 46:289–303,


Gulf Stream Separation Dynamics (FSU Dissertation)

FSU RCC - Gulf Stream Separation Explored Through HPC