MPI DTFE Grid Routines¶

This tutorial explains the MPI-enabled DTFE routines for constructing density and interpolated fields on 2D and 3D grids.

The routines covered are:

mpi_delaunay_density4grid2D
mpi_delaunay_field4grid2D
mpi_delaunay_density4grid3D
mpi_delaunay_field4grid3D

These functions parallelise DTFE calculations by decomposing the simulation domain into slabs distributed across MPI ranks.

MPI Parallelisation Strategy¶

The MPI versions divide the simulation box along the x-axis.

Each MPI rank:

Receives a slab of particles.
Computes a local DTFE.
Exchanges boundary simplices with neighbouring ranks.
Merges border tessellations.
Evaluates grid values inside its local domain.

This enables very large DTFE calculations that would not fit into memory on a single process.

mpi_delaunay_density4grid2D¶

Computes a 2D DTFE density field on a regular grid using MPI.

Minimal Example: Density¶

# Assuming correct initialisation of MPI object, see tutorial.
import numpy as np
import fiesta

N = 100000

x = np.random.uniform(0, 1000, N)
y = np.random.uniform(0, 1000, N)

rho = fiesta.dtfe.mpi_delaunay_density4grid2D(
    x=x,
    y=y,
    boxsize=1000.0,
    ngrid=256,
    MPI=MPI,
    periodic=True
)

Each rank owns only its slab.

Mass Weighting¶

# Assuming correct initialisation of MPI object, see tutorial.
import fiesta

field = fiesta.dtfe.mpi_delaunay_field4grid2D(
    x=x,
    y=y,
    f=temperature,
    mass=particle_mass,
    boxsize=1000,
    ngrid=256,
    MPI=MPI
)

With Coordinate Output¶

# Assuming correct initialisation of MPI object, see tutorial.
import fiesta

rho, x2d, y2d = fiesta.dtfe.mpi_delaunay_density4grid2D(
    x,
    y,
    boxsize=1000,
    ngrid=256,
    MPI=MPI,
    outputgrid=True
)

mpi_delaunay_field4grid2D¶

Interpolates an arbitrary scalar field onto a 2D grid. Unlike the density routine, this function uses particle field values:

Example: Temperature Interpolation¶

# Assuming correct initialisation of MPI object, see tutorial.
import numpy as np
import fiesta

temperature = np.sin(x / 100.0) * np.cos(y / 100.0)

field = fiesta.dtfe.mpi_delaunay_field4grid2D(
    x=x,
    y=y,
    f=temperature,
    boxsize=1000,
    ngrid=256,
    MPI=MPI,
    periodic=True
)

mpi_delaunay_density4grid3D¶

Computes a full 3D DTFE density field. This is the MPI parallel version of volumetric density estimation.

Example¶

# Assuming correct initialisation of MPI object, see tutorial.
import numpy as np
import fiesta

N = 2_000_000

x = np.random.uniform(0, 500, N)
y = np.random.uniform(0, 500, N)
z = np.random.uniform(0, 500, N)

rho = fiesta.dtfe.mpi_delaunay_density4grid3D(
    x=x,
    y=y,
    z=z,
    boxsize=500.0,
    ngrid=256,
    MPI=MPI,
    periodic=True,
    subsampling=2
)

mpi_delaunay_field4grid3D¶

Interpolates scalar fields in full 3D.

Examples:

temperature
velocity divergence
metallicity
pressure
potential

Example¶

# Assuming correct initialisation of MPI object, see tutorial.
import numpy as np
import fiesta

velocity_divergence = np.random.normal(size=N)

field = fiesta.dtfe.mpi_delaunay_field4grid3D(
    x=x,
    y=y,
    z=z,
    f=velocity_divergence,
    boxsize=500,
    ngrid=256,
    MPI=MPI,
    periodic=True
)

Performance Tuning¶

Increase MPI Ranks¶

Large DTFE calculations scale well with:

more particles
larger grids
higher memory demand

Adjust partition¶

Higher values:

partition = 4

reduce local tessellation memory.

Reduce subsampling¶

Subsampling is expensive.

Fastest:

subsampling = 1

Higher accuracy:

subsampling = 2 # 4 or more, but larger values increase accuracy but increase computation time.

Common Pitfalls¶

Mismatched Periodicity¶

If your simulation is periodic:

periodic=True

must be enabled. Otherwise discontinuities appear at boundaries.

Memory Explosion in 3D¶

3D Delaunay tessellations can become extremely large.

Recommendations:

increase MPI ranks
use smaller partitions
reduce grid size
lower subsampling

API Reference¶

fiesta.dtfe.mpi_delaunay_density4grid2D(x: ndarray, y: ndarray, boxsize: float | List[float], ngrid: int | List[int], MPI: object, origin: float | List[float] = 0.0, mass: ndarray | None = None, partition: int = 1, periodic: bool | List[bool] = True, fbuffer: float = 0.5, subsampling: int = 1, outputgrid: bool = False) → ndarray | Tuple[ndarray, ndarray, ndarray]

Returns the Delaunay tesselation density on a grid.

Parameters:

x (array) – Coordinates of particles.
y (array) – Coordinates of particles.
boxsize (float or list) – Dimensions of the grid.
ngrid (int or int list) – Grid dimensions.
MPI (class object) – shift.mpiutils MPI object.
origin (float or list, optional) – Origin for grid.
mass (array, optional) – Mass or mass weights for each the particles.
partition (int or list, optional) – The number of internal Delaunay tesselations used to compute the density grid.
periodic (bool or list, optional) – Determines whether periodic boundaries are applied.
fbuffer (float, optional) – Buffer factor length, used to trim exterior buffer region.
subsampling (int, optional) – The pixel subsampling rate. Each pixel is evaluated subsampling^2 points on a grid within each pixel. This is to ensure each pixel is assigned a mean pixel value and not the value at the center.
outputgrid (bool, optional) – Outputs coordinate grid.
outputexterior (bool, optional) – Ouput exterior information, including exterior border dtfe and current unassigned pixels.

Returns:

dens (ndarray) – Density field values on a grid.
x2d, y2d (ndarray, optional) – Pixel coordinate points.

fiesta.dtfe.mpi_delaunay_field4grid2D(x: ndarray, y: ndarray, f: ndarray, boxsize: float | List[float], ngrid: int | List[int], MPI: object, origin: float | List[float] = 0.0, mass: ndarray | None = None, partition: int = 1, periodic: bool | List[bool] = True, fbuffer: float = 0.5, subsampling: int = 1, outputgrid: bool = False)

Returns the Delaunay tesselation field on a grid.

Parameters:

x (array) – Coordinates of particles.
y (array) – Coordinates of particles.
f (array) – Field values to be interpolated.
boxsize (float or list) – Dimensions of the grid.
ngrid (int or int list) – Grid dimensions.
MPI (class object) – shift.mpiutils MPI object.
origin (float or list, optional) – Origin for grid.
mass (array, optional) – Mass or mass weights for each the particles.
partition (int or list, optional) – The number of internal Delaunay tesselations used to compute the field grid.
periodic (bool or list, optional) – Determines whether periodic boundaries are applied.
fbuffer (float, optional) – Buffer factor length, used to trim exterior buffer region.
subsampling (int, optional) – The pixel subsampling rate. Each pixel is evaluated subsampling^2 points on a grid within each pixel. This is to ensure each pixel is assigned a mean pixel value and not the value at the center.
outputgrid (bool, optional) – Outputs coordinate grid.
outputexterior (bool, optional) – Ouput exterior information, including exterior border dtfe and current unassigned pixels.

Returns:

field (ndarray) – Density field values on a grid.
x2d, y2d (ndarray, optional) – Pixel coordinate points.

fiesta.dtfe.mpi_delaunay_density4grid3D(x: ndarray, y: ndarray, z: ndarray, boxsize: float | List[float], ngrid: int | List[int], MPI: object, origin: float | List[float] = 0.0, mass: ndarray | None = None, partition: int = 1, periodic: bool | List[bool] = True, fbuffer: float = 0.5, subsampling: int = 1, outputgrid: bool = False) → ndarray | Tuple[ndarray, ndarray, ndarray]

Returns the Delaunay tesselation density on a grid.

Parameters:

x (array) – Coordinates of particles.
y (array) – Coordinates of particles.
z (array) – Coordinates of particles.
boxsize (float or list) – Dimensions of the grid.
ngrid (int or int list) – Grid dimensions.
MPI (class object) – shift.mpiutils MPI object.
origin (float or list, optional) – Origin for grid.
mass (array, optional) – Mass or mass weights for each the particles.
partition (int or list, optional) – The number of internal Delaunay tesselations used to compute the density grid.
periodic (bool or list, optional) – Determines whether periodic boundaries are applied.
fbuffer (float, optional) – Buffer factor length, used to trim exterior buffer region.
subsampling (int, optional) – The pixel subsampling rate. Each pixel is evaluated subsampling^2 points on a grid within each pixel. This is to ensure each pixel is assigned a mean pixel value and not the value at the center.
outputgrid (bool, optional) – Outputs coordinate grid.
outputexterior (bool, optional) – Ouput exterior information, including exterior border dtfe and current unassigned pixels.

Returns:

dens (ndarray) – Density field values on a grid.
x3d, y3d, z3d (ndarray, optional) – Pixel coordinate points.

Returns the Delaunay tesselation field on a grid.

Parameters:

x (array) – Coordinates of particles.
y (array) – Coordinates of particles.
f (array) – Field values to be interpolated.
boxsize (float or list) – Dimensions of the grid.
ngrid (int or int list) – Grid dimensions.
MPI (class object) – shift.mpiutils MPI object.
origin (float or list, optional) – Origin for grid.
mass (array, optional) – Mass or mass weights for each the particles.
partition (int or list, optional) – The number of internal Delaunay tesselations used to compute the field grid.
periodic (bool or list, optional) – Determines whether periodic boundaries are applied.
fbuffer (float, optional) – Buffer factor length, used to trim exterior buffer region.
subsampling (int, optional) – The pixel subsampling rate. Each pixel is evaluated subsampling^2 points on a grid within each pixel. This is to ensure each pixel is assigned a mean pixel value and not the value at the center.
outputgrid (bool, optional) – Outputs coordinate grid.
outputexterior (bool, optional) – Ouput exterior information, including exterior border dtfe and current unassigned pixels.

Returns:

field (ndarray) – Density field values on a grid.
x2d, y2d (ndarray, optional) – Pixel coordinate points.