sdf_xarray.open_mfdataset

Contents

sdf_xarray.open_mfdataset#

sdf_xarray.open_mfdataset(paths, *, separate_times=False, keep_particles=False, probe_names=None, data_vars=None, chunks='auto', deck_path=None)[source]#

Open a set of EPOCH SDF files as one xarray.Dataset. Variables related to boundaries, cpu and output file are excluded as they are problematic. If you wish to load these variables in see Loading raw files.

EPOCH can output variables at different periods, so each individal SDF file from one EPOCH run may have different variables in it. In order to combine all files into one xarray.Dataset, we need to concatenate variables across their time dimension.

We have two choices:

  1. One time dimension where some variables may not be defined at all time points, and so will be filled with NaNs at missing points; or

  2. Multiple time dimensions, one for each output frequency

The second option is better for memory consumption, as the missing data with the first option still takes up space. However, proper lazy-loading may mitigate this.

The separate_times argument can be used to switch between these choices.

Parameters:

  • paths (Iterable | str | Path | Callable[..., Iterable[Path]]) – List of filenames or string glob pattern

  • separate_times (bool) – If True, create separate time dimensions for variables defined at different output frequencies

  • keep_particles (bool) – If True, also load particle data (this may use a lot of memory!)

  • probe_names (list[str] | None) – List of EPOCH probe names

  • data_vars (list[str] | None) – List of data vars to load in (If not specified loads in all variables)

  • chunks (Union[str, int, Literal['auto'], tuple[int, ...], None, Mapping[Any, Union[str, int, Literal['auto'], tuple[int, ...], None]]]) – Dictionary with keys given by dimension names and values given by chunk sizes. In general, these should divide the dimensions of each dataset. By default chunks are automatically set so that they are the same size as the dimensions stored in each of the SDF files. See Xarray chunking-and-performance for details on why this is useful for large datasets. The default behaviour is to do this automatically and can be disabled by chunks=None.

  • deck_path (str | PathLike | None) – If None, attempt to load the "input.deck" from the same directory as the SDF files and silently fail if it does not exist. If a path is given, load the specified deck from a relative or absolute file path. See Loading the input.deck for details.

Return type:

Dataset

Contents