wrfhydropy.core package

Submodules

wrfhydropy.core.domain module

class wrfhydropy.core.domain.Domain(domain_top_dir: str, domain_config: str, compatible_version: str | None = None, hydro_namelist_patch_file: str = 'hydro_namelist_patches.json', hrldas_namelist_patch_file: str = 'hrldas_namelist_patches.json')

Bases: object

Class for a WRF-Hydro domain, which constitutes all domain-specific files needed for a setup.

Methods

copy_files(dest_dir[, symlink])

Copy domain files to a new directory :Parameters: * dir -- The destination directory for domain files * symlink -- Symlink domain files instead of copy

compatible_version

Source-code version for which the domain is to be used.

Type:

str

copy_files(dest_dir: str, symlink: bool = True)

Copy domain files to a new directory :Parameters: * dir – The destination directory for domain files

• symlink – Symlink domain files instead of copy

domain_config

Specified configuration for which the domain is to be used, e.g. ‘NWM_ana’

Type:

str

domain_top_dir

pathlib.Paths to *.TBL files generated at compile-time.

Type:

pathlib.Path

forcing_dir

path to the forcing directory

Type:

pathlib.Path

hrldas_namelist_patches

Domain-specific hrldas namelist settings.

Type:

Namelist

hydro_files

Files specified in hydro_nlist section of the domain namelist patches

Type:

list

hydro_namelist_patches

Domain-specific hydro namelist settings.

Type:

Namelist

lsm_files

Files specified in noahlsm_offline section of the domain namelist patches

Type:

list

nudging_dir

path to the nudging obs directory

Type:

pathlib.Path

nudging_files

Files specified in nudging_nlist section of the domain namelist patches

Type:

list

wrfhydropy.core.ensemble module

class wrfhydropy.core.ensemble.EnsembleSimulation(ncores: int = 1)

Bases: object

Class for a WRF-Hydro EnsembleSimulation object. The Ensemble Simulation object is used to orchestrate a set of ‘N’ WRF-Hydro simulations. It requires members with pre-compiled models and there are set and get methods across the ensemble (member_diffs & set_member_diffs). Jobs and scheduler set on the EnsembleSimulation object are set on all the members.

Attributes:

N

member_diffs

Get method for ensemble member differences.

Methods

add(obj)	Add an approparite object to an EnsembleSimulation, such as a Simulation, Job, or Scheduler.
compose([symlink_domain, force, ...])	Ensemble compose simulation directories and files :Parameters: * symlink_domain -- Symlink the domain files rather than copy * force -- Compose into directory even if not empty. This is considered bad practice but * is necessary in certain circumstances. * rm_members_from_memory -- Most applications will remove the members from the * ensemble object upon compose. Testing and other reasons may keep them around. * check_nlst_warn -- Allow the namelist checking/validation to only result in warnings. * This is also not great practice, but necessary in certain circumstances..
pickle(path)	Pickle ensemble sim object to specified file path :Parameters: path -- The file path for pickle
replicate_member(N[, copy_members])
restore_members([ens_dir, recursive])	Restore members from disk, replace paths with the loaded pickle.
rm_members()	Remove members from memory, replace with their paths.
run([n_concurrent, teams, teams_exe_cmd, ...])	Run the ensemble of simulations. Inputs: n_concurrent: int = 1, Only used for non-team runs. teams: bool = False, Use teams? See parallel_teams_run for details. teams_exe_cmd: str, The mpi-specific syntax needed. For example: 'mpirun --hosts {nodelist} -np {nproc} {cmd}' teams_exe_cmd_nproc: int, The number of cores per model/wrf_hydro simulation to be run. teams_node_file: dict = None, Optional file that acts like a node file. It is not currently implemented but the key specifies the scheduler format that the file follows. An example pbs node file is in tests/data and this argument is used here to test without a sched. env: dict = None, optional envionment to pass to the run. teams_dict: dict, Skip the arguments if you already have a teams_dict to use (backwards compatibility) Returns: 0 for success.
set_member_diffs(att_tuple, values)	Set method for ensemble member differences.

collect

property N

add(obj: list | Scheduler | Job)

Add an approparite object to an EnsembleSimulation, such as a Simulation, Job, or Scheduler. :Parameters: obj – the object to add.

collect(output=True)

compose(symlink_domain: bool = True, force: bool = False, check_nlst_warn: bool = False, rm_members_from_memory: bool = True)

Ensemble compose simulation directories and files :Parameters: * symlink_domain – Symlink the domain files rather than copy

• force – Compose into directory even if not empty. This is considered bad practice but

• is necessary in certain circumstances.

• rm_members_from_memory – Most applications will remove the members from the

• ensemble object upon compose. Testing and other reasons may keep them around.

• check_nlst_warn – Allow the namelist checking/validation to only result in warnings.

• This is also not great practice, but necessary in certain circumstances.

jobs

a list containing Job objects

Type:

list

property member_diffs

Get method for ensemble member differences. Only differences are reported.

members

a list of simulations which are the members of the ensemble.

Type:

list

ncores

integer number of cores for running parallelizable methods.

Type:

ncores

pickle(path: str)

Pickle ensemble sim object to specified file path :Parameters: path – The file path for pickle

replicate_member(N: int, copy_members: bool = True)

restore_members(ens_dir: Path | None = None, recursive: bool = True)

Restore members from disk, replace paths with the loaded pickle.

rm_members()

Remove members from memory, replace with their paths.

run(n_concurrent: int = 1, teams: bool = False, teams_exe_cmd: str | None = None, teams_exe_cmd_nproc: int | None = None, teams_node_file: dict | None = None, env: dict | None = None, teams_dict: dict | None = None)

Run the ensemble of simulations. Inputs:

n_concurrent: int = 1, Only used for non-team runs. teams: bool = False, Use teams? See parallel_teams_run for

details.

teams_exe_cmd: str, The mpi-specific syntax needed. For

example: ‘mpirun –hosts {nodelist} -np {nproc} {cmd}’

teams_exe_cmd_nproc: int, The number of cores per model/wrf_hydro

simulation to be run.

teams_node_file: dict = None, Optional file that acts like a node

file. It is not currently implemented but the key specifies the scheduler format that the file follows. An example pbs node file is in tests/data and this argument is used here to test without a sched.

env: dict = None, optional envionment to pass to the run. teams_dict: dict, Skip the arguments if you already have a

teams_dict to use (backwards compatibility)

Returns: 0 for success.

scheduler

A scheduler object to use for each Job in self.jobs

Type:

Scheduler

set_member_diffs(att_tuple: tuple, values: list)

Set method for ensemble member differences. (Currently fails silently when requested fields are not found.)

wrfhydropy.core.ensemble.parallel_compose(arg_dict)

Parallelizable function to compose an EnsembleSimuation.

wrfhydropy.core.ensemble.parallel_compose_addjobs(arg_dict)

Parallelizable function to add jobs to EnsembleSimuation.

wrfhydropy.core.ensemble.parallel_compose_addscheduler(arg_dict)

Parallelizable function to add a scheduler to EnsembleSimuation.

wrfhydropy.core.ensemble.parallel_run(arg_dict)

Parallelizable function to run an EnsembleSimuation.

wrfhydropy.core.ensemble_tools module

class wrfhydropy.core.ensemble_tools.DeepDiffEq(t1, t2, eq_types, ignore_order=False, report_repetition=False, significant_digits=None, exclude_paths={}, exclude_types={}, verbose_level=1, view='text', **kwargs)

Bases: DeepDiff

Extend Deep Diff to handle __eq__ for specified types.

Attributes:

affected_paths

Get the list of paths that were affected.

affected_root_keys

Get the list of root keys that were affected.

Methods

clear()
copy()
custom_report_result(report_type, level[, ...])	Add a detected change to the reference-style result dictionary. report_type will be added to level. (We'll create the text-style report from there later.) :param report_type: A well defined string key describing the type of change. Examples: "set_item_added", "values_changed" :param parent: A DiffLevel object describing the objects in question in their before-change and after-change object structure. :param extra_info: A dict that describe this result :rtype: None.
from_json_pickle(value)	from_json_pickle_label Load DeepDiff object with all the bells and whistles from the json pickle dump.
fromkeys(iterable[, value])	Create a new dictionary with keys from iterable and values set to value.
get(key[, default])	Return the value for key if key is in the dictionary, else default.
get_stats()	Get some stats on internals of the DeepDiff run.
items()
keys()
pop(key[, default])	If the key is not found, return the default if given; otherwise, raise a KeyError.
popitem(/)	Remove and return a (key, value) pair as a 2-tuple.
pretty()	The pretty human readable string output for the diff object regardless of what view was used to generate the diff.
remove_empty_keys()	Remove empty keys from this object.
setdefault(key[, default])	Insert key with a value of default if key is not in the dictionary.
to_dict([view_override])	convert the result to a python dictionary.
to_json([default_mapping])	Dump json of the text view.
to_json_pickle()	to_json_pickle_label Get the json pickle of the diff object.
update([E, ]**F)	If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
values()

get_ignore_types_in_groups
get_significant_digits

wrfhydropy.core.ensemble_tools.dictify(obj)

Recursively transform deepcopy sub __dicts__ of an object into dicts for dictionary traversal of a deepcopy of the object.

wrfhydropy.core.ensemble_tools.get_ens_dotfile_end_datetime(run_dir)

Use the the .model_end_time files to get the current ensemble time.

wrfhydropy.core.ensemble_tools.get_sub_objs(obj)

Identify which attributes of an object are objects with __dicts__.

wrfhydropy.core.ensemble_tools.is_sub_obj(obj)

Test if an object is has a __dict__ (may not be the best definition of an object, but it works for classes in wrfhydropy).

wrfhydropy.core.ensemble_tools.mute()

A initializer for multiprocessing.Pool to keep the processes quiet.

wrfhydropy.core.ioutils module

class wrfhydropy.core.ioutils.WrfHydroStatic(*args, **kwargs)

Bases: PosixPath

WRF-Hydro static data class

Attributes:

anchor

The concatenation of the drive and root, or ‘’.

drive

The drive prefix (letter or UNC path), if any.

name

The final path component, if any.

parent

The logical parent of the path.

parents

A sequence of this path’s logical parents.

parts

An object providing sequence-like access to the components in the filesystem path.

root

The root of the path, if any.

stem

The final path component, minus its last suffix.

suffix

The final component’s last suffix, if any.

suffixes

A list of the final component’s suffixes, if any.

Methods

absolute()	Return an absolute version of this path.
as_posix()	Return the string representation of the path with forward (/) slashes.
as_uri()	Return the path as a 'file' URI.
check_nans()	Return dictionary of counts of NA values for each data variable
chmod(mode, *[, follow_symlinks])	Change the permissions of the path, like os.chmod().
cwd()	Return a new path pointing to the current working directory (as returned by os.getcwd()).
exists()	Whether this path exists.
expanduser()	Return a new path with expanded ~ and ~user constructs (as returned by os.path.expanduser)
glob(pattern)	Iterate over this subtree and yield all existing files (of any kind, including directories) matching the given relative pattern.
group()	Return the group name of the file gid.
hardlink_to(target)	Make this path a hard link pointing to the same file as target.
home()	Return a new path pointing to the user's home directory (as returned by os.path.expanduser('~')).
is_absolute()	True if the path is absolute (has both a root and, if applicable, a drive).
is_block_device()	Whether this path is a block device.
is_char_device()	Whether this path is a character device.
is_dir()	Whether this path is a directory.
is_fifo()	Whether this path is a FIFO.
is_file()	Whether this path is a regular file (also True for symlinks pointing to regular files).
is_mount()	Check if this path is a POSIX mount point
is_relative_to(*other)	Return True if the path is relative to another path or False.
is_reserved()	Return True if the path contains one of the special names reserved by the system, if any.
is_socket()	Whether this path is a socket.
is_symlink()	Whether this path is a symbolic link.
iterdir()	Iterate over the files in this directory.
joinpath(*args)	Combine this path with one or several arguments, and return a new path representing either a subpath (if all arguments are relative paths) or a totally different path (if one of the arguments is anchored).
lchmod(mode)	Like chmod(), except if the path points to a symlink, the symlink's permissions are changed, rather than its target's.
link_to(target)	Make the target path a hard link pointing to this path.
lstat()	Like stat(), except if the path points to a symlink, the symlink's status information is returned, rather than its target's.
match(path_pattern)	Return True if this path matches the given pattern.
mkdir([mode, parents, exist_ok])	Create a new directory at this given path.
open()	Open a WrfHydroStatic object :Parameters: self
owner()	Return the login name of the file owner.
read_bytes()	Open the file in bytes mode, read it, and close the file.
read_text([encoding, errors])	Open the file in text mode, read it, and close the file.
readlink()	Return the path to which the symbolic link points.
relative_to(*other)	Return the relative path to another path identified by the passed arguments.
rename(target)	Rename this path to the target path.
replace(target)	Rename this path to the target path, overwriting if that path exists.
resolve([strict])	Make the path absolute, resolving all symlinks on the way and also normalizing it (for example turning slashes into backslashes under Windows).
rglob(pattern)	Recursively yield all existing files (of any kind, including directories) matching the given relative pattern, anywhere in this subtree.
rmdir()	Remove this directory.
samefile(other_path)	Return whether other_path is the same or not as this file (as returned by os.path.samefile()).
stat(*[, follow_symlinks])	Return the result of the stat() system call on this path, like os.stat() does.
symlink_to(target[, target_is_directory])	Make this path a symlink pointing to the target path.
touch([mode, exist_ok])	Create this file with the given access mode, if it doesn't exist.
unlink([missing_ok])	Remove this file or link.
with_name(name)	Return a new path with the file name changed.
with_stem(stem)	Return a new path with the stem changed.
with_suffix(suffix)	Return a new path with the file suffix changed.
write_bytes(data)	Open the file in bytes mode, write to it, and close the file.
write_text(data[, encoding, errors, newline])	Open the file in text mode, write to it, and close the file.

check_nans()

Return dictionary of counts of NA values for each data variable

open()

Open a WrfHydroStatic object :Parameters: self

Returns:

An xarray dataset object.

class wrfhydropy.core.ioutils.WrfHydroTs(iterable=(), /)

Bases: list

WRF-Hydro netcdf timeseries data class

Methods

append(object, /)	Append object to the end of the list.
check_nans()	Return dictionary of counts of NA values for each data variable summed across files
clear(/)	Remove all items from list.
copy(/)	Return a shallow copy of the list.
count(value, /)	Return number of occurrences of value.
extend(iterable, /)	Extend list by appending elements from the iterable.
index(value[, start, stop])	Return first index of value.
insert(index, object, /)	Insert object before index.
open([chunks, forecast])	Open a WrfHydroTs object :Parameters: * self * chunks -- chunks argument passed on to xarray.DataFrame.chunk() method * forecast -- If forecast the reference time dimension is retained, if not then * reference_time dimension is set to a dummy value (1970-01-01) * and analysis
pop([index])	Remove and return item at index (default last).
remove(value, /)	Remove first occurrence of value.
reverse(/)	Reverse IN PLACE.
sort(*[, key, reverse])	Sort the list in ascending order and return None.

check_nans()

Return dictionary of counts of NA values for each data variable summed across files

open(chunks: dict | None = None, forecast: bool = True)

Open a WrfHydroTs object :Parameters: * self

• chunks – chunks argument passed on to xarray.DataFrame.chunk() method

• forecast – If forecast the reference time dimension is retained, if not then

• reference_time dimension is set to a dummy value (1970-01-01)

• and analysis

Returns:

An xarray mfdataset object concatenated on dimension ‘Time’.

wrfhydropy.core.ioutils.check_file_nans(dataset_or_path: str | Path | Dataset, n_cores: int = 1) → DataFrame | None

Opens the specified netcdf file and checks all data variables for NA values. NA assigned according to xarray __FillVal parsing. See xarray.Dataset documentation :Parameters: dataset_or_path – The path to the netcdf dataset file, or a dataset itself

Returns: string summary of nans if present

wrfhydropy.core.ioutils.check_input_files(hydro_namelist: dict, hrldas_namelist: dict, sim_dir: str, ignore_restarts: bool = False, check_nlst_warn: bool = False)

Given hydro and hrldas namelists and a directory, check that all files listed in the namelist exist in the specified directory. :Parameters: * hydro_namelist – A wrfhydropy hydro_namelist dictionary

• hrldas_namelist – A wrfhydropy hydro_namelist dictionary

• file_str – A wrfhydropy hrldas_namelist dictionary

• sim_dir – The path to the directory containing input files.

• ignore_restarts – Ignore restart files.

• check_nlst_warn – Allow the namelist checking/validation to only result in warnings.

wrfhydropy.core.ioutils.group_lead_time(ds: Dataset) → int

wrfhydropy.core.ioutils.group_member(ds: Dataset) → int

wrfhydropy.core.ioutils.group_member_lead_time(ds: Dataset) → int

wrfhydropy.core.ioutils.is_not_none(x)

wrfhydropy.core.ioutils.md5(fname)

wrfhydropy.core.ioutils.merge_lead_time(ds_list: list) → Dataset

wrfhydropy.core.ioutils.merge_member(ds_list: list) → Dataset

wrfhydropy.core.ioutils.merge_reference_time(ds_list: list) → Dataset

wrfhydropy.core.ioutils.merge_time(ds_list: list) → Dataset

wrfhydropy.core.ioutils.nwm_forcing_to_ldasin(nwm_forcing_dir: Path | str, ldasin_dir: Path | str, range: str, copy: bool = False, forc_type=1)

Convert nwm dir and naming format to wrf-hydro read format. :Parameters: * nwm_forcing_dir – the pathlib.Path or str for the source dir or a list of source

directories. If a pathlib.Path object or str is provided, it is assume that this single directory contains nwm.YYYYMMDDHH downloaded from NOMADS and that their subdirectory structure is unchanged. If a list of pathlib.Path (or str) is provided, these should be the desired nwm.YYYYMMDD to translate with no changed to their subdirectory structure.

• ldasin_dir – the pathlib.Path or str for a new NONEXISTANT output dir.

• range – str range as on nomads in: analysis_assim, analysis_assim_extend, analysis_assim_hawaii, medium_range, short_range, short_range_hawaii

• copy – True or false. Default is false creates symlinks.

• forc_type – 1 (hour) or 2 (minute) formats are supported.

Returns:

None on success.

wrfhydropy.core.ioutils.open_dart_dataset(paths: list, chunks: dict | None = None, spatial_indices: list | None = None, drop_variables: list | None = None, npartitions: int | None = None, attrs_keep: list | None = None) → Dataset

Open a multi-file ensemble wrf-hydro output dataset

Args:

paths: List ,iterable, or generator of file paths to wrf-hydro netcdf output files

chunks: chunks argument passed on to xarray DataFrame.chunk() method preprocess_member: A function that identifies the member from the file or filename. attrs_keep: A list of the global attributes to be retained.

Returns:

An xarray dataset of dask arrays chunked by chunk_size along the feature_id dimension concatenated along the time and member dimensions.

wrfhydropy.core.ioutils.open_ensemble_dataset(paths: list, chunks: dict | None = None, preprocess_member: callable = <function preprocess_dart_member>, attrs_keep: list | None = None) → Dataset

Open a multi-file ensemble wrf-hydro output dataset :Parameters: * paths – List ,iterable, or generator of file paths to wrf-hydro netcdf output files

• chunks – chunks argument passed on to xarray DataFrame.chunk() method

• preprocess_member – A function that identifies the member from the file or filename.

• attrs_keep – A list of the global attributes to be retained.

Returns:

An xarray dataset of dask arrays chunked by chunk_size along the feature_id dimension concatenated along the time and member dimensions.

wrfhydropy.core.ioutils.open_nwm_dataset(paths: list, chunks: dict | None = None, attrs_keep: list = ['featureType', 'proj4', 'station_dimension', 'esri_pe_string', 'Conventions', 'model_version'], spatial_indices: list | None = None, drop_variables: list | None = None, npartitions: int | None = None, profile: int = False) → Dataset

wrfhydropy.core.ioutils.open_wh_dataset(paths: list, chunks: dict | None = None, forecast: bool = True) → Dataset

Open a multi-file wrf-hydro output dataset :Parameters: * paths – List ,iterable, or generator of file paths to wrf-hydro netcdf output files

• chunks – chunks argument passed on to xarray DataFrame.chunk() method

• forecast – If forecast the reference time dimension is retained, if not then

• reference_time dimension is set to a dummy value (1970-01-01)

• and analysis

Returns:

An xarray dataset of dask arrays chunked by chunk_size along the feature_id dimension concatenated along the time and reference_time dimensions

wrfhydropy.core.ioutils.preprocess_dart_data(path, chunks: dict | None = None, spatial_indices: list | None = None, drop_variables: list | None = None) → Dataset

wrfhydropy.core.ioutils.preprocess_dart_member(ds)

wrfhydropy.core.ioutils.preprocess_nwm_data(path, spatial_indices: list | None = None, drop_variables: list | None = None) → Dataset

wrfhydropy.core.ioutils.sort_files_by_time(file_list: list)

Given a list of file paths, sort list by file modified time :Parameters: file_list – The list of file paths to sort

Returns: A list of file paths sorted by file modified time

wrfhydropy.core.ioutils.timesince(when=None)

wrfhydropy.core.job module

class wrfhydropy.core.job.Job(job_id: str, model_start_time: str | datetime | None = None, model_end_time: str | datetime | None = None, restart_freq_hr: int | dict | None = None, output_freq_hr: int | dict | None = None, restart: bool = True, restart_file_time: str | datetime | dict | None = None, restart_dir: str | Path | dict | None = None, exe_cmd: str | None = None, entry_cmd: str | None = None, exit_cmd: str | None = None)

Bases: object

A Job represents run-time specific information for a given WRF-Hydro run. A Simulation consists of one or more jobs. For example, adding multiple Jobs can be used to split a Simulation into multiple runs to limit the wall-clock duration of each individual run.

Attributes:

hrldas_namelist

hrldas_times

hydro_namelist

hydro_times

job_dir

Path: Path to the run directory

model_end_time

datetime: The model time at the end of the execution.

model_start_time

datetime: The model time at the start of the execution.

Methods

clone(N)	Clone a job object N-times using deepcopy.
pickle(path)	Pickle job object to specified file path :Parameters: path -- The file path for pickle

clone(N) → list

Clone a job object N-times using deepcopy. :Parameters: N – The number of time to clone the Job

Returns:

A list of Job objects

exit_status

The exit status of the model job parsed from WRF-Hydro diag files

Type:

int

property hrldas_namelist

property hrldas_times

property hydro_namelist

property hydro_times

property job_dir

Path to the run directory

Type:

Path

job_id

The job id.

Type:

str

property model_end_time

The model time at the end of the execution.

Type:

datetime

property model_start_time

The model time at the start of the execution.

Type:

datetime

output_freq_hr_hrldas

Hrldas output write frequency in hours.

Type:

int

output_freq_hr_hydro

Hydro output write frequency in hours.

Type:

int

pickle(path: str)

Pickle job object to specified file path :Parameters: path – The file path for pickle

restart

Start model from a restart.

Type:

bool

restart_file_time

Time on the restart file to use, if different from model_start_time. The path in any supplied restart file path in the namelists is preserved while modifying the date and time.

Type:

np.datetime

restart_freq_hr_hrldas

Hrldas restart write frequency in hours.

Type:

int

restart_freq_hr_hydro

Hydro restart write frequency in hours.

Type:

int

wrfhydropy.core.model module

class wrfhydropy.core.model.Model(source_dir: str, model_config: str, hydro_namelist_config_file: str | None = None, hrldas_namelist_config_file: str | None = None, compile_options_config_file: str | None = None, compiler: str = 'gfort', pre_compile_cmd: str | None = None, compile_options: dict | None = None)

Bases: object

Class for a WRF-Hydro model, which consitutes the model source code and compiled binary.

Methods

compile(compile_dir)	Compiles WRF-Hydro using specified compiler and compile options.
copy_files(dest_dir[, symlink])	Copy domain files to new directory :Parameters: * dest_dir -- The destination directory for files * symlink -- Symlink files instead of copy

compile(compile_dir: Path) → str

Compiles WRF-Hydro using specified compiler and compile options. :Parameters: compile_dir – A non-existant directory to use for compilation.

Returns:

Success of compilation and compile directory used. Sets additional attributes to WrfHydroModel

compile_dir

pathlib.Path object pointing to the compile directory.

Type:

pathlib.Path

compile_log

The subprocess object generated at compile.

Type:

CompletedProcess

compile_options

Hydro namelist for specified model config

Type:

Namelist

compile_options_config_file

Compile options file specified for model config.

Type:

Namelist

compiler

The compiler chosen at compile time.

Type:

str

configure_log

The subprocess object generated at configure.

Type:

CompletedProcess

copy_files(dest_dir: str, symlink: bool = True)

Copy domain files to new directory :Parameters: * dest_dir – The destination directory for files

• symlink – Symlink files instead of copy

git_hash

The git revision hash if seld.source_dir is a git repository

Type:

str

hrldas_namelist_config_file

HRLDAS namelist file specified for model config.

Type:

Namelist

hrldas_namelists

HRLDAS namelist for specified model config

Type:

Namelist

hydro_namelist_config_file

Hydro namelist file specified for model config

Type:

Namelist

hydro_namelists

Hydro namelist for specified model config

Type:

Namelist

model_config

Specified configuration for which the model is to be used, e.g. ‘nwm_ana’

Type:

str

object_id

A unique id to join object to compile directory.

Type:

str

pre_compile_cmd

Command string to be executed prior to model compilation, e.g. to load modules

Type:

str

source_dir

pathlib.Path object for source code directory.

Type:

pathlib.Path

table_files

pathlib.Paths to *.TBL files generated at compile-time.

Type:

list

version

Source code version from .version file stored with the source code.

Type:

str

wrf_hydro_exe

pathlib.Path to wrf_hydro.exe file generated at compile-time.

Type:

pathlib.Path

wrfhydropy.core.model.get_git_revision_hash(the_dir: str) → str

Get the last git revision hash from a directory if directory is a git repository :Parameters: the_dir – String for the directory path

Returns:

String with the git hash if a git repo or message if not

wrfhydropy.core.namelist module

class wrfhydropy.core.namelist.JSONNamelist(file_path: str)

Bases: object

Class for a WRF-Hydro JSON namelist containing one more configurations

Methods

get_config(config)

Get a namelist for a given configuration.

get_config(config: str)

Get a namelist for a given configuration. This works internally by grabbing the base namelist and updating with the config-specific changes. :Parameters: config – The configuration to retrieve

class wrfhydropy.core.namelist.Namelist

Bases: dict

Class for a WRF-Hydro namelist

Methods

clear()
copy()
fromkeys(iterable[, value])	Create a new dictionary with keys from iterable and values set to value.
get(key[, default])	Return the value for key if key is in the dictionary, else default.
items()
keys()
patch(patch)	Recursively patch a namelist with key values from another namelist :Parameters: patch -- A Namelist or dict object containing the patches
pop(key[, default])	If the key is not found, return the default if given; otherwise, raise a KeyError.
popitem(/)	Remove and return a (key, value) pair as a 2-tuple.
setdefault(key[, default])	Insert key with a value of default if key is not in the dictionary.
update([E, ]**F)	If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]
values()
write(path[, mode])	Write a namelist to file as a fortran-compatible namelist :Parameters: path -- The file path

patch(patch: dict)

Recursively patch a namelist with key values from another namelist :Parameters: patch – A Namelist or dict object containing the patches

write(path: str, mode='x')

Write a namelist to file as a fortran-compatible namelist :Parameters: path – The file path

wrfhydropy.core.namelist.dict_merge(dct: dict, merge_dct: dict) → dict

Recursive dict merge. Inspired by :meth:dict.update(), instead of updating only top-level keys, dict_merge recurses down into dicts nested to an arbitrary depth, updating keys. The merge_dct is merged into dct. :Parameters: * dct – dict onto which the merge is executed

• merge_dct – dct merged into dct

Returns:

The merged dict

wrfhydropy.core.namelist.diff_namelist(old_namelist: Namelist | str, new_namelist: Namelist | str, **kwargs) → dict

Diff two Namelist objects or fortran 90 namelist files and return a dictionary of differences.

Parameters:

• old_namelist – String containing path to the first namelist file, referred to as ‘old’ in

• outputs.

• new_namelist – String containing path to the second namelist file, referred to as ‘new’ in

• outputs.

• **kwargs – Additional arguments passed onto deepdiff.DeepDiff method

Returns:

The differences between the two namelists

wrfhydropy.core.namelist.load_namelist(nml_path: str) → dict

Load a F90 namelist into a wrfhydropy.Namelist object :Parameters: nml_path – String containing path to F90 namelist

Returns:

dict interpretation of namelist

wrfhydropy.core.outputdiffs module

class wrfhydropy.core.outputdiffs.OutputDataDiffs(candidate_output: SimulationOutput, reference_output: SimulationOutput, nccmp_options: list | None = None, exclude_vars: list | None = None, exclude_atts: list | None = None, xrcmp_n_cores: int = 0)

Bases: object

channel_rt

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

channel_rt_grid

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

chanobs

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

diff_counts

Counts of diffs by restart type

Type:

dict

gwout

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

lakeout

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

ldasout

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

restart_hydro

List of pandas dataframes if possible or subprocess objects containing hydro restart file diffs

Type:

list

restart_lsm

List of pandas dataframes if possible or subprocess objects containing lsm restart file diffs

Type:

list

restart_nudging

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

rtout

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

class wrfhydropy.core.outputdiffs.OutputMetaDataDiffs(candidate_output: SimulationOutput, reference_output: SimulationOutput, stats_only=False, nccmp_options: list | None = None, exclude_vars: list | None = None, exclude_atts: list | None = None, xrcmp_n_cores: int = 0)

Bases: object

channel_rt

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

chanobs

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

diff_counts

Counts of diffs by restart type

Type:

dict

gwout

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

lakeout

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

ldasout

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

restart_hydro

List of pandas dataframes if possible or subprocess objects containing hydro restart file diffs

Type:

list

restart_lsm

List of pandas dataframes if possible or subprocess objects containing lsm restart file diffs

Type:

list

restart_nudging

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

rtout

List of pandas dataframes if possible or subprocess objects containing nudging restart file diffs

Type:

list

wrfhydropy.core.outputdiffs.check_unprocessed_diffs(unexpected_diffs: list)

Function to the unexpected_diffs of DeepDiff :Parameters: unexpected_diffs – The unexpected diffs list

Returns:

None

wrfhydropy.core.outputdiffs.compare_ncfiles(candidate_files: list, reference_files: list, stats_only: bool = False, nccmp_options: list | None = None, exclude_vars: list | None = None, exclude_atts: list | None = None, xrcmp_n_cores: int = 0)

Compare lists of netcdf restart files element-wise. Files must have common names :Parameters: * candidate_files – List of candidate netcdf file paths

• reference_files – List of reference netcdf file paths

• stats_only – Only return statistics on differences in data values

• nccmp_options – List of long-form command line options passed to nccmp,

• see http – //nccmp.sourceforge.net/ for options. Defaults are ‘–metadata’, ‘–force’

• exclude_vars – A list of strings containing variables names to

• exclude from the comparison.

• exclude_atts – A list of strings containing attribute names to exclude from the

• comparison. Defaults are ‘valid_min’

Returns:

A named list of either pandas dataframes if possible or subprocess objects

wrfhydropy.core.schedulers module

class wrfhydropy.core.schedulers.PBSCheyenne(account: str, nproc: int, nnodes: int, mem: int | None = None, ppn: int | None = None, queue: str = 'regular', walltime: str = '12:00:00', email_who: str | None = None, email_when: str = 'abe', custom: dict = {})

Bases: Scheduler

A Scheduler object compatible with PBS on the NCAR Cheyenne system.

Attributes:

nnodes

nproc

ppn

Methods

schedule(jobs)

Schedule one or more jobs using the scheduler scheduler :Parameters: jobs -- list of jobs to schedule

property nnodes

property nproc

property ppn

schedule(jobs: list)

Schedule one or more jobs using the scheduler scheduler :Parameters: jobs – list of jobs to schedule

class wrfhydropy.core.schedulers.Scheduler

Bases: ABC

Methods

schedule

abstract schedule(jobs)

wrfhydropy.core.simulation module

class wrfhydropy.core.simulation.Simulation

Bases: object

Class for a WRF-Hydro Simulation object. The Simulation object is used to orchestrate a WRF-Hydro simulation by accessing methods of Model, Domain, and Job objects. Optionally, a scheduler can also be added.

Methods

add(obj)	Add an approparite object to a Simulation, such as a Model, Domain, Job, or Scheduler
collect([sim_dir, output])	Collect simulation output after a run
compose([symlink_domain, force, check_nlst_warn])	Compose simulation directories and files :Parameters: * symlink_domain -- Symlink the domain files rather than copy * force -- Compose into directory even if not empty. This is considered bad practice but * is necessary in certain circumstances. * check_nlst_warn -- Allow the namelist checking/validation to only result in warnings. * This is also not great practice, but necessary in certain circumstances..
pickle(path)	Pickle sim object to specified file path :Parameters: path -- The file path for pickle
run([env])	Run the composed simulation.

pickle_sub_objs
restore_sub_objs

add(obj: Model | Domain | Scheduler | Job)

Add an approparite object to a Simulation, such as a Model, Domain, Job, or Scheduler

base_hrldas_namelist

base hrldas namelist produced from model and domain

Type:

dict

base_hydro_namelist

base hydro namelist produced from model and domain

Type:

dict

collect(sim_dir=None, output=True)

Collect simulation output after a run

compose(symlink_domain: bool = True, force: bool = False, check_nlst_warn: bool = False)

Compose simulation directories and files :Parameters: * symlink_domain – Symlink the domain files rather than copy

• force – Compose into directory even if not empty. This is considered bad practice but

• is necessary in certain circumstances.

• check_nlst_warn – Allow the namelist checking/validation to only result in warnings.

• This is also not great practice, but necessary in certain circumstances.

domain

A Domain object

Type:

Domain

jobs

a list containing Job objects

Type:

list

model

A Model object

Type:

Model

output

A CompletedSim object returned by the self.collect() method

Type:

CompletedSim

pickle(path: str)

Pickle sim object to specified file path :Parameters: path – The file path for pickle

pickle_sub_objs(obj_list: list = ['model', 'domain', 'output'])

restore_sub_objs(obj_list: list = ['model', 'domain', 'output'])

run(env: dict | None = None)

Run the composed simulation. Returns: 0 for success.

scheduler

A scheduler object to use for each Job in self.jobs

Type:

Scheduler

class wrfhydropy.core.simulation.SimulationOutput

Bases: object

Class containing output objects from a completed Simulation, retrieved using the Simulation.collect() method

Methods

check_output_nans([n_cores])	Check all outputs for NA values
collect_output([sim_dir])	Collect simulation output after a run :Parameters: sim_dir -- The simulation directory to collect

open

channel_rt

Timeseries dataset of CHRTOUT files

Type:

WrfHydroTs

channel_rt_grid

Timeseries dataset of CHRTOUT gridded files

Type:

WrfHydroTs

chanobs

Timeseries dataset of CHANOBS files

Type:

WrfHydroTs

check_output_nans(n_cores: int = 1)

Check all outputs for NA values

collect_output(sim_dir: str | Path | None = None)

Collect simulation output after a run :Parameters: sim_dir – The simulation directory to collect

gwout

Timeseries dataset of GWOUT files

Type:

WrfHydroTs

lakeout

Timeseries dataset of LAKEOUT files

Type:

WrfHydroTs

ldasout

Timeseries dataset of LDASOUT files

Type:

WrfHydroTs

open(name, n_cores=None)

restart_hydro

List of HYDRO_RST WrfHydroStatic objects

Type:

list

restart_lsm

List of RESTART WrfHydroStatic objects

Type:

list

restart_nudging

List of nudgingLastObs WrfHydroStatic objects

Type:

list

rtout

Timeseries dataset of RTOUT files

Type:

WrfHydroTs

Module contents