Skip to content

Commit

Permalink
Use custom tokenization function for NetCDFDataProxy objects
Browse files Browse the repository at this point in the history
  • Loading branch information
bouweandela committed Nov 15, 2024
1 parent 5bd3351 commit db0b8b3
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions lib/iris/fileformats/netcdf/_thread_safe_nc.py
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,12 @@ def ndim(self):
def dask_meta(self):
return np.ma.array(np.empty((0,) * self.ndim, dtype=self.dtype), mask=True)

def __dask_tokenize__(self):
# Dask uses this function to uniquely identify the "array".
# A custom function is slightly faster than general object tokenization,
# which improves the speed of loading small NetCDF files.
return f"<{self.__class__.__name__} path={self.path!r} variable_name={self.variable_name!r}>"

def __getitem__(self, keys):
# Using a DatasetWrapper causes problems with invalid ID's and the
# netCDF4 library, presumably because __getitem__ gets called so many
Expand Down

0 comments on commit db0b8b3

Please sign in to comment.