Commit b7dc8831 authored by Clément Haëck's avatar Clément Haëck
Browse files

Remove remove_dims processing

Now zones are all in the same file, and with the same thresholds
parent ca1ebc0f
......@@ -11,7 +11,6 @@ For this reason, I convert the distribution of each quantity (SST, CHL, PFT, ...
* Dataset
The corresponding dataset is file:../lib/data/hists.py.
Each file keeps some parameters as dimensions of size 1 (scale, number, zone, threshold, ...)
Note that different zones have different thresholds, so to combine different zones it is useful to remove the threshold dimension when using ~get_data(remove_dims=['threshold'])~, this avoids an array with a row full of np.nan.
The ~hist~ variable stores the number of hits for each bin and each set of parameters.
* Binning
......
......@@ -46,27 +46,7 @@ def ROOT(args):
return lib.data.get_default_directory(args, 'Hists')
auto_attr = lib.data.create_dataset(__name__, PREGEX, ROOT, ARGS, defaults)
def get_data(args=None, remove_dims=None, **kwargs):
import xarray as xr
def preprocess(ds):
for d in remove_dims:
if d in ds.coords and ds.coords[d].size <= 1:
if d in ds.dims:
ds = ds.isel({d: 0})
ds = ds.reset_coords(remove_dims, drop=True)
return ds
prepro_func = None
if remove_dims is not None:
prepro_func = preprocess
finder = auto_attr['get_finder'](args, **kwargs)
return xr.open_mfdataset(finder.get_files(), preprocess=prepro_func,
parallel=False, data_vars='different')
lib.data.create_dataset(__name__, PREGEX, ROOT, ARGS, defaults)
def var_name(name, var):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment