Skip to content

Conversation

@VeckoTheGecko
Copy link
Contributor

@VeckoTheGecko VeckoTheGecko commented Feb 27, 2025

Remove obsolete DeferredNetcdfFileBuffer, _FileBuffer classes and unused attributes.

I also realised while doing this that fieldfilebuffer is really where a lot of the magic happens when it comes to reading in from netcdf files and coercing field input into a unified format parcels can understand. Good for me to know - I think there can be some improvements made here :)

  • Chose the correct base branch (ie, swap_space_and_time_interpolation_order)

🚧 merge after #1892
mypy fixed in next PR

variable is a (str, str) tuple of field name to variable name in the NetCDF file. variable[1] is never a list
@github-project-automation github-project-automation bot moved this from In review to Ready in Parcels v4 release Feb 28, 2025
@github-project-automation github-project-automation bot moved this from Backlog to Ready in Parcels development Feb 28, 2025
@erikvansebille
Copy link
Member

Good for me to know - I think there can be some improvements made here :)

Yes, I think that the entire fieldfilebuffer.py file can be removed at some point as we transition to native xarray. But better to do it in small steps

A next step could be to also remove the support for the indices dictionary to subset the domain? This would also be felt with by xarray/dask in v4?

Base automatically changed from removing_explicit_field_chunking to v4-dev February 28, 2025 07:05
@VeckoTheGecko
Copy link
Contributor Author

as we transition to native xarray. But better to do it in small steps

I think there is quite some code in this file that deals with coercing the data into a unified format. Though I think that this code should be replaced with functions that consume and produce dask arrays (or produce and consume xarray objects without triggering computation).

A next step could be to also remove the support for the indices dictionary to subset the domain? This would also be felt with by xarray/dask in v4?

Yes, I think so. I'm currently working on a few things - but I think this can do with further clearing

@VeckoTheGecko VeckoTheGecko merged commit adb096e into v4-dev Feb 28, 2025
15 of 16 checks passed
@VeckoTheGecko VeckoTheGecko deleted the v/fieldbuffer-cleanup branch February 28, 2025 09:59
@github-project-automation github-project-automation bot moved this from Ready to Done in Parcels v4 release Feb 28, 2025
@github-project-automation github-project-automation bot moved this from Ready to Done in Parcels development Feb 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done
Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants