how to speed up the process of `getvar`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 498
- Forks
- 178
- PR merge metrics
- No merged PRs in 30d
Description
I found that using wrf.getvar to get variables is quite slow.
For example
import time
import wrf
import netCDF4 as nc
wrfout = nc.Dataset(wrfout_file)
t0 = time.time()
SINALPHA_WRF = wrf.getvar(wrfout, 'SINALPHA', timeidx=wrf.ALL_TIMES, squeeze=False)
COSALPHA_WRF = wrf.getvar(wrfout, 'COSALPHA', timeidx=wrf.ALL_TIMES, squeeze=False)
t1 = time.time()
sinalpha = wrfout['SINALPHA'][:]
cosalpha = wrfout['COSALPHA'][:]
t2 = time.time()
print(f'getvar: {t1 - t0},directly: {t2 - t1}')
In my PC, the time of getvar is about 0.084 s, and the time of directly is about 0.015 s.
To be a little more complicated, to get the wind speed and the wind direction using wrf.getvar(wrfout, 'uvmet_wspd_wdir', timeidx=wrf.ALL_TIMES, squeeze=False) would take the time about 14.596 s, while we first get ua, va, SINALPHA, COSALPHA by wrf.getvarand then using the relation between these variables to get the wspd and wdir, whose values are not significant different from the result of getvar, but it only takes 2.263 s.
So, what reason make it seems that the calculation speed of getvar is slow and how to improve it?
In my case, I want to extract some variables from wrfout and backup, e.g, most of the variables may be the shape of (4,50,300,300), and I want to save some diagnostic variables to be (4, 10, 300, 300)(interpolated on certain pressure level or height). Any suggestion to efficiently accomplish this kind of task.
wrf.__version__='1.3.2'
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the timing comparison in the issue with Python, wrf.getvar, netCDF4, and the listed SINALPHA, COSALPHA, and uvmet_wspd_wdir variables. Trace the getvar entry point and compare its handling of ALL_TIMES and derived diagnostics with direct reads; done means identifying the bottleneck and demonstrating a faster approach without changing results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100