PyPSA / PyPSA/atlite

Create cutout with SARAH data - Time resolution mismatch

Open
#427 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: bug
Dominant language
Python
Stars
402
Forks
134
PR merge metrics
No merged PRs in 30d

Description

Version Checks (indicate both or one)
  • I have confirmed this bug exists on the lastest release of atlite.

  • I have confirmed this bug exists on the current master branch of atlite.

Issue Description

Hi all,
I am having issues to create atlitle cutouts with the Sarah module. The time resolution (dt) of the .nc files from Sarah is defined as "H", while atlite is looking for cutout.dt in ("30min", "30T", "h", "1h"). I have tried to modify the .nc files by opening each one with xarray and modify the "timefreq" attribute to "1h", so that atlite detects it. When I chech the attributes of the modified .nc files, I see that "timefreq" is set as "1h". However, when trying to do the atlite cutout, time resolution still shows as "H". I am working with instantaneous SIS and SID downloaded data from Sarah for the month of january 2013 and in Europe, and I downloaded and extracted all the files as explained in https://atlite.readthedocs.io/en/latest/examples/create_cutout_SARAH.html. I would appreciate some input on how to fix this bug. Thanks.

Reproducible Example
# I am trying something
import logging

import atlite
import xarray as xr

logging.basicConfig(level=logging.INFO)
import cdsapi
c = cdsapi.Client()

import os
import glob
import xarray as xr

# Original directory containing your SARAH NetCDF files
sarah_dir = r"C:/Users/marta/Desktop/Thesis/Climate-Change-Impacted-Solar-Energy-Generation/atlite examples/sarah2"

# New directory where fixed files will be saved
sarah_dir_fixed = r"C:/Users/marta/Desktop/Thesis/Climate-Change-Impacted-Solar-Energy-Generation/atlite examples/sarah2_fixed"

# Ensure the new directory exists
os.makedirs(sarah_dir_fixed, exist_ok=True)

# Pattern to find all .nc files in the original directory
file_pattern = os.path.join(sarah_dir, "*.nc")

# Loop over each NetCDF file in the directory
for file_path in glob.glob(file_pattern):
    print(f"Processing: {file_path}")
    
    # Open the dataset
    ds = xr.open_dataset(file_path)
    
    # Update or set the 'timefreq' attribute to "1h"
    ds.attrs["timefreq"] = "1h"
    
    # Optional resampling if needed
    # ds = ds.resample(time="1H").nearest()

    # Build a new filename in the 'sarah2_fixed' directory
    # e.g. if file_path = ".../sarah2/file1.nc", 
    # we create ".../sarah2_fixed/file1_fixed.nc"
    base_name = os.path.splitext(os.path.basename(file_path))[0]
    new_file_path = os.path.join(sarah_dir_fixed, base_name + "_fixed.nc")

    # Save the modified dataset
    ds.to_netcdf(new_file_path)
    
    # Close the dataset
    ds.close()
    
    print(f"Saved fixed file to: {new_file_path}")

print("All NetCDF files processed!")

cutout = atlite.Cutout(
    path="europe-2013-01.nc",
    module=["sarah", "era5"],
    sarah_dir="C:/Users/marta/Desktop/Thesis/Climate-Change-Impacted-Solar-Energy-Generation/atlite examples/sarah2_fixed",
    x=slice(-13.6913, 1.7712),
    y=slice(49.9096, 60.8479),
    time="2013-01",
)
print(f"Time resolution (dt): {cutout.dt}")

cutout.prepare()
Expected Behavior

Atlite should be able to prepare the cutout with time resolution of SARAH but it is not.

Installed Versions
atlite-0.4.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the SARAH workflow in create_cutout_SARAH.html and the atlite.Cutout(...)/prepare() entry points shown in the example; trace where the SARAH NetCDF timefreq value is read and compared with cutout.dt. Reproduce the issue with the January 2013 SIS/SID files and verify that the cutout prepares with the intended hourly resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.