openclimatefix / openclimatefix/data-platform

Discuss: Store price data

Open
#182 13 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
3
Forks
1
Avg merge
44m
Merged PRs (30d)
2

Description

Verification
  • I have searched the existing issues to ensure this feature has not already been requested.
Provide a detailed description of the proposed feature.

Store price data in the data-platform

What is the motivation for the feature?

We currently collect DA price data from Entsoe everytime the NL Consumer and Forecast runs. This seems slightly over kill, and we should only collect this one.

Its possible to believe that we might need to store more of this data in the future

Outline a potential approach or solution (optional).
  • New schema called prices
  • one table thats stores the prices, columns could be, location_uuid, timestamp_utc, value, market (note market could be in a separate table to save space rather than reparting 'NL_DA')
  • write function
  • read function

Note it is quite similar to an observation (just not MW, its £/MW), so it might be possible to squeeze it into there

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining how observations are stored and how DA price data is collected for the NL Consumer and Forecast runs. Define the prices schema and decide whether it belongs in the observation model, then add the write and read paths. Done means price data is collected once and can be stored and retrieved for the proposed location, timestamp, value, and market fields.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.