kedro-org / kedro-org/kedro-plugins

Kedro-dataset release process

Open
#405 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
119
Forks
136
Avg merge
4d 10h
Merged PRs (30d)
5

Description

## Introduction
How are we going to release when certain libraries are not compatible? i.e. if tensorflow has no support for Python3.11, how do we handle this in our CI?

## Background
Since the separation of `kedro-datasets`, it's now possible to upgrade `kedro` / `kedro-datasets` separately. Prior to this, kedro was always compatible will all datasets so we didn't have this challenge before.

## Problem
* How do we make our CI works and allow certain DataSets to skip CI?
* Should the user always install the latest version?
* For example, let's say version `1.0.10` support Python3.10 for Tensorflow and `1.0.11` add support for Python3.11. In theory, if users are using Python<3.11, it would not be a problem if they install `1.0.11`.

## Possible Solution
* We could create some kind of tag/decorators to skip tests in "file" or "module" level to skip tests. It may get a little bit messy w

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by reviewing the repository's CI configuration and the kedro/kedro-datasets compatibility requirements, including the TensorFlow and Python 3.11 example. Done means agreeing on and documenting a release and CI strategy for incompatible datasets, including how users select compatible versions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
ci-cd, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.