blaze / blaze/odo

"Simulation mode" to show conversion paths in odo

Open
#265 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1k
Forks
131
PR merge metrics
No merged PRs in 30d

Description

When writing new backends and debugging them, it would be useful if there was an option I could specify to the odo function that would "simulate" a conversion by printing out (or returning in a list) the function path it would take from source to destination without actually doing the conversion.

This could help discover issues without having to attempt to move data, and also shed light on potentially awkward paths odo may be taking to convert through the networkx graph to the target data type, allowing for more direct shortcuts to be taken. If nothing else, it could throw the curtain back on some of the magic going on behind the scenes and make the process more understandable.

I've wished this existed enough lately that I very well may just implement it, but welcome any thoughts on how it should be structured, how it should deal with append vs. convert (i.e. MultipleDispatch vs. networkx and cases where both are used), if the output should be printed/visualized/returned in a Python structure, etc.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the odo function and trace how conversion paths are selected through the networkx graph, including the MultipleDispatch behavior for append versus convert. Determine whether simulation should print or return a Python structure and how mixed dispatch and graph paths are represented. Done means the issue's proposed simulation behavior is specified and implemented with coverage for the relevant path-selection cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.