Azure / Azure/azureml-examples

How to Read a DataFrameDirectory in a custom component

Open
#1,240 0 comments 0 reactions 0 assignees View on GitHub
example request pipeline
Dominant language
Jupyter Notebook
Stars
2k
Forks
1.7k
Avg merge
18h 18m
Merged PRs (30d)
2

Description

I'm trying to combine existing designer components (e.g: Remove Duplicate Rows) with a custom component. The output of the Remove Duplicate Rows (RDR) component goes to the custom component. The designer tells me the type of the RDR output is a DataFrameDirectory:

![image](https://user-images.githubusercontent.com/13122883/166712055-ef6eddc0-8cfc-407a-b821-f74b97bd111e.png)

In my custom component i can see these files are passed:

![image](https://user-images.githubusercontent.com/13122883/166713101-c9d3d51a-3e32-4761-9337-7f9382785865.png)

I tried several things like: Dataset.Tabular.from_parquet(path) but this does not work.

My request:
Since all existing designer data transformation components ouput a DataFrameDirectory. I think it will be great if there is a example how to use this output type as input in a custom component.

Contributor guide

Open the contributing guide

Research direction

No file or test is named. Start by reviewing the existing designer data-transformation and custom-component examples, then trace the DataFrameDirectory output from Remove Duplicate Rows into a custom component. Done means adding an example that shows how to consume this output type as custom-component input.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.