Data Frame Library Support needed to integrate in Pythons Data Science Ecosystem.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2
- Forks
- 0
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 6
Description
The current API is low level resulting in a high learning curve and manual work in order to leverage commonly used data science libraries in python like Pandas, Polars etc. I have not come across a python customer that did not use data frame libraries for post processing, data analysis and automating report generation.
Some of the pain points:
- It requires lots of iterating over objects, no table / column level operations.
- Hard to merge / join different query results (tables).
- Need for using separate clients (Data vs MetaData)
- Need for separate calls to read measurement data.
- String based queries
To better enable customer workflow's and reduce the learning curve our API needs options to return data frames. Using something like pyArrow would allow customers to choose their data frame library of choice.
Data Frames support will significantly reduce effort needed for adoption of MDS and integration with existing workflows at customers.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by mapping the current Python API's data and metadata clients, measurement reads, and string-based queries, then assess a PyArrow-based return path for Pandas and Polars. Done should include an agreed scope and acceptance criteria for data frame support.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- api, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100