pydata / pydata/xarray

Dataset.from_records()?

Open
#2,824 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

topic-documentation
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

Currently, to easily create a Dataset from an existing numpy recarray (not a DataArray, which is currently bugged anyway with recarrays due to #1434), I couldn't find an easier way than

df = xr.Dataset.from_dataframe(pd.DataFrame(my_recarray).set_index('foo'))

(which is kind of dumb since it allocates the memory twice)

It would definitely be nice to be able to do just this (perhaps with extra arguments to set index on the fly etc):

df = xr.Dataset.from_records(my_recarray, ...)

(Apologies if I'm missing something obvious.)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing Dataset.from_dataframe path and how Dataset construction handles NumPy recarrays. Compare the proposed Dataset.from_records API with the current pandas-based workaround, including possible index arguments. Done should mean a recarray can create a Dataset without the intermediate DataFrame allocation, with the supported behavior and arguments defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.