py4dstem / py4dstem/py4DSTEM

Add Support for RosettaScIIO

Open
#702 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
279
Forks
177
PR merge metrics
No merged PRs in 30d

Description

Is your feature request related to a problem? Please describe.

I was going to spend a couple of days adding support for loading different file types using rosetasciio directly into py4DSTEM. Is there is anything that I might need to consider before doing this.

Describe the solution you'd like

Most of the complications are going to be limiting what filetypes should be allowed to be loaded and which filetypes are not supported. Currently Rosettasciio requires dask as a dependancy, is this an issue? I could potentially remove it as a requirement and only allow for non-lazy loading if this is something that you would prefer not to add. Do you have a current process for handling memory mapped arrays, or is treating them like numpy arrays and then having numpy automatically convert to in memory arrays fine.

Additional context

This should be a good start to understanding how to generalize the process of loading/ transferring data. I think that ultimately we need to have a .yaml file for each file format which details if it outputs 4D Data, and what metadata it has.

Edit:

It seems like there is already a quite nice version here for loading using memmap vs RAM that we could extend.

https://github.com/py4dstem/py4DSTEM/blob/966a41c7611161d4b3d8d082af5a145be0653b76/py4DSTEM/io/importfile.py#L17C5-L23

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading py4DSTEM/io/importfile.py, especially the linked memory-map versus RAM handling. Before implementation, clarify which RosettaSciIO file types are supported, how the dask dependency and memmap behavior should work, and whether per-format YAML metadata is in scope; done means those decisions and the loading interface are defined.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.