turicas / turicas/rows

Dot not assume Table._rows will always be in memory

Open
#45 1 comment 0 reactions 1 assignee View on GitHub

@turicas is already working on this.

Since May 18, 2017.

enhancement plugin question rows.table
Dominant language
Python
Stars
886
Forks
137
No merged PRs in 30d

Description

Currently some operations assume all rows are in memory (such as order_by). We may move all the code to something lazy.
For order_by specifically we could sort in-disk instead of in-memory, like csvsort does.

  • Implement LazyTable class
  • Add lazy parameter to rows.plugins.utils.create_table
  • Change all plugins to force (or not) lazyness (example: in HTML
    lazy=False, in CSV lazy=True by default)
  • Change default sample size to 1000 (or any other non-arbitrary number)
  • Create documentation about it

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.