Faster iterator for arbitrary order

Open
#469 15 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
rust
Domain
data

Research direction

Review ArrayBase's existing fold, scalar_sum, and visit methods, then compare the proposed iter(), iter_mut(), indexed_iter(), and indexed_iter_mut() APIs with the listed Iterator and Itertools adapters. Done means the naming and arbitrary-order semantics are agreed and the adapters are implemented without requiring separate methods for every consumer.

Written by the indexing model from the issue text.

Description

.iter() provides an iterator over all the elements, but it always iterates in logical order, which may be slow depending on the memory layout of the array. In some cases, however, the order of iteration doesn't matter. Recent issues regarding these types of cases include #466 and #468. Examples of methods where order doesn't matter include the most common uses of these from the Iterator trait

  • .fold()
  • .for_each()
  • .all() and .any()
  • .find()
  • .min(), .max(), .min_by(), .max_by(), .min_by_key(), .max_by_key()
  • .sum(), .product()

and these from Itertools

  • .cartesian_product()
  • .unique(), .unique_by()
  • .combinations()
  • .all_equal()
  • .foreach()
  • .fold_results(), .fold_options(), .fold1(), .tree_fold(), .fold_while()
  • .sorted(), .sorted_by(), .sorted_by_key()
  • .partition_map()
  • .into_group_map()
  • .minmax(), .minmax_by_key(), minmax_by()

We have already implemented some of these "arbitrary order" adapters as individual methods on ArrayBase, including .fold(), .scalar_sum(), and .visit(). However, it doesn't make sense to create separate methods for all of the possible iterator adapters.

As a result, I'd like to add "arbitrary order" .iter(), .iter_mut(), .indexed_iter(), and .indexed_iter_mut() methods designed to iterate in the fastest possible order so that we can hopefully get good performance with iterator adapters.

What does everyone think these "arbitrary order" iterators should be named?

I've thought of .iter_arbitrary() and .iter_unordered(), but those names seem somewhat unclear and unnecessarily verbose.

Dominant language
Rust
Stars
4.3k
Forks
391
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from rust-ndarray/ndarray

All issues in rust-ndarray/ndarray

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.