ash-project / ash-project/ash

Possibility to emit indices in bulk action with `return_stream?: true`

Open
#1,367 9 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Elixir
Stars
2.5k
Forks
422
Avg merge
23h 26m
Merged PRs (30d)
46

Description

**Is your feature request related to a problem? Please describe.**

I'm writing a Mix task, which imports a large file via a bulk insert. To display progress, I would like to be able to log a message every n entries so that I can see the progress. TO do this, I have to enable `return_records?` at the moment. This uses memory unnecessarily.

**Describe the solution you'd like**

A new option `emit_indices?` which will return the current index instead of the whole record. Only available when `return_stream?: true` and `return_records?: false`.

**Describe alternatives you've considered**

None

**Express the feature either with a change to resource syntax, or with a change to the resource interface**

For example

```elixir
rows
|> Ash.bulk_create(Resource, :action, return_stream?: true, emit_indices?: true)
|> Enum.each(fn
{:omitted_record, i} when mod(i, 100) -> IO.puts "imported #{i} records"
_ -> :discard
end)
```

**Additional context**

None

Contributor guide

Open the contributing guide

Research direction

Start at the Ash.bulk_create entry point and trace how return_stream?: true and return_records?: false are handled. The work is done when emit_indices? can produce {:omitted_record, i} for streamed items without returning full records, while remaining unavailable with incompatible options.

Written by the indexing model from the issue text.

Assessment

Tech stack
elixir
Domain
backend-api-design
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.