spring-projects / spring-projects/spring-batch

Sequential Aggregate Reader

Open
#4,777 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-triage type: feature
Dominant language
Java
Stars
3k
Forks
2.5k
Avg merge
6d 53m
Merged PRs (30d)
3

Description

I have written once a reader that I want to share (I can create a PR if you like it):

The idea of this reader is to read an ordered set of data and to group items based on a condition. Exemple :

Given :

fk data
1 data-1
1 data-2
2 data-1
2 data-2
3 data-1

Based on the fk column, the reader must give me :

Aggregate : (1, {data-1, data-2})
Aggregate : (2, {data-1, data-2})
Aggregate : (3, {data-1})

Implementation (Not thread safe) :

//Implemented by the model to tell us how to aggregate, which line match the next one
public interface AggregateMatcher<T> {
    boolean match(T t);
}
//The aggregate model
public class Aggregate<T> implements Iterable<T> {

    private final List<T> items = new ArrayList<>();

    public static <E> Aggregate<E> of(E item) {
        return new Aggregate<>(item);
    }

    @SafeVarargs
    public static <E> Aggregate<E> of(E... item) {
        Aggregate<E> aggregate = new Aggregate<>();
        Arrays.stream(item).forEach(aggregate::add);
        return aggregate;
    }
    
    private Aggregate() {
    }

    private Aggregate(T first) {
        add(first);
    }

    public Aggregate<T> add(T item) {
        items.add(item);
        return this;
    }

    public boolean isEmpty() {
        return items.isEmpty();
    }

    @Override
    public Iterator<T> iterator() {
        return items.iterator();
    }

    public Stream<T> stream() {
        return items.stream();
    }

    public int size() {
        return items.size();
    }
}
public class AggregateReader<T extends AggregateMatcher<T>> implements ItemReader<Aggregate<T>>, ItemStream {

    private final PeekableItemReader<T> delegate;

    public AggregateReader(PeekableItemReader<T> delegate) {
        this.delegate = delegate;
    }

    @Nullable
    @Override
    public Aggregate<T> read() throws Exception {

        T item = delegate.read();
        if (item == null) {
            return null;
        }

        Aggregate<T> aggregate = Aggregate.of(item);
        T candidate;
        while ((candidate = delegate.peek()) != null && candidate.match(item)) {
            aggregate.add(delegate.read());
        }

        return aggregate;
    }

    @Override
    public void open(ExecutionContext executionContext) throws ItemStreamException {
        if (delegate instanceof ItemStream) {
            ((ItemStream) delegate).open(executionContext);
        }
    }

    @Override
    public void update(ExecutionContext executionContext) throws ItemStreamException {
        if (delegate instanceof ItemStream) {
            ((ItemStream) delegate).update(executionContext);
        }
    }

    @Override
    public void close() throws ItemStreamException {
        if (delegate instanceof ItemStream) {
            ((ItemStream) delegate).close();
        }
    }
}

And finally the test class:

class AggregateReaderTest {

    @Test
    void should_read_multiple_aggregates_when_multiple_ides_found() throws Exception {

        AggregateReader<SomeData> reader = aggregateReader(List.of(
                SomeData.of(1, "1"),
                SomeData.of(1, "2"),
                SomeData.of(2, "1"),
                SomeData.of(2, "2"),
                SomeData.of(2, "3"),
                SomeData.of(2, "4"),
                SomeData.of(3, "1"),
                SomeData.of(3, "2")
        ));

        assertThat(reader.read())
                .containsExactly(
                        SomeData.of(1, "1"),
                        SomeData.of(1, "2")
                );
        assertThat(reader.read())
                .containsExactly(
                        SomeData.of(2, "1"),
                        SomeData.of(2, "2"),
                        SomeData.of(2, "3"),
                        SomeData.of(2, "4")
                );
        assertThat(reader.read())
                .containsExactly(
                        SomeData.of(3, "1"),
                        SomeData.of(3, "2")
                );

        assertThat(reader.read()).isNull();
    }

    @Test
    void should_read_one_aggregate_of_one_item_when_data_size_isOne() throws Exception {

        AggregateReader<SomeData> reader = aggregateReader(List.of(SomeData.of(1, "1")));
        assertThat(reader.read())
                .containsExactly(SomeData.of(1, "1"));
        assertThat(reader.read()).isNull();
    }

    @Test
    void should_read_one_aggregate_when_same_id_on_all_data() throws Exception {

        AggregateReader<SomeData> reader = aggregateReader(List.of(SomeData.of(1, "1"), SomeData.of(1, "2")));
        assertThat(reader.read())
                .containsExactly(SomeData.of(1, "1"), SomeData.of(1, "2"));
        assertThat(reader.read()).isNull();
    }

    @Test
    void should_read_aggregate_of_one_item_when_not_same_id_provided() throws Exception {

        AggregateReader<SomeData> reader = aggregateReader(List.of(SomeData.of(1, "1"), SomeData.of(2, "2"), SomeData.of(3, "3")));
        assertThat(reader.read())
                .containsExactly(SomeData.of(1, "1"));
        assertThat(reader.read())
                .containsExactly(SomeData.of(2, "2"));
        assertThat(reader.read())
                .containsExactly(SomeData.of(3, "3"));
        assertThat(reader.read()).isNull();
    }

    @Test
    void should_read_null_when_empty_data() throws Exception {

        AggregateReader<SomeData> reader = aggregateReader(List.of());
        assertThat(reader.read()).isNull();
    }

    private static AggregateReader<SomeData> aggregateReader(List<SomeData> someData) {
        SingleItemPeekableItemReader<SomeData> peekableItemReader = new SingleItemPeekableItemReader<>();
        peekableItemReader.setDelegate(new ListItemReader<>(someData));
        return new AggregateReader<>(peekableItemReader);
    }

    @Value(staticConstructor = "of")
    private static class SomeData implements AggregateMatcher<SomeData> {
        int id;
        String data;

        @Override
        public boolean match(SomeData someData) {
            return id == someData.id;
        }
    }
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the proposed AggregateMatcher, Aggregate, AggregateReader, and AggregateReaderTest entry points in the issue, along with the ItemReader and ItemStream contracts they use. Compare the behavior against existing Spring Batch reader patterns and determine whether the proposed API belongs in the project. Done means an agreed design, integrated implementation, and passing coverage for the listed aggregate cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.