spring-projects / spring-projects/spring-batch

Make FlatFileItemReader and FlatFileItemWriter and its builders more reusable/abstract.

Open
#4,254 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

status: waiting-for-reporter type: feature
Dominant language
Java
Stars
3k
Forks
2.5k
Avg merge
6d 53m
Merged PRs (30d)
3

Description

Hi,

I was wondering if there's a way to make FlatFileItemReader and FlatFileItemWriter and its builder implementations more abstract or reusable.

I was creating a SftpFlatFileItemReader and Writer for reading and writing direclty to the SFTP Server, and I encountered that the FlatFileItemWriter implementation is really attached to the definition of 'File' as in a FileSystem file.

On the other hand, the SftpFlatFileItemReader implementation was way easier since it accepts an InputStream as a Resource, and the only thing I had to override was the open and close methods, to manage the SFTP Server session.

But like I said, there's no way to write directly to a SFTP Server file with the current FlatFileItemWriter implementation with only the modifications I did for the SftpFlatFileItemReader.

Also, in order to create a SftpFlatFileItemReaderBuilder and WriterBuilder, I had to copy-paste many of the code that really has the same functionality in both environments, local FileSystem and SFTP, and of course, this code could change in Spring, and it's another level of complexity for us, since we would have to be "leveled" with this implementation as time goes and the classes changes.

So the issue splits in two:

  1. Is there a way to make FlatFileItemWriter accept a Resource that is not a local File, but an abstract 'Resource' as in the FlatFileItemReader, which is more "abstract"?
  2. Would it be possible to abstract the builders so there's no need to re-implement all the methods by copy-pasting?

Sorry for the inconvenience and thanks in advance 😄

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the FlatFileItemReader and FlatFileItemWriter implementations alongside their builder implementations, then compare the existing SftpFlatFileItemReader approach described in the issue. Determine whether the reader and writer can share reusable Resource-based behavior and builders without copy-pasted functionality; done means the requested SFTP use case can be supported by the shared abstractions.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spring
Domain
backend
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.