ApacheCommonsFileTailingMessageProducer remember state when restarting [INT-3743]

Benedikt Linse opened INT-3743 and commented

The FileTailingMessageProducer (both the ApacheCommonsFileTailingMessageProducer and the OSDelegatingFileTailingMessageProducer) currently do not save their state to disk. Therefore the information about the number of lines of the file that were processed is lost when the JVM exits.

A requirement many organizations have is that a tailer can be restarted and upon restart continues tailing the file at the line number that it was interrupted at. Therefore the line numbers must be saved to a db or a disk.

The following information is from Gary's answer to my question on SO and describes how this should be implemented using the MetadataStore:

Spring Integration has an an abstraction MetadataStore - it's a simple key/value abstraction so would be perfect for this use case.

There are several implementations. The PropertiesPersistingMetadataStore persists to a properties file and, by default, only persists on an ApplicationContext close() (destroy()).

It implements Flushable so it can be flush()ed more often.

The other implementations (Redis, MongoDB, Gemfire) don't need flushing because the data is written immediately.

Reference URL: http://stackoverflow.com/questions/30774430/spring-integration-filetailingmessageproducer-remember-current-line-when-restar

1 votes, 2 watchers

Start with ApacheCommonsFileTailingMessageProducer and OSDelegatingFileTailingMessageProducer, then read the MetadataStore abstraction and the PropertiesPersistingMetadataStore behavior described in the issue. Trace where processed line numbers are held and restored, and verify that a restart resumes from the saved position for both producers, including persistence behavior on context close or flush.