JanusGraph / JanusGraph/janusgraph
LogProcessor is not processing event logs from beginning.
- Dominant language
- Java
- Stars
- 5.8k
- Forks
- 1.2k
- Avg merge
- 13h 53m
- Merged PRs (30d)
- 6
Description
Starting a LogProcessor to process log fom Begining using setStartTime(Instant.EPOCH) doesn't work.
Following is a sample code used.
I have validated that, the event log is created and persisted in HBase Janusgraph tabe.
JanusGraphTransaction tx = graph.buildTransaction().logIdentifier("addedPerson").start();
JanusGraphVertex u = tx.addVertex("human");
u.property("name", "proteros");
u.property("age", 36);
JanusGraphVertex u1 = tx.addVertex("human");
u.property("name", "sandeep");
u.property("age", 34);
tx.commit();
and trying to replay it using below code.
LogProcessorFramework logProcessor1 = JanusGraphFactory.openTransactionLog(graph);
logProcessor1.addLogProcessor("addedPerson").setStartTime(Instant.EPOCH).setProcessorIdentifier("addedPersonProcessor").setStartTime(Instant.EPOCH).addProcessor(new ChangeProcessor() {
@Override
public void process(JanusGraphTransaction janusGraphTransaction, TransactionId transactionId, ChangeState changeState) {
System.out.println("processing old log");
for (Vertex v : changeState.getVertices(Change.ADDED)) {
System.out.println("vertex " + v.label());
if (v.label().equals("human")) totalHumansAdded.incrementAndGet();
}
}
}).build();
However, my logProcessor doesn't process any previously saved vertices and only detect changes which are performed in after the LogProcessor has been built.
I tried setting up Start Time to INSTANT.EPOCH, which as per documentation should make log processing to start from BEGINING but no help.
The only difference is see in logs is the MessagePuller is started from EPOCH.
22:59:19,039 INFO KCVSLog:744 - Loaded unidentified ReadMarker start time 2018-05-01T14:59:19.023Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@57db2b13
22:59:19,054 INFO KCVSLog:748 - Loaded indentified ReadMarker start time 1970-01-01T02:18:20Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@2f953efd
22:59:19,054 INFO KCVSLog:748 - Loaded indentified ReadMarker start time 1970-01-01T02:18:20Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@14bdbc74
22:59:19,054 INFO KCVSLog:748 - Loaded indentified ReadMarker start time 1970-01-01T02:18:20Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@5a7fe64f
22:59:19,054 INFO KCVSLog:748 - Loaded indentified ReadMarker start time 1970-01-01T02:18:20Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@41330d4f
Contributor guide
Research direction
Start with the LogProcessorFramework and JanusGraphFactory.openTransactionLog entry points, then inspect how setStartTime(Instant.EPOCH) reaches the MessagePuller. Reproduce the example with a persisted event log and compare replay from EPOCH with events created after the processor is built. Done means previously saved vertices are processed as expected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100