hpcc-systems / hpcc-systems/HPCC-Platform

[Streaming] Implement a Logic File concept

Open
#554 3 comments 0 reactions 1 assignee View on GitHub

@rengolin is already working on this.

Since Oct 3, 2011.

Docs Impact Feature M:Normal
Dominant language
C++
Stars
616
Forks
311
PR merge metrics
No merged PRs in 30d

Description

Implement a logic file, which contains simple information on how to access separate chunks of the same "file".

When streaming, inputs can create several different chunks when buffering to the same file. These chunks are ordered, but not necessarily consumed in the same order (especially if one consumer fails and has to retry).

So, at least, we need logic to say what is the next available chunk and how many chunks we still have to read at any given time.

Investigate the dali/base/dadfs.cpp and ask Jake in case of doubts. The common area would probably be a good place to put the file in, and in the future, this logic file could end up replacing the current Dali super-file?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.