QuantConnect / QuantConnect/Lean

Expand subscription data source to support other web protocols such as gRPC, etc.

Open
#5,697 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature
Dominant language
C#
Stars
21.7k
Forks
5.3k
Avg merge
2d 22h
Merged PRs (30d)
34

Description

Expected Behavior

When adding custom data from the web, we are limited to rest calls. I would expect to be able to define our own stream reader that handles the communication.

Actual Behavior

You can only choose from an enum that creates the respective stream reader later on in the engine.

Potential Solution

Instead of using an enum. Change Subscription data source to an abstract class. The new abstract class will take a Func that returns a stream reader that will be called in the engine to match the current behavior of stream reader initialization in the engine. Derived classes will be created to create cleaner constructors. Derived classes are LocalFileSubscriptionDataSource, RemoteFileSubscriptionDataSource, WebSubscriptionDataSource, and RestSubscriptionDataSource. RestSubscriptionDataSource is derived from WebSubscriptionDataSource. WebSubscriptionDataSource should be used for any additional web protocols a person would want to use.

Checklist
  • I have completely filled out this template
  • I have confirmed that this issue exists on the current master branch
  • I have confirmed that this is not a duplicate issue by searching issues
  • I have provided detailed steps to reproduce the issue

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the subscription data source enum and the engine code that initializes its stream reader. Compare the proposed LocalFileSubscriptionDataSource, RemoteFileSubscriptionDataSource, WebSubscriptionDataSource, and RestSubscriptionDataSource hierarchy, then verify that a caller can provide a custom reader for protocols such as gRPC.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, grpc
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.