QuantConnect / QuantConnect/Documentation

Importing Data: a TradeBar-derived custom type must set DataType = MarketDataType.Base for typed History and Slice.Get

Open Beginner friendly
#2,696 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
HTML
Stars
261
Forks
191
Avg merge
16h 28m
Merged PRs (30d)
56

Description

Summary

The Importing Data pages never say that a custom data type deriving from TradeBar must set DataType = MarketDataType.Base (for example in its constructor). Without that line the point is routed as a plain trade bar, so History<T>(), Slice.Get<T>() and Securities[symbol].Cache.GetData<T>() return nothing for the custom type while the subscription, slice[symbol], the untyped History(new[] { symbol }, ...) and History<TradeBar>() all return the full series. The behaviour is the same locally and in the cloud, and it is a common way to be caught out because the objects the algorithm receives really are instances of the custom type: only the typed accessors ignore them.

What LEAN does

Slice groups each point by DataType, not by runtime type (Common/Data/Slice.cs, CreateDynamicDataDictionary, the switch (datum.DataType) at lines 633 to 657): MarketDataType.Base becomes a custom entry, MarketDataType.TradeBar goes into slice.Bars. Slice.Get<T>() for a type other than TradeBar / QuoteBar / Tick reads only the custom entries, and History<T>() is built on Slice.Get<T>() (Algorithm/QCAlgorithm.History.cs, GetDataTypedHistory). TradeBar's constructors set DataType = MarketDataType.TradeBar, so a derived type that does not override it inherits that routing.

Every TradeBar-derived custom type in the LEAN tree sets it explicitly: Common/Data/Custom/Tiingo/TiingoPrice.cs line 136, Common/Data/Custom/IconicTypes/UnlinkedDataTradeBar.cs line 42, Common/Data/Custom/FxcmVolume.cs line 137, and the Lean.DataSource.* types (CBOE). The CustomDataDerivedFromTradeBarRegressionAlgorithm regression test asserts DataType == MarketDataType.Base for its custom type.

Reproduction

Cloud backtest, LEAN 2.5.0.0.18114, three custom types reading the same daily CSV from the Object Store over 2024-01-02 to 2024-03-01:

Type DataType Subscription slice.Get<T>() History<T>(symbol, 20) untyped History
: TradeBar, constructor sets Base Base 45 points, runtime type preserved 1 15 15
: TradeBar, default TradeBar 45 points, runtime type preserved, lands in slice.Bars 0 0 15 (instances of the custom type)
: BaseData Base 44 points 1 15 15

Proposed change

On 03 Writing Algorithms/16 Importing Data/02 Streaming Data/01 Key Concepts/05 Parse Custom Data.php (or 02 Data Formats.html), add a note along the lines of: "If your custom type derives from TradeBar (or another market data type) instead of BaseData, set DataType = MarketDataType.Base in its constructor. Otherwise the engine treats the points as ordinary trade bars and History<T> / Slice.Get<T> return nothing for your type." A one-line C# and Python example would be enough.

Ref: support conversation 215475999386559.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with 03 Writing Algorithms/16 Importing Data/02 Streaming Data/01 Key Concepts/05 Parse Custom Data.php and compare the related 02 Data Formats.html page if needed. Use Common/Data/Slice.cs and the CustomDataDerivedFromTradeBarRegressionAlgorithm test as references for the documented behavior. Done means the importing-data guidance includes the TradeBar-derived DataType requirement and concise C# and Python examples.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, python
Domain
documentation
Issue type
Documentation
Difficulty
1/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.