QuantConnect / QuantConnect/Documentation
Importing Data: a TradeBar-derived custom type must set DataType = MarketDataType.Base for typed History and Slice.Get
Nobody has claimed this yet.
- Dominant language
- HTML
- Stars
- 261
- Forks
- 191
- Avg merge
- 16h 28m
- Merged PRs (30d)
- 56
Description
Summary
The Importing Data pages never say that a custom data type deriving from TradeBar must set DataType = MarketDataType.Base (for example in its constructor). Without that line the point is routed as a plain trade bar, so History<T>(), Slice.Get<T>() and Securities[symbol].Cache.GetData<T>() return nothing for the custom type while the subscription, slice[symbol], the untyped History(new[] { symbol }, ...) and History<TradeBar>() all return the full series. The behaviour is the same locally and in the cloud, and it is a common way to be caught out because the objects the algorithm receives really are instances of the custom type: only the typed accessors ignore them.
What LEAN does
Slice groups each point by DataType, not by runtime type (Common/Data/Slice.cs, CreateDynamicDataDictionary, the switch (datum.DataType) at lines 633 to 657): MarketDataType.Base becomes a custom entry, MarketDataType.TradeBar goes into slice.Bars. Slice.Get<T>() for a type other than TradeBar / QuoteBar / Tick reads only the custom entries, and History<T>() is built on Slice.Get<T>() (Algorithm/QCAlgorithm.History.cs, GetDataTypedHistory). TradeBar's constructors set DataType = MarketDataType.TradeBar, so a derived type that does not override it inherits that routing.
Every TradeBar-derived custom type in the LEAN tree sets it explicitly: Common/Data/Custom/Tiingo/TiingoPrice.cs line 136, Common/Data/Custom/IconicTypes/UnlinkedDataTradeBar.cs line 42, Common/Data/Custom/FxcmVolume.cs line 137, and the Lean.DataSource.* types (CBOE). The CustomDataDerivedFromTradeBarRegressionAlgorithm regression test asserts DataType == MarketDataType.Base for its custom type.
Reproduction
Cloud backtest, LEAN 2.5.0.0.18114, three custom types reading the same daily CSV from the Object Store over 2024-01-02 to 2024-03-01:
| Type | DataType |
Subscription | slice.Get<T>() |
History<T>(symbol, 20) |
untyped History |
|---|---|---|---|---|---|
: TradeBar, constructor sets Base |
Base | 45 points, runtime type preserved | 1 | 15 | 15 |
: TradeBar, default |
TradeBar | 45 points, runtime type preserved, lands in slice.Bars |
0 | 0 | 15 (instances of the custom type) |
: BaseData |
Base | 44 points | 1 | 15 | 15 |
Proposed change
On 03 Writing Algorithms/16 Importing Data/02 Streaming Data/01 Key Concepts/05 Parse Custom Data.php (or 02 Data Formats.html), add a note along the lines of: "If your custom type derives from TradeBar (or another market data type) instead of BaseData, set DataType = MarketDataType.Base in its constructor. Otherwise the engine treats the points as ordinary trade bars and History<T> / Slice.Get<T> return nothing for your type." A one-line C# and Python example would be enough.
Ref: support conversation 215475999386559.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with 03 Writing Algorithms/16 Importing Data/02 Streaming Data/01 Key Concepts/05 Parse Custom Data.php and compare the related 02 Data Formats.html page if needed. Use Common/Data/Slice.cs and the CustomDataDerivedFromTradeBarRegressionAlgorithm test as references for the documented behavior. Done means the importing-data guidance includes the TradeBar-derived DataType requirement and concise C# and Python examples.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100