QuantConnect / QuantConnect/Documentation

Morningstar US Fundamentals: state that SecurityReference status/delisting fields (and classification) are current values, not point-in-time

Open Beginner friendly
#2,670 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
HTML
Stars
261
Forks
191
Avg merge
16h 28m
Merged PRs (30d)
56

Description

Summary

The US Fundamental Data (Morningstar) pages do not state that the SecurityReference status fields are current reference values stamped onto every historical daily snapshot, not point-in-time values. Users qualifying point-in-time universe selection read share_class_status on a historical date and treat D (Deactive) as "delisted as of that date", which silently excludes every later-delisted company at formation time.

What we observed (LEAN 2.5.0.0.18057, cloud backtest, history[Fundamental](symbol, 2005-01-01, 2024-01-01))

ticker rows security_reference.share_class_status security_reference.delisting_date
CELG (delisted 2019-11-21) 3748 (2005-01-04 .. 2019-11-21) D on every row 2019-11-26 on every row, including 2005
TWTR (delisted 2022-10-28) 2260 (2013-11-08 .. 2022-10-29) D on every row 2022-10-28 on every row
AAPL 4781 A on every row none
XOM 4781 A on every row none

The explicit FundamentalService.Get<string>(date, sid, FundamentalProperty.SecurityReference_ShareClassStatus) route returns the same: CELG is D on 2010-06-30, 2015-06-30, 2019-06-28 and 2019-11-20, and null from 2019-11-21 (no snapshot after the last trading day). delisting_reason, company_status, is_primary_share (TWTR: False on every row) and exchange_id behave the same way. This is consistent with the earlier finding that the sector / industry codes are today's assignment backfilled across history (see QuantConnect/Lean#9777's discussion of provenance).

Field definition in Lean: Common/Data/Fundamental/Generated/SecurityReference.cs (ShareClassStatus, Morningstar DataId 1014: "(A) Active, (D) Deactive, (I) Inactive, or (O) Obsolete").

Proposed documentation change

On 03 Writing Algorithms/14 Datasets/04 Morningstar/01 US Fundamental Data (Introduction and/or Data Point Attributes), add a short note along the lines of:

The financial statement and earnings-report values are as-originally-reported per snapshot date. The reference and classification fields (security_reference.share_class_status, delisting_date, delisting_reason, company_status, is_primary_share, exchange_id, and the asset_classification sector/industry codes) are Morningstar's current values and are the same on every historical snapshot. Do not use them to decide historical membership; a company delisted in 2019 reads D with its 2019 delisting date on its 2005 rows. For point-in-time membership use the daily universe itself (history[Fundamentals] / universe_history, which include later-delisted companies) and the Delisting events for cessation dates.

Optionally cross-link this from the Fundamental Universes page, which is where users build formation-date selections.

Open question for the data team

Whether Morningstar supplies effective-dated status / delisting history that could be written into the daily files instead. If so, this becomes a data change rather than a docs note.

Intercom conversation: 215475854522100.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with 03 Writing Algorithms/14 Datasets/04 Morningstar/01 US Fundamental Data and review the Fundamental Universes page for the best cross-link location. Check field definitions in Common/Data/Fundamental/Generated/SecurityReference.cs and the examples in this issue. Done means the documentation clearly distinguishes current reference/classification values from historical financial values and explains how to select point-in-time membership; confirm whether the open data-team question changes this from a docs-only task.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.