microsoft / microsoft/fhir-server
SMART compartment-scoped search with _sort by a search parameter returns empty results (SQL)
- Dominant language
- TSQL
- Stars
- 1.4k
- Forks
- 592
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 41
Description
**Describe the bug**
A SMART-on-FHIR compartment-scoped search (e.g. a patient-scoped token with `patient/*.read`) that sorts by a search parameter value returns an **empty** result set, even though the identical search without `_sort` returns matches. Sorting by `_lastUpdated` works correctly; the failure occurs when sorting by an indexed search parameter (e.g. `_sort=date` on Observation), which takes the sort-value emission path in the SQL query generator.
**FHIR Version?**
R4/R4B (likely all versions; verified on R4)
**Data provider?**
SQL Server
**To Reproduce**
Steps to reproduce the behavior:
1. Configure a SMART patient-scoped request context: scope `patient/*.read`, `AccessControlContext.CompartmentId = `, `CompartmentResourceType = Patient` (equivalent to a patient-launch token).
2. Search `Observation?_tag=` (or any Observation search matching resources in the patient's compartment where the observations have `effectiveDateTime` values) — the in-compartment observations are returned.
3. Add `_sort=date` to the same search.
This is also reproducible with the integration test harness (`SmartSearchTests`-style setup in `test/Microsoft.Health.Fhir.Shared.Tests.Integration/Features/Smart`): the same query returns matches without `_sort` and zero results with `_sort=date`, despite every matched resource having a date value.
**Expected behavior**
The same compartment matches are returned, ordered by the sort parameter (with the documented two-phase handling for resources missing the sort value).
**Actual behavior**
Zero results are returned. No error is raised — the bundle is simply empty, so SMART apps sorting clinical data (e.g. observations by date) silently see no data.
**Additional context**
- The suspected area is the interplay between the SMART compartment UNION CTEs (`SmartCompartmentSearchExpression` → `SqlCompartmentSearchRewriter`) and the sort-value table expression in `SqlQueryGenerator` (`HandleTableKindSort`, `SortExpressionsByQueryLogic`, `_unionAggregateCTEIndex` chaining), and/or the `SortQuerySecondPhase` logic in `SqlServerSearchService`.
- `_sort=-_lastUpdated` with the same compartment context works, so the defect is specific to the sort-value (`isSortValueNeeded`) path.
- Found while adding regression tests for SMART `_include`/`_revinclude` compartment enforcement; it reproduces without any `_include`/`_revinclude` parameters and predates those changes.
Contributor guide
Research direction
Start with SmartCompartmentSearchExpression and SqlCompartmentSearchRewriter, then trace sort-value handling in SqlQueryGenerator, especially HandleTableKindSort, SortExpressionsByQueryLogic, and _unionAggregateCTEIndex chaining. Review SortQuerySecondPhase in SqlServerSearchService and reproduce the issue in the SmartSearchTests-style integration tests. Done means the compartment-scoped search returns the same matches with _sort as without it, in the documented order.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100