microsoft / microsoft/fhir-server

SMART compartment-scoped search with _sort by a search parameter returns empty results (SQL)

Open
#5,672 1 comment 0 reactions 0 assignees View on GitHub
Area-Search Area-SMART Bug
Dominant language
TSQL
Stars
1.4k
Forks
592
Avg merge
2d 7h
Merged PRs (30d)
41

Description

**Describe the bug**
A SMART-on-FHIR compartment-scoped search (e.g. a patient-scoped token with `patient/*.read`) that sorts by a search parameter value returns an **empty** result set, even though the identical search without `_sort` returns matches. Sorting by `_lastUpdated` works correctly; the failure occurs when sorting by an indexed search parameter (e.g. `_sort=date` on Observation), which takes the sort-value emission path in the SQL query generator.

**FHIR Version?**
R4/R4B (likely all versions; verified on R4)

**Data provider?**
SQL Server

**To Reproduce**
Steps to reproduce the behavior:
1. Configure a SMART patient-scoped request context: scope `patient/*.read`, `AccessControlContext.CompartmentId = `, `CompartmentResourceType = Patient` (equivalent to a patient-launch token).
2. Search `Observation?_tag=` (or any Observation search matching resources in the patient's compartment where the observations have `effectiveDateTime` values) — the in-compartment observations are returned.
3. Add `_sort=date` to the same search.

This is also reproducible with the integration test harness (`SmartSearchTests`-style setup in `test/Microsoft.Health.Fhir.Shared.Tests.Integration/Features/Smart`): the same query returns matches without `_sort` and zero results with `_sort=date`, despite every matched resource having a date value.

**Expected behavior**
The same compartment matches are returned, ordered by the sort parameter (with the documented two-phase handling for resources missing the sort value).

**Actual behavior**
Zero results are returned. No error is raised — the bundle is simply empty, so SMART apps sorting clinical data (e.g. observations by date) silently see no data.

**Additional context**
- The suspected area is the interplay between the SMART compartment UNION CTEs (`SmartCompartmentSearchExpression` → `SqlCompartmentSearchRewriter`) and the sort-value table expression in `SqlQueryGenerator` (`HandleTableKindSort`, `SortExpressionsByQueryLogic`, `_unionAggregateCTEIndex` chaining), and/or the `SortQuerySecondPhase` logic in `SqlServerSearchService`.
- `_sort=-_lastUpdated` with the same compartment context works, so the defect is specific to the sort-value (`isSortValueNeeded`) path.
- Found while adding regression tests for SMART `_include`/`_revinclude` compartment enforcement; it reproduces without any `_include`/`_revinclude` parameters and predates those changes.

Contributor guide

Open the contributing guide

Research direction

Start with SmartCompartmentSearchExpression and SqlCompartmentSearchRewriter, then trace sort-value handling in SqlQueryGenerator, especially HandleTableKindSort, SortExpressionsByQueryLogic, and _unionAggregateCTEIndex chaining. Review SortQuerySecondPhase in SqlServerSearchService and reproduce the issue in the SmartSearchTests-style integration tests. Done means the compartment-scoped search returns the same matches with _sort as without it, in the documented order.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.