Azure / Azure/azure-sdk-for-python

[Cosmos] Remove ephemeral-tenant provisioning from fixed-account live-test lanes

Open
#48,780 0 comments 1 reaction 0 assignees View on GitHub
Client Cosmos
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

**Is your feature request related to a problem? Please describe.**

PR #48459 moves the Python Cosmos key-auth live tests onto fixed, team-owned accounts and proves that the selected accounts work across the live matrix. However, the shared Python live-test stage still deploys and removes a temporary Cosmos account for every matrix leg before the fixed credentials are reapplied.

As a result, the migrated key-auth tests still authenticate through the ephemeral live-test tenant during ARM deployment. A tenant rotation or broken service connection can therefore fail the job before pytest reaches the fixed account. This is the final gap between the current Python implementation and Java's no-provisioning fixed-account model.

**Describe the solution you'd like**

Separate the live matrix by resource-lifecycle requirements and make the fixed-account lane independent of ephemeral-tenant provisioning.

- [ ] Split the 12 fixed key-auth jobs and the already-fixed GSI job from the 7 AAD jobs that require a tenant-scoped Cosmos data-plane role assignment.
- [ ] Run the fixed-account jobs through a Cosmos-owned stage or job that resolves credentials from the shared versioned JSON secret but does not invoke `build-test-resource-config.yml`, `deploy-test-resources.yml`, or `remove-test-resources.yml`.
- [ ] Ensure the fixed-account jobs do not require the `azure-sdk-tests-cosmos` service connection or any authentication against the ephemeral tenant.
- [ ] Keep the AAD jobs on the existing provisioned-resource path so `test-resources.bicep` can create their tenant-specific `sqlRoleAssignment`.
- [ ] Move values currently supplied only as Bicep outputs into the fixed-account matrix. In particular, set `AZURE_COSMOS_ENABLE_CIRCUIT_BREAKER=True` on the applicable circuit-breaker jobs.
- [ ] Remove the transitional post-deployment credential reapplication once fixed-account jobs no longer deploy resources; the resolver pre-step should become their sole source of `ACCOUNT_HOST` and `ACCOUNT_KEY`.
- [ ] Add best-effort cleanup for databases left by cancelled or aborted runs. Cleanup must be limited to stale `PythonSDKTest-*` databases owned by Python and must run across each fixed account without affecting Java or concurrent Python runs.
- [ ] Document ownership for Java's shared account provisioning/reconciliation script and the shared JSON secret update procedure, including a smoke test after account or key rotation.
- [ ] Validate overlapping fixed-account runs and prove they remain runnable when ephemeral-tenant deployment credentials are unavailable.

**Describe alternatives you've considered**

1. Keep the current transitional deployment followed by credential override. This is already proven by PR #48459, but it does not remove the tenant dependency.
2. Move AAD tests onto the fixed accounts and refresh their role assignments after every tenant rotation. This preserves one matrix but requires cross-subscription permissions and additional identity lifecycle work.
3. Add a per-leg deployment condition to shared `eng` templates. This is generic, but changes monorepo-wide infrastructure for a Cosmos-specific requirement.
4. Register a separate AAD pipeline while keeping the fixed-account pipeline Cosmos-owned. This is fully service-local but adds another Azure DevOps pipeline definition to maintain.

**Additional context**

PR #48459 already supplies the prerequisites for this work:

- A cross-platform PowerShell resolver compatible with Java's shared JSON-secret schema.
- Stable-account selectors for all 12 key-auth matrix jobs; GSI already uses dedicated fixed credentials.
- Run-scoped `PythonSDKTest-*` database naming, bounded control-plane retries, scoped normal cleanup, and test isolation needed for concurrent use.
- Early resolver validation through `PreSteps` and a Cosmos-local post-deployment hook used during the transitional rollout.
- Successful live verification of the fixed-account query, split, multi-region, and circuit-breaker coverage.

Related tracking issue: #48236. Java reference: Azure/azure-sdk-for-java#49735.

Contributor guide

Open the contributing guide

Research direction

Start with PR #48459 and the shared live-test stage, then trace build-test-resource-config.yml, deploy-test-resources.yml, remove-test-resources.yml, and test-resources.bicep. Separate the fixed-account and AAD paths so fixed jobs use the resolver and shared JSON secret without ephemeral provisioning, while AAD retains tenant-scoped role assignment. Done includes scoped stale-database cleanup, ownership documentation, and validation of overlapping runs and unavailable deployment credentials.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, powershell, python
Domain
cloud, databases, devops
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.