dotnet / dotnet/arcade-services

darc delete-* can be very slow

Open
#6,211 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
86
Forks
86
Avg merge
1d 8h
Merged PRs (30d)
35

Description

When mass deleting subscriptions, or generally making some kind of mass update, the maestro config manager is far slower than it should be. It appears to iteratively make each individual each to a file for each item that was added or deleted, rather than batching them. For an example, see below. This leads to very long running commands. Deleting all preview3 targeting subscriptions took ~15mins and made numerous commits.

The commands need to be much smarter about how they use AzDO other APIs.

```
info: Committing 1 files to https://dev.azure.com/dnceng/internal/_git/maestro-configuration branch darc/production-701bdf53 via API
info: Getting the contents of file 'configuration/subscriptions/11.0.1xx-preview3.yml' from repo 'dnceng/internal/maestro-configuration' in branch/commit 'darc/production-701bdf53'...
info: Successfully committed 1 files to https://dev.azure.com/dnceng/internal/_git/maestro-configuration branch darc/production-701bdf53
info: An active pull request already exists at https://dev.azure.com/dnceng/internal/_git/maestro-configuration/pullrequest/60350, new changes were pushed to it
info: The changes will take effect in Maestro shortly after the pull request is merged.
info: No configuration file path provided. Trying default location first...
info: Getting the contents of file 'configuration\subscriptions\dotnet-sourcelink.yml' from repo 'dnceng/internal/maestro-configuration' in branch/commit 'darc/production-701bdf53'...
info: Couldn't find configuration object at the default location. Searching all files in folder... this might take a few minutes
info: Getting the contents of file 'configuration/subscriptions/10.0.1xx-release.yml' from repo 'dnceng/internal/maestro-configuration' in branch/commit 'darc/production-701bdf53'...
info: Getting the contents of file 'configuration/subscriptions/11.0.1xx-preview2.yml' from repo 'dnceng/internal/maestro-configuration' in branch/commit 'darc/production-701bdf53'...
info: Getting the contents of file 'configuration/subscriptions/11.0.1xx-preview3.yml' from repo 'dnceng/internal/maestro-configuration' in branch/commit 'darc/production-701bdf53'...
info: Object with key ea3aed50-92ba-422d-a9d2-556a907f42c7 found in configuration/subscriptions/11.0.1xx-preview3.yml
info: Committing 1 files to https://dev.azure.com/dnceng/internal/_git/maestro-configuration branch darc/production-701bdf53 via API
info: Getting the contents of file 'configuration/subscriptions/11.0.1xx-preview3.yml' from repo 'dnceng/internal/maestro-configuration' in branch/commit 'darc/production-701bdf53'...
info: Successfully committed 1 files to https://dev.azure.com/dnceng/internal/_git/maestro-configuration branch darc/production-701bdf53
info: An active pull request already exists at https://dev.azure.com/dnceng/internal/_git/maestro-configuration/pullrequest/60350, new changes were pushed to it
info: The changes will take effect in Maestro shortly after the pull request is merged.
info: No configuration file path provided. Trying default location first...
info: Getting the contents of file 'configuration\subscriptions\dotnet-dotnet.yml' from repo 'dnceng/internal/maestro-configuration' in branch/commit 'darc/production-701bdf53'...
info: Couldn't find configuration object at the default location. Searching all files in folder... this might take a few minutes
info: Getting the contents of file 'configuration/subscriptions/10.0.1xx-release.yml' from repo 'dnceng/internal/maestro-configuration' in branch/commit 'darc/production-701bdf53'...
info: Getting the contents of file 'configuration/subscriptions/11.0.1xx-preview2.yml' from repo 'dnceng/internal/maestro-configuration' in branch/commit 'darc/production-701bdf53'...
info: Getting the contents of file 'configuration/subscriptions/11.0.1xx-preview3.yml' from repo 'dnceng/internal/maestro-configuration' in branch/commit 'darc/production-701bdf53'...
info: Object with key 41d8039b-38e3-4eff-ad2c-55be713da409 found in configuration/subscriptions/11.0.1xx-preview3.yml
info: Committing 1 files to https://dev.azure.com/dnceng/internal/_git/maestro-configuration branch darc/production-701bdf53 via AP
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.