devopshq / devopshq/artifactory-cleanup

Enable pagination in order to avoid heavy AQL

Open
#191 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
160
Forks
87
PR merge metrics
No merged PRs in 30d

Description

In some cases, you might want to paginate the AQL results to avoid really heavy load on the Artifactory server.
It doesn't apply to every use case, this needs to be cornered down, I don't even know. In my example, I wanted to remove 5 millions artifacts but couldn't because the AQL request was way too heavy for the server. My filters were simple, I just wanted to remove an entire directory without more regard to other filters.
```yaml
artifactory-cleanup:
server: https://my-artifactory.local/artifactory/
user: $API_USER
password: $API_PASSWORD
policies:
- name: cleanup-png-images
rules:
- rule: Repo
name: "my-generic-repo"
- rule: IncludePath
masks: "my/folder/path/"
```

My dream would have been to be able to add like
```yaml
- rule: PageSize
count: 10000
```
And to see the cleanup to first delete 10 000 artifacts, then the 10 000 next, etc. until deleting less than 10 000 which means the end of the pages.

Pagination exists : https://docs.jfrog.com/artifactory/docs/aql-examples#pagination so something with items.find(...).offset(50).limit(25)
But I'm not too good of a coder to code it and submit a PR.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the code that builds AQL queries and iterates through cleanup results; the issue does not name specific files or tests. Read the linked JFrog AQL pagination documentation and compare offset/limit behavior with deletion. Done means an optional page size can process matching artifacts in batches without imposing pagination on other use cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
57/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.