kernelci / kernelci/kernelci-pipeline
Docker images cleanup
- Dominant language
- Python
- Stars
- 11
- Forks
- 40
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 14
Description
As part of preparation to versioned API/Pipeline release we need to create mechanism that will delete images that are old and not used anymore.
As first step we might do semi-automatic cleanup of existing images.
For example i took snapshots of images with following format at 7 and 8 February: "imagename,last_update,num_pulls" and identified several stale images that was not updated and dont have any new pulls:
```
2 build-base,2022-08-15T14:46:00.107953Z,3568751
2 build-clang-10,2021-10-11T09:30:23.692938Z,24702
2 build-clang-11,2021-10-11T09:31:52.344053Z,3657
2 build-clang-12,2021-10-11T09:33:21.871013Z,4381
2 build-clang-8,2020-02-07T13:33:10.186608Z,16015
2 build-clang-9,2020-09-11T12:24:23.794136Z,4115
2 build-gcc-6.3.0_mips,2019-01-02T19:22:41.424879Z,74641
2 build-gcc-7_arc,2019-04-26T12:07:11.857286Z,44494
2 build-gcc-7_arm,2019-04-26T12:07:15.222898Z,422144
2 build-gcc-7_arm64,2019-04-26T12:07:18.605918Z,22994
2 build-gcc-7_mips,2019-04-26T12:07:21.970747Z,150251
2 build-gcc-7_riscv64,2019-04-26T12:07:25.41024Z,12862
2 build-gcc-7_x86,2019-04-26T12:07:28.428278Z,26355
2 build-gcc-8_arc,2021-10-11T09:27:48.156805Z,416342
2 build-gcc-8_arm,2021-10-11T09:28:03.844757Z,2739324
2 build-gcc-8_arm64,2021-10-11T09:28:24.603019Z,200543
2 build-gcc-8_mips,2021-10-11T09:28:37.372275Z,1236382
2 build-gcc-8_riscv64,2021-10-11T09:28:49.307957Z,146526
2 build-gcc-8_x86,2021-10-11T09:28:58.174458Z,281651
2 build-k8s,2021-10-11T09:36:37.426002Z,2266357
2 buildroot,2023-10-06T19:01:55.263699Z,1231
2 chromeos-flash,2022-05-23T02:03:33.745155Z,158
2 chromeos-lavapost,2022-05-20T02:06:25.633994Z,338
2 chromeos-tast,2022-05-20T02:06:35.408823Z,888
2 chromiumos,2022-05-20T02:06:42.177093Z,222
2 clang-10,2021-12-13T22:50:40.672498Z,9773
2 clang-11,2024-02-06T11:02:42.852681Z,11597
2 clang-12,2024-02-06T10:21:35.064746Z,111
2 clang-13,2024-02-06T10:24:38.197558Z,10963
2 clang-14,2024-02-06T11:26:30.607968Z,84417
2 clang-15,2024-02-06T11:50:22.579164Z,7006
2 clang-17,2024-02-06T12:40:57.498448Z,4122
2 clang-base,2022-08-15T14:58:02.019422Z,62
2 coccinelle,2022-08-15T14:58:12.506947Z,15
2 cros-clang-11,2023-02-01T15:25:45.894674Z,4
2 cros-clang-14,2023-08-25T07:20:37.6602Z,7785
2 cros-clang-16,2023-02-01T15:26:15.328995Z,4
2 cvehound,2024-02-06T14:28:44.7563Z,29
2 dt-validation,2024-02-06T14:29:59.817239Z,312
2 gcc-10,2024-02-06T13:14:37.265856Z,274361
2 gcc-10_arc,2022-08-15T14:49:03.747169Z,183591
2 gcc-10_arm,2022-08-15T14:49:38.493312Z,499481
2 gcc-10_arm64,2022-08-15T14:50:44.181487Z,63658
2 gcc-10_armv5,2022-08-15T14:51:19.350513Z,1
2 gcc-10_mips,2022-08-15T14:51:34.29928Z,246272
2 gcc-10_riscv64,2022-08-15T14:51:47.765388Z,52263
2 gcc-10_sparc,2022-08-15T14:52:02.525232Z,435
2 gcc-10_x86,2022-08-15T14:52:16.083198Z,72443
2 jenkins-agent,2020-04-06T20:33:56.674786Z,17
2 k8s,2024-02-06T10:15:07.566997Z,3379028
2 kcidb,2020-08-27T12:34:58.106175Z,22
2 kernelci,2024-02-06T10:09:35.067404Z,33583
2 pipeline,2023-09-05T08:17:46.959999Z,56
2 qemu,2024-02-06T14:27:20.079026Z,331743
2 rust,2023-10-06T17:58:23.886998Z,0
2 rustc-1.62,2023-07-17T18:46:47.731383Z,2702
2 rustc-1.66,2023-07-17T19:41:01.815591Z,37
2 rustc-1.68,2023-04-24T09:39:11.238997Z,0
2 rustc-1.70,2023-08-22T20:33:51.302541Z,285
2 rustc-1.71,2023-10-02T11:07:01.41494Z,374
2 rustc-1.72,2023-10-09T16:20:47.071371Z,123
2 rustc-1.73,2023-12-21T22:58:16.359128Z,323
2 rustc-1.74,2024-02-06T14:26:17.171256Z,14
2 staging-build-base,2022-08-04T00:07:23.254115Z,110375
2 staging-build-clang-8,2020-01-22T12:30:00.472868Z,1005
2 staging-build-clang-9,2020-09-15T08:04:42.435954Z,4512
2 staging-build-gcc-6.3.0_mips,2018-11-07T17:36:41.482595Z,30
2 staging-build-gcc-7_arm,2018-11-07T17:12:42.25613Z,24
2 staging-build-gcc-7_arm64,2018-11-07T17:17:28.311113Z,29
2 staging-build-gcc-7_mips,2018-11-07T17:26:30.366179Z,31
2 staging-build-gcc-7_x86,2018-11-07T17:21:50.792404Z,31
2 staging-build-gcc-8_arc,2021-09-17T06:06:21.600348Z,52204
2 staging-build-gcc-8_arm,2021-09-17T06:06:24.010075Z,383214
2 staging-build-gcc-8_arm64,2021-09-17T06:06:26.570485Z,44352
2 staging-build-gcc-8_mips,2021-09-17T06:06:28.90399Z,84833
2 staging-build-gcc-8_riscv64,2021-09-17T06:06:31.258531Z,23326
2 staging-build-gcc-8_x86,2021-09-17T06:06:33.69435Z,44969
2 staging-build-gcc-9_mips,2021-09-16T20:30:58.705415Z,0
2 staging-build-k8s,2021-09-29T08:23:56.018989Z,55560
2 staging-chromiumos,2022-05-20T02:06:49.420492Z,26
2 staging-clang-10,2021-12-02T08:05:25.048825Z,18339
2 staging-clang-11,2023-04-04T16:11:38.071066Z,8490
2 staging-clang-12,2022-08-04T00:10:47.071355Z,567
2 staging-clang-13,2022-08-04T00:11:43.920914Z,5219
2 staging-clang-15,2022-08-18T08:31:08.322704Z,2123
2 staging-clang-16,2023-02-13T08:17:28.268825Z,2423
2 staging-clang-base,2022-08-04T00:08:33.596014Z,436
2 staging-coccinelle,2022-08-18T08:54:25.927787Z,669
2 staging-cros-baseline,2023-05-30T13:23:54.412257Z,1
2 staging-cros-clang-14,2023-05-30T13:16:42.832196Z,4
2 staging-cros-gcc-10,2023-05-30T13:20:18.495393Z,3
2 staging-cros-k8s,2023-05-30T13:12:55.110666Z,1
2 staging-cros-kernelci,2023-05-30T13:10:28.704068Z,3
2 staging-cros-qemu-modules,2023-05-30T13:26:20.895741Z,1
2 staging-cros-sdk,2023-05-30T13:22:42.146207Z,1
2 staging-cros-tast,2023-05-30T13:27:42.403389Z,1
2 staging-deploy,2019-11-08T10:21:31.935004Z,21
2 staging-gcc-10_arc,2022-08-04T00:08:36.152499Z,3701
2 staging-gcc-10_arm,2022-08-04T00:08:39.048897Z,14893
2 staging-gcc-10_arm64,2022-08-04T00:08:41.999232Z,8803
2 staging-gcc-10_armv5,2022-08-04T00:08:44.912545Z,1
2 staging-gcc-10_mips,2022-08-04T00:08:47.37552Z,7370
2 staging-gcc-10_riscv64,2022-08-04T00:08:50.164073Z,3563
2 staging-gcc-10_sparc,2022-08-04T00:08:58.375326Z,6
2 staging-gcc-10_sparc64,2022-08-03T00:06:47.978503Z,58
2 staging-gcc-10_x86,2022-08-04T00:09:01.313404Z,23985
2 staging-rustc-1.62,2023-04-21T11:14:47.483571Z,210
2 staging-rustc-1.66,2023-04-04T16:30:36.98395Z,35
2 staging-rustc-1.68,2023-07-20T16:04:52.460948Z,105
2 staging-rustc-1.70,2023-07-29T16:39:13.415872Z,10
2 staging-rustc-1.71,2023-10-06T11:34:47.375281Z,47
2 staging-rustc-1.72,2023-10-09T09:06:23.303455Z,10
2 staging-rustc-1.73,2023-12-14T16:34:26.261662Z,93
```
Likely i will need to take larger time span, such as week.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the repository entry points responsible for publishing or managing Docker images and determine how image age and pull counts can be collected over a longer period. Define the retention criteria and whether cleanup is semi-automatic or fully automatic. Done means stale, unused images can be identified and removed without affecting images still needed by the versioned API/Pipeline release.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker
- Domain
- devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100