aboutcode-org / aboutcode-org/scancode.io

Report progress when doing matching in d2d step

Offen
#1,298 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
Python
Sterne
215
Forks
203
Ø Merge
4 T. 8 Std.
Gemergte PRs (30 T.)
6

Beschreibung

When doing a JS and Java D2d on a large codebase, there is either little or no progress provided
```
2024-06-27 03:27:44.41 Step [match_directories_to_purldb] starting
2024-06-27 03:27:45.65 Matching 78,856 directories from to/ in PurlDB
2024-06-27 03:41:29.99 Progress: 10% (7886/78856) ETA: 7419 seconds (2.1 hours)
2024-06-27 04:00:04.20 Progress: 20% (15772/78856) ETA: 7754 seconds (2.2 hours)
2024-06-27 04:10:07.26 Progress: 30% (23657/78856) ETA: 5930 seconds (1.6 hours)
2024-06-27 04:30:06.73 Progress: 40% (31543/78856) ETA: 5612 seconds (1.6 hours)
2024-06-27 04:45:46.57 Progress: 50% (39428/78856) ETA: 4681 seconds (1.3 hours)
2024-06-27 05:03:54.35 Progress: 60% (47314/78856) ETA: 3846 seconds (1.1 hours)
2024-06-27 05:24:31.20 Progress: 70% (55200/78856) ETA: 3002 seconds (50.0 minutes)
2024-06-27 05:45:52.57 Progress: 80% (63085/78856) ETA: 2072 seconds (34.5 minutes)
2024-06-27 06:05:29.40 Progress: 90% (70971/78856) ETA: 1052 seconds (17.5 minutes)
2024-06-27 06:25:34.22 Progress: 100% (78856/78856)
2024-06-27 06:25:34.41 16,889 directories matched in PurlDB
2024-06-27 06:25:34.42 Step [match_directories_to_purldb] completed in 10670 seconds (3.0 hours)
```
for these I would like to see progress displayed at a fixed time interval, rather than based percentage completion.

and also
```
2024-06-27 06:25:34.42 Step [match_resources_to_purldb] starting
2024-06-27 06:25:36.44 Matching 201,361 .map, .js, .mjs, .ts, .d.ts, .jsx, .tsx, .css, .scss, .less, .sass, .soy, .class resources in PurlDB, using SHA1
```

for these I would like to see some progress displayed at a fixed time interval, with percentage completion. Nothing is displayed at all

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.