MetOffice / MetOffice/fab

Do not use file_walk in PSyclone for finding kernels

Open
#542 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
9
Forks
9
Avg merge
12d 12h
Merged PRs (30d)
2

Description

The PSyclone step analyses potential kernel files (based on the PSyclone kernel directories provided) using:
```
file_lists = [list(file_walk(root, ignore_folders=[config.prebuild_folder])) for root in kernel_roots]
```
The problem with this is (besides performance, since Fab already has a list of all files, so there is no need to query the file system again) that a previous step might replace kernel files (e.g. if a PSyclone transmute step is done before the kernel analysis, there might be two files in the kernel directory, one the original kernel file, one a transmuted one with a slightly modified name).

This then causes failure during the analysis step (two files with the same kernel).

Instead of the above code, this step should take the files from the artefact store (and check for files that are in one of the kernel directories)

Contributor guide

Open the contributing guide

Research direction

Find the PSyclone kernel-analysis code containing the file_walk call and trace how Fab's artefact store exposes its files. Check how kernel directories are identified and how transmuted files are represented, then update the analysis input so it uses artefact-store files and avoids duplicate kernels. Verify the analysis succeeds when a previous step replaces a kernel file.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.