Combine ModuleManager and search/kernel path

Open
#2,011 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
20/100
Issue type
Refactor
Clarity
Needs clarification
Activity status
Stale
Tech stack
fortran, python

Research direction

Start by reviewing issue #1991, the ModuleManager, and the existing kernel_path handling for -d and -I. Investigate whether separate search-path lists can share a cache without losing duplicate-module detection. The issue is done only after the duplicate-module policy is agreed and the resulting design is implemented.

Written by the indexing model from the issue text.

Description

In #1991 a ModuleManager is added, which helps with the driver creation by allowing copying all required source files for a driver into the driver source file, creating truly stand-alone drivers without outside dependencies (otherwise, the driver will depend on the kernel, which because of meta-data declaration will always depend on constanst_mod, kernel_mod, argument_mod).

We should investigate if the current path handling (-d/-I) could be combined with the ModuleManager. The module manager could maintain two separate list of search paths - one for -d, one for -I - but at the same time add them to its internal cache. This should make the finding of kernels faster, and avoids code duplication (and frequent accesses to the file system).

The one big problem that I noticed when looking at this:
ATM the kernel_path handling will detect if there is more than one file that defines the same module, and raises an exception. This would be extremely difficult to implement: if there are two files with the same name in different paths, searching would stop the moment the first file is found, and since the information is cached, it would not be detected that there is a duplicate. It might happen later, when a different kernel requires the ModuleManager to look at more paths that a clash is detected. Or we would need to parse all files initially, but that can be a huge overhead (e.g. if we want to be able to find all LFRic files, and we pass the root of the working directory as path, which will add over 700 files for gungho ... each time we start PSyclone :( ).

It seems unlikely that we will ever have a duplicated file (since even the compiler will not complain, though at link time an error might be raised if the same symbol is defined in different .o files ... but if the files are in library, I don't think the linker would even complain). Additionally, imho it should be the task of the build system to detect this, not PSyclone (if I am not mistaken, the dependency analysis that the LFRic build system runs to detect module dependencies, will crash if the same module exists, since it would try to add a duplicated key into the DB), so I would think that this condition would be detected at some stage (not sure if this is before or after PSyclone, and how this might change with FAB).

@arporter , @sergisiso, @rupertford , @TeranIvy - feedback welcome. If we agree to ignore duplicated modules, the implementation is really trivial, but if we agree that we want PSyclone to detect this, the overhead seems not to be worth the effort (and I would close this ticket as invalid).

Dominant language
Python
Stars
137
Forks
36
Avg merge
6d 16h
Merged PRs (30d)
18

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from stfc/PSyclone

All issues in stfc/PSyclone

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.