Azure / Azure/azureml-examples

entry_relative_path issues when specifying the directory of a script.

Open
#3,027 0 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Jupyter Notebook
Stars
2k
Forks
1.7k
Avg merge
18h 18m
Merged PRs (30d)
2

Description

### Describe your suggestion

while utilizing the command_component from sdk v2 in order to build a pipeline. I wanted to write a wrapper around the command_component in order to automate some of the repeated values that we will be using. However this then forces the _resolve_source_file to be targeting the directory where the utility function is instead of the source directory.

```python
def _resolve_source_file():
"""Resolve source file as last customer frame's module file position."""
try:
frame_list = inspect.stack()
# We find the last frame which is in SDK code instead of customer code or dependencies code
# by checking whether the package name of the frame belongs to azure.ai.ml.component.
pattern = r"(^mldesigner(?=\..*|$).*)"
for frame, last_frame in zip(frame_list, frame_list[1:]):
if _assert_frame_package_name(pattern, frame.frame) and not _assert_frame_package_name(
pattern, last_frame.frame
):
module = inspect.getmodule(last_frame.frame)
return Path(module.__file__).absolute() if module else None
# pylint: disable=broad-except
except Exception:
return None
```

The workaround is to abuse the regex check in _resolve_source_file by making the utility module that overwrites the functions currently in the sdk for command component and add the modules we would like to exclude as well as passing code variable into the _resolve_source_file function to seek out a directory if provided, but I would rather not go that route if possible. The solutions I think that may work are:

pass the code argument to _resolve_source_file and look for it in the frame_list to then return the correct one. Something like :

```python
def _resolve_source_file(code = None):
"""Resolve source file as last customer frame's module file position."""
try:
frame_list = inspect.stack()
# We find the last frame which is in SDK code instead of customer code or dependencies code
# by checking whether the package name of the frame belongs to azure.ai.ml.component.
pattern = r"(^mldesigner(?=\..*|$).*)"
for frame, last_frame in zip(frame_list, frame_list[1:]):
if _assert_frame_package_name(pattern, frame.frame) and not _assert_frame_package_name(
pattern, last_frame.frame and
):
if code and code not in last_frame.filename:
continue
module = inspect.getmodule(last_frame.frame)
return Path(module.__file__).absolute() if module else None
# pylint: disable=broad-except
except Exception:
return None
```

or

change this into a warning or removing it all together since i am specifying the directory I would like to use.

```python
raise ComponentDefiningError(
name=entity_args["name"],
cause=(
f"Mldesigner component {entity_args['name']!r} source directory {func_entry_path!r} "
f"not under code directory {entity_args['code']!r}"
),
)
```

### Additional details

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by tracing command_component into _resolve_source_file and the ComponentDefiningError path shown in the issue. Reproduce the wrapper case with an explicit code directory, then compare source-file resolution and the resulting directory validation. Done means the intended source directory is selected without requiring regex workarounds or incorrectly raising the shown error.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.