microsoft / microsoft/promptflow

[Feature Request] Improve compatibility between PromptFlow and Python modules with relative imports

Open
#3,209 9 comments 0 reactions 1 assignee View on GitHub

@D-W- is already working on this.

Since May 11, 2024.

enhancement long-term no-recent-activity
Dominant language
Python
Stars
11.2k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

I am trying to use the promptflow-evals SDK in a project where I am using relative imports, which works fine because of how I call the modules (with python -m modulename).

However, PromptFlow tries to import my file for some reason, and then errors:

  File "/Users/pamelafox/ai-rag-chat-evaluator/scripts/evaluate.py", line 142, in run_evaluation
    results = evaluate(
              ^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/evals/evaluate/_evaluate.py", line 252, in evaluate
    input_data_df, target_generated_columns, target_run = _apply_target_to_data(target, data, pf_client,
                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/evals/evaluate/_evaluate.py", line 128, in _apply_target_to_data
    run = pf_client.run(
          ^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_pf_client.py", line 301, in run
    return self._run(
           ^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_pf_client.py", line 226, in _run
    return self.runs.create_or_update(run=run, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_telemetry/activity.py", line 265, in wrapper
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/operations/_run_operations.py", line 134, in create_or_update
    created_run = RunSubmitter(client=self._client).submit(run=run, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_orchestrator/run_submitter.py", line 42, in submit
    self._run_bulk(run=run, stream=stream, **kwargs)
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_orchestrator/run_submitter.py", line 111, in _run_bulk
    with flow_overwrite_context(flow_obj, tuning_node, variant, connections=run.connections) as flow:
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_orchestrator/utils.py", line 264, in flow_overwrite_context
    override_flow_yaml(
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_orchestrator/utils.py", line 219, in override_flow_yaml
    update_signatures(code=flow_dir_path, data=flow_dag)
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_utilities/signature_utils.py", line 148, in update_signatures
    signatures, _, _ = infer_signature_for_flex_flow(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_sdk/_utilities/signature_utils.py", line 72, in infer_signature_for_flex_flow
    flow_meta = inspector_proxy.get_entry_meta(entry=entry, working_dir=code)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_proxy/_python_inspector_proxy.py", line 47, in get_entry_meta
    return _generate_flow_meta(
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/pamelafox/ai-rag-chat-evaluator/.venv/lib/python3.11/site-packages/promptflow/_core/entry_meta_generator.py", line 84, in _generate_flow_meta
    raise GenerateFlowMetaJsonError(error_message)
promptflow.core._errors.GenerateFlowMetaJsonError: Generate meta failed, detail error:
["Failed to load python module from file '/Users/pamelafox/ai-rag-chat-evaluator/scripts/evaluate.py': (ImportError) attempted relative import with no known parent package"]

So now I have to restructure my project to avoid relative imports. Please loosen this constraint.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.