Azure / Azure/azureml-examples
Error in Productionize Vector Index with Test Data Generation, Auto Prompt, Evaluations and Prompt Flow
- Dominant language
- Jupyter Notebook
- Stars
- 2k
- Forks
- 1.7k
- Avg merge
- 18h 18m
- Merged PRs (30d)
- 2
Description
### Operating System
Linux
### Version Information
Python Version:
azure-ai-ml==1.10.0
azureml-core==1.53.0
azureml-rag[faiss,document_parsing,cognitive_search,hugging_face]==0.2.2
### Steps to reproduce
Run the notebook available here:
https://github.com/Azure/azureml-examples/blob/main/sdk/python/generative-ai/rag/notebooks/mlindex_with_testgen_autoprompt.ipynb
This will set up a pipeline to pull some code from a public Github repo and run the job in Azure ML
### Expected behavior
The first step of the pipeline should pull the code from this Github repo https://github.com/MicrosoftDocs/azure-docs/
### Actual behavior
The first step fails, with the following logs:
```
[2023-09-21 10:38:49] INFO azureml.rag.git_clone.git_clone - ActivityStarted, git_clone (activity.py:108)
[2023-09-21 10:38:49] INFO azureml.rag.utils.git - Cloning https://github.com/MicrosoftDocs/azure-docs/ to /mnt/azureml/cr/j/16637992a6bb45fd983208619fe47612/cap/data-capability/wd/output_data (git.py:73)
[2023-09-21 10:38:49] INFO azureml.rag.utils.git - Cloning with depth=1 (git.py:82)
[2023-09-21 10:38:49] ERROR azureml.rag.utils.git - Failed to clone to /mnt/azureml/cr/j/16637992a6bb45fd983208619fe47612/cap/data-capability/wd/output_data
git stdout:
git stderr: (git.py:86)
[2023-09-21 10:38:49] ERROR azureml.rag.git_clone.git_clone - git_clone failed with exception: Traceback (most recent call last):
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/git_clone.py", line 49, in main_wrapper
main(args, logger, activity_logger)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/git_clone.py", line 41, in main
raise e
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/git_clone.py", line 38, in main
clone_repo(args.git_repository, args.output_data, args.branch_name, authentication)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/utils/git.py", line 88, in clone_repo
raise e
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/utils/git.py", line 84, in clone_repo
repo = git.Repo.clone_from(git_repo_branch.git_url, local_path, progress=GitCloneProgress(), depth=1 if git_repo_branch.branch_name is None else None)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/git/repo/base.py", line 1327, in clone_from
return cls._clone(
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/git/repo/base.py", line 1223, in _clone
handle_process_output(
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/git/cmd.py", line 205, in handle_process_output
return finalizer(process)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/git/util.py", line 434, in finalize_process
proc.wait(**kwargs)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/git/cmd.py", line 602, in wait
raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
cmdline: git clone -v --depth=1 --progress -- https://github.com/MicrosoftDocs/azure-docs/.git /mnt/azureml/cr/j/16637992a6bb45fd983208619fe47612/cap/data-capability/wd/output_data
(git_clone.py:51)
[2023-09-21 10:38:49] ERROR azureml.rag.git_clone.git_clone - ActivityCompleted: Activity=git_clone, HowEnded=Failure, Duration=286.79 [ms], Exception=GitCommandError (activity.py:127)
Traceback (most recent call last):
File "/azureml-envs/rag-embeddings/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/azureml-envs/rag-embeddings/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/git_clone.py", line 66, in
main_wrapper(args, logger)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/git_clone.py", line 49, in main_wrapper
main(args, logger, activity_logger)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/git_clone.py", line 41, in main
raise e
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/tasks/git_clone.py", line 38, in main
clone_repo(args.git_repository, args.output_data, args.branch_name, authentication)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/utils/git.py", line 88, in clone_repo
raise e
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/azureml/rag/utils/git.py", line 84, in clone_repo
repo = git.Repo.clone_from(git_repo_branch.git_url, local_path, progress=GitCloneProgress(), depth=1 if git_repo_branch.branch_name is None else None)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/git/repo/base.py", line 1327, in clone_from
return cls._clone(
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/git/repo/base.py", line 1223, in _clone
handle_process_output(
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/git/cmd.py", line 205, in handle_process_output
return finalizer(process)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/git/util.py", line 434, in finalize_process
proc.wait(**kwargs)
File "/azureml-envs/rag-embeddings/lib/python3.8/site-packages/git/cmd.py", line 602, in wait
raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
cmdline: git clone -v --depth=1 --progress -- https://github.com/MicrosoftDocs/azure-docs/.git /mnt/azureml/cr/j/16637992a6bb45fd983208619fe47612/cap/data-capability/wd/output_data
```
### Addition information
_No response_
Contributor guide
Research direction
Start with the referenced notebook, mlindex_with_testgen_autoprompt.ipynb, and rerun its pipeline setup to reproduce the failing git clone step. Inspect the reported GitCommandError and clone configuration; done means the first pipeline step successfully pulls the MicrosoftDocs/azure-docs repository.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, git, github, python
- Domain
- cloud, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100