Azure / Azure/MachineLearningNotebooks
Using ParallelRunConfig and DockerConfig not working
- Lenguaje dominante
- Jupyter Notebook
- Estrellas
- 4.4k
- Forks
- 2.6k
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
Hello, I am attempting to use the ParallelRunStep for a set of python scripts that I currently run inside a docker container. I am trying to deploy this using an AzureML pipeline that reads from the docker image saved in our ACR. However, the init and run scripts in my parallel script never seem to be able to access the files I have moved into the docker container. Particularly when I try and search for a hydra config file in my init and run scripts, I am unable to find the file, even when I look for it explicitly. I have included a snippet that is based on the example code from this repo: [example of parallel run config](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/machine-learning-pipelines/parallel-run/tabular-dataset-partition-per-column.ipynb)
Any idea what I am doing wrong? I have included a snippet of what I am trying below
```import os
docker_config = DockerConfiguration(use_docker=True, arguments=docker_args)
environment_name = "my-environment"
environment = Environment(environment_name)
base_image_name = os.getenv("ACR_BASE_IMAGE_NAME")
base_image_tag = os.getenv("ACR_IMAGE_TAG")
environment.docker.base_image = f"{base_image_name}:{base_image_tag}"
environment.docker.base_image_registry.address = f"{acr_name}.azurecr.io"
environment.docker.base_image_registry.username = os.getenv("ACR_USER")
environment.docker.base_image_registry.password = os.getenv("ACR_PASSWORD")
environment.python.user_managed_dependencies = True
environment.docker.enabled=True
run_config = RunConfiguration()
run_config.environment = environment
run_config.docker = docker_config
parallel_run_config = ParallelRunConfig(
source_directory=".",
entry_script="path/to/docker/script_1.py",
compute_target=compute_target,
environment=environment,
node_count=2,
error_threshold=10,
output_action="append_row",
mini_batch_size=1,
logging_level='DEBUG'
)
step_parallel = ParallelRunStep(
name="parallel-step",
parallel_run_config=parallel_run_config,
inputs=my_inputs,
output=output_dir,
arguments=args,
allow_reuse=True,
)
step_parallel._runconfig.docker = docker_config #Tried with and without, does not seem to make a difference
pipeline_steps = StepSequence(steps=[step_parallel])
pipeline_run = Pipeline(workspace=ws, steps=pipeline_steps)
# Submit your pipeline run
submitted_pipeline_run = Experiment(ws, "Azure Pipeline").submit(pipeline_run, regenerate_outputs=True)```
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Línea de trabajo
Comienza con el notebook vinculado de ejecución en paralelo y compara sus configuraciones ParallelRunConfig, DockerConfiguration y source_directory con el fragmento proporcionado. Sigue path/to/docker/script_1.py y los scripts init y run para determinar cómo localizan los archivos en la imagen. Se considera terminado cuando los scripts pueden encontrar de forma fiable la configuración de Hydra esperada cuando se ejecuta el pipeline.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- azure, docker, jupyter-notebook, python
- Área
- devops, machine-learning
- Tipo de issue
- Error
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Estancado
- Claridad
- Necesita aclaración
- Aptitud para principiantes
- 30/100