snowflakedb / snowflakedb/snowpark-python

SNOW-1545867: COPY INTO command doesn't get submitted asynchronously by using collect_nowait

Open
#1,941 7 comments 0 reactions 2 assignees View on GitHub

@sfc-gh-aalam is already working on this.

Since Aug 14, 2024.

status-triage_done
Dominant language
Python
Stars
341
Forks
155
Avg merge
4d 16h
Merged PRs (30d)
27

Description

Please answer these questions before submitting your issue. Thanks!

  1. What version of Python are you using?

3.8

  1. What operating system and processor architecture are you using?

Snowpark Snowflake Stored Procedure

  1. What are the component versions in the environment (pip freeze)?

Snowpark Snowflake Stored Procedure

  1. What did you do?

when submitting copy commands to async jobs.. the copy commands still run in sequence. Whereas other sql's/CTAS does get submitted asynchronously.
for table in table_list:
sql_command -> COPY INTO command
async_job = session.sql(sql_command).collect_nowait()
async_jobs.append(async_job)

  1. What did you expect to see?

COPY INTO should be submitted asynchronously and I should see multiple parallel running COPY commands.

  1. Can you set logging to DEBUG and collect the logs?

Running within Snowflake

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.