apache / apache/beam

Python Gcp tests stalling on machine without credentials

Open
#19,472 0 comments 0 reactions 0 assignees View on GitHub
bug failing test P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

When executing
```

python setup.py nosetests --tests apache_beam.io.gcp.bigquery_file_loads_test:TestBigQueryFileLoads.test_records_traverse_transform_with_mocks

```

the test get stuck in between.

It seems, this test requires proper GCP credentials available. If those are not set, used libraries seem to fall back to some 'local mode' which results in starting a custom webserver and waiting forever.

Impact:
- bad user experience
- creating kind of zombie processes. After killing the test, the web server is still running 'forever' and blocking standard ports (8080, 8090) on dev machine. Also gradle daemon 'never' stops but keeps running in background.

 

 See also thread on mailing list [1]:
```

Running

python setup.py nosetests --tests apache_beam.io.gcp.bigquery_file_loads_test:TestBigQueryFileLoads.test_records_traverse_transform_with_mocks

and
hitting 'Ctrl-C' after it got stuck, results in following output:

'KeyboardInterrupt [while running
\'WriteToBigQuery/BigQueryBatchFileLoads/RemoveTempTables/Delete\']\n------------
Your browser has
been opened to visit:
https://accounts.google.com/o/oauth2/v2/auth?scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fbigquery+https%3A%
If
your browser is on a different machine then exit and re-run this
application with the command-line
parameter
--noauth_local_webserver
Failed to find "code" in the query parameters of the redirect.
Invalid
authorization: Try running with --noauth_local_webserver.

```

[1] https://lists.apache.org/thread.html/80f3de15b04ffb93bac114dbf2fc8623fa5cd8994592072352362ba7@%3Cdev.beam.apache.org%3E

Imported from Jira [BEAM-6949](https://issues.apache.org/jira/browse/BEAM-6949). Original Jira may contain additional context.
Reported by: michel.

Contributor guide

Open the contributing guide

Research direction

Start with apache_beam.io.gcp.bigquery_file_loads_test and run the reported setup.py nosetests command without GCP credentials. Trace the TestBigQueryFileLoads.test_records_traverse_transform_with_mocks execution and the authorization output; done means the test no longer waits indefinitely or leaves web-server and Gradle processes running.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, python
Domain
cloud, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.