tensorflow / tensorflow/privacy
gaussian_query_test.py tests fail
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2k
- Forks
- 477
- Avg merge
- 22h 12m
- Merged PRs (30d)
- 1
Description
Cloned the repo, conda install tensorflow, followed setup instructions inside the repo and then tried to run gaussian_query_test.py but it seems that some of the tests fail.
Running tests under Python 3.7.5: //anaconda3/envs/tf/bin/python
[ RUN ] GaussianQueryTest.test_gaussian_average_no_noise
2019-11-14 00:57:05.454391: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations: SSE4.1 SSE4.2 AVX AVX2 FMA
To enable them in non-MKL-DNN operations, rebuild TensorFlow with the appropriate compiler flags.
2019-11-14 00:57:05.456672: I tensorflow/core/common_runtime/process_util.cc:115] Creating new thread pool with default inter op setting: 4. Tune using inter_op_parallelism_threads for best performance.
[ OK ] GaussianQueryTest.test_gaussian_average_no_noise
[ RUN ] GaussianQueryTest.test_gaussian_average_with_noise
[ FAILED ] GaussianQueryTest.test_gaussian_average_with_noise
[ RUN ] GaussianQueryTest.test_gaussian_sum_merge
[ OK ] GaussianQueryTest.test_gaussian_sum_merge
[ RUN ] GaussianQueryTest.test_gaussian_sum_no_clip_no_noise
[ OK ] GaussianQueryTest.test_gaussian_sum_no_clip_no_noise
[ RUN ] GaussianQueryTest.test_gaussian_sum_with_changing_clip_no_noise
[ FAILED ] GaussianQueryTest.test_gaussian_sum_with_changing_clip_no_noise
[ RUN ] GaussianQueryTest.test_gaussian_sum_with_clip_no_noise
[ OK ] GaussianQueryTest.test_gaussian_sum_with_clip_no_noise
[ RUN ] GaussianQueryTest.test_gaussian_sum_with_noise
[ FAILED ] GaussianQueryTest.test_gaussian_sum_with_noise
[ RUN ] GaussianQueryTest.test_incompatible_records_too_few_on_left
[ OK ] GaussianQueryTest.test_incompatible_records_too_few_on_left
[ RUN ] GaussianQueryTest.test_incompatible_records_too_few_on_right
[ OK ] GaussianQueryTest.test_incompatible_records_too_few_on_right
[ RUN ] GaussianQueryTest.test_incompatible_records_type_mismatch
[ OK ] GaussianQueryTest.test_incompatible_records_type_mismatch
[ RUN ] GaussianQueryTest.test_session
[ SKIPPED ] GaussianQueryTest.test_session
======================================================================
ERROR: test_gaussian_sum_with_changing_clip_no_noise (__main__.GaussianQueryTest)
test_gaussian_sum_with_changing_clip_no_noise (__main__.GaussianQueryTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "gaussian_query_test.py", line 62, in test_gaussian_sum_with_changing_clip_no_noise
l2_norm_clip_placeholder = tf.compat.v1.placeholder(tf.float32)
File "//anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/ops/array_ops.py", line 2627, in placeholder
raise RuntimeError("tf.placeholder() is not compatible with "
RuntimeError: tf.placeholder() is not compatible with eager execution.
======================================================================
FAIL: test_gaussian_average_with_noise (__main__.GaussianQueryTest)
test_gaussian_average_with_noise (__main__.GaussianQueryTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "gaussian_query_test.py", line 149, in test_gaussian_average_with_noise
self.assertNear(result_stddev, avg_stddev, 0.1)
File "//anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/framework/test_util.py", line 1122, in decorated
return f(*args, **kwds)
File "//anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/framework/test_util.py", line 2225, in assertNear
(f1, f2, err, " (%s)" % msg if msg is not None else ""))
AssertionError: False is not true : 0.000000 != 0.500000 +/- 0.100000
======================================================================
FAIL: test_gaussian_sum_with_noise (__main__.GaussianQueryTest)
test_gaussian_sum_with_noise (__main__.GaussianQueryTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "gaussian_query_test.py", line 93, in test_gaussian_sum_with_noise
self.assertNear(result_stddev, stddev, 0.1)
File "//anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/framework/test_util.py", line 1122, in decorated
return f(*args, **kwds)
File "//anaconda3/envs/tf/lib/python3.7/site-packages/tensorflow_core/python/framework/test_util.py", line 2225, in assertNear
(f1, f2, err, " (%s)" % msg if msg is not None else ""))
AssertionError: False is not true : 0.000000 != 1.000000 +/- 0.100000
----------------------------------------------------------------------
Ran 11 tests in 0.645s
FAILED (failures=2, errors=1, skipped=1)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with gaussian_query_test.py and rerun the test file in the documented Python/TensorFlow environment. Inspect the eager-execution failure and the noise-related assertions shown in the traceback. Done means the previously failing tests pass without breaking the tests that already succeed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100