google / google/tfp-causalimpact

[Import error] Possible conflicts with `tensorflow==2.16.1`, `tensorflow_probaobility==0.23.0`, `causalimpact==0.2.0` on `python==3.11.8` and `python==3.12.2`

Open
#35 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
166
Forks
19
PR merge metrics
No merged PRs in 30d

Description

Dear `tfp-causalimpact` team,

Thank you for a fantastic package. This is just a note for fresh installs in a conda environment.

Premise
=======

I noticed something this morning while trying to install the package on a fresh conda environment (`conda 23.11.0`) with `python==3.11.8`. Running `pip install tfp-causalimpact` installs the latest version of `tensorflow==2.16.1`, and `tensorflow_probability==0.23.0`, which while installing without any errors, produces errors while `importing` (arising from `tfp`).

I have also included a possible workaround below.

Steps to reproduce
================

For `python==3.11.8`
----------------------
```
conda --version
conda 23.11.0
conda create -n causalimpact python==3.11.8
pip install tfp-causalimpact ipython jupyter
ipython

import causalimpact
import pandas as pd
import tensorflow as tf
import tensorflow_probability as tfp
tfd = tfp.distributions
```

For `python==3.12.2`
----------------

The same problem is also reproduced with the following configuration:
```
%watermark -v -m -p numpy,scipy,tensorflow,keras
Python implementation: CPython
Python version : 3.12.2
IPython version : 8.22.2

numpy : 1.26.4
scipy : not installed
tensorflow: 2.16.1
keras : 3.0.5

Compiler : Clang 14.0.6
OS : Darwin
Release : 22.6.0
Machine : x86_64
Processor : i386
CPU cores : 4
Architecture: 64bit
```
Issue
======
There seems to be some clash between these `tensorflow_probability==0.23.0` and `tensorflow==2.16.1`, which causes `import error` for `causalimpact` and here is the error log:
```
In [1]: import causalimpact
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[1], line 1
----> 1 import causalimpact

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/causalimpact/__init__.py:29
26 os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
28 # pylint: disable=g-import-not-at-top
---> 29 from causalimpact.causalimpact_lib import CausalImpactAnalysis
30 from causalimpact.causalimpact_lib import DataOptions
31 from causalimpact.causalimpact_lib import fit_causalimpact

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/causalimpact/causalimpact_lib.py:23
20 import math
21 from typing import Dict, List, Optional, Tuple, Union
---> 23 from causalimpact import posterior_processing
24 import causalimpact.data as cid
25 from causalimpact.indices import InputDateType

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/causalimpact/posterior_processing.py:20
16 """Library for working with (results from) the posterior."""
18 from typing import List, Text, Tuple
---> 20 from causalimpact import data as cid
21 import numpy as np
22 import pandas as pd

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/causalimpact/data.py:23
21 import pandas as pd
22 import tensorflow as tf
---> 23 import tensorflow_probability as tfp
26 class CausalImpactData:
27 """Class for storing and preparing data for modeling.
28
29 This class handles all of the data-related functions of CausalImpact. It
(...)
74 `pre_period`.
75 """

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/__init__.py:20
15 """Tools for probabilistic reasoning in TensorFlow."""
17 # Contributors to the `python/` dir should not alter this file; instead update
18 # `python/__init__.py` as necessary.
---> 20 from tensorflow_probability import substrates
21 # from tensorflow_probability.google import staging # DisableOnExport
22 # from tensorflow_probability.google import tfp_google # DisableOnExport
23 from tensorflow_probability.python import * # pylint: disable=wildcard-import

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/substrates/__init__.py:17
1 # Copyright 2019 The TensorFlow Probability Authors.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
13 # limitations under the License.
14 # ============================================================================
15 """TensorFlow Probability alternative substrates."""
---> 17 from tensorflow_probability.python.internal import all_util
18 from tensorflow_probability.python.internal import lazy_loader # pylint: disable=g-direct-tensorflow-import
21 jax = lazy_loader.LazyLoader(
22 'jax', globals(),
23 'tensorflow_probability.substrates.jax')

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/python/__init__.py:138
135 if _tf_loaded():
136 # Non-lazy load of packages that register with tensorflow or keras.
137 for pkg_name in _maybe_nonlazy_load:
--> 138 dir(globals()[pkg_name]) # Forces loading the package from its lazy loader.
141 all_util.remove_undocumented(__name__, _lazy_load + _maybe_nonlazy_load)

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/python/internal/lazy_loader.py:57, in LazyLoader.__dir__(self)
56 def __dir__(self):
---> 57 module = self._load()
58 return dir(module)

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/python/internal/lazy_loader.py:40, in LazyLoader._load(self)
38 self._on_first_access = None
39 # Import the target module and insert it into the parent's namespace
---> 40 module = importlib.import_module(self.__name__)
41 if self._parent_module_globals is not None:
42 self._parent_module_globals[self._local_name] = module

File ~/anaconda3/envs/tf2161/lib/python3.11/importlib/__init__.py:126, in import_module(name, package)
124 break
125 level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/python/experimental/__init__.py:31
15 """TensorFlow Probability API-unstable package.
16
17 This package contains potentially useful code which is under active development
(...)
27 You are welcome to try any of this out (and tell us how well it works for you!).
28 """
30 from tensorflow_probability.python.experimental import auto_batching
---> 31 from tensorflow_probability.python.experimental import bayesopt
32 from tensorflow_probability.python.experimental import bijectors
33 from tensorflow_probability.python.experimental import distribute

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/python/experimental/bayesopt/__init__.py:17
1 # Copyright 2023 The TensorFlow Probability Authors.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
13 # limitations under the License.
14 # ============================================================================
15 """TensorFlow Probability experimental Bayesopt package."""
---> 17 from tensorflow_probability.python.experimental.bayesopt import acquisition
18 from tensorflow_probability.python.internal import all_util
20 _allowed_symbols = [
21 'acquisition',
22 ]

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/python/experimental/bayesopt/acquisition/__init__.py:19
17 from tensorflow_probability.python.experimental.bayesopt.acquisition.acquisition_function import AcquisitionFunction
18 from tensorflow_probability.python.experimental.bayesopt.acquisition.acquisition_function import MCMCReducer
---> 19 from tensorflow_probability.python.experimental.bayesopt.acquisition.expected_improvement import GaussianProcessExpectedImprovement
20 from tensorflow_probability.python.experimental.bayesopt.acquisition.expected_improvement import ParallelExpectedImprovement
21 from tensorflow_probability.python.experimental.bayesopt.acquisition.expected_improvement import StudentTProcessExpectedImprovement

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/python/experimental/bayesopt/acquisition/expected_improvement.py:19
15 """Expected Improvement."""
17 import tensorflow.compat.v2 as tf
---> 19 from tensorflow_probability.python.distributions import normal
20 from tensorflow_probability.python.distributions import student_t
21 from tensorflow_probability.python.experimental.bayesopt.acquisition import acquisition_function

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/python/distributions/__init__.py:110
108 from tensorflow_probability.python.distributions.pareto import Pareto
109 from tensorflow_probability.python.distributions.pert import PERT
--> 110 from tensorflow_probability.python.distributions.pixel_cnn import PixelCNN
111 from tensorflow_probability.python.distributions.plackett_luce import PlackettLuce
112 from tensorflow_probability.python.distributions.poisson import Poisson

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/python/distributions/pixel_cnn.py:33
31 from tensorflow_probability.python.internal import reparameterization
32 from tensorflow_probability.python.internal import tensorshape_util
---> 33 from tensorflow_probability.python.layers import weight_norm
36 class PixelCNN(distribution.Distribution):
37 """The Pixel CNN++ distribution.
38
39 Pixel CNN++ [(Salimans et al., 2017)][1] models a distribution over image
(...)
228 Learning_, 2016. https://arxiv.org/pdf/1601.06759.pdf
229 """

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/python/layers/__init__.py:27
25 from tensorflow_probability.python.layers.dense_variational import DenseReparameterization
26 from tensorflow_probability.python.layers.dense_variational_v2 import DenseVariational
---> 27 from tensorflow_probability.python.layers.distribution_layer import CategoricalMixtureOfOneHotCategorical
28 from tensorflow_probability.python.layers.distribution_layer import DistributionLambda
29 from tensorflow_probability.python.layers.distribution_layer import IndependentBernoulli

File ~/anaconda3/envs/tf2161/lib/python3.11/site-packages/tensorflow_probability/python/layers/distribution_layer.py:68
47 from tensorflow_probability.python.layers.internal import tensor_tuple
50 __all__ = [
51 'CategoricalMixtureOfOneHotCategorical',
52 'DistributionLambda',
(...)
64 'VariationalGaussianProcess',
65 ]
---> 68 tf.keras.__internal__.utils.register_symbolic_tensor_type(dtc._TensorCoercible) # pylint: disable=protected-access
71 def _event_size(event_shape, name=None):
72 """Computes the number of elements in a tensor with shape `event_shape`.
73
74 Args:
(...)
82 a scalar tensor.
83 """

AttributeError: module 'keras._tf_keras.keras' has no attribute '__internal__'
```

Note that `conda`'s latest `python==3.12.2`

However everything works when I downgrade to `tensorflow==2.15.0`.

Possible workaround
=====================
So a possible solution is to use
`pip install tensorflow==2.15.0 tfp-causalimpact`.

Would it be possible for the team to publish a `REQUIREMENTS.txt` with a frozen `tensorflow==2.15.0` version, until things get sorted with `tensorflow==2.16.1` and `tensorflow_probability==0.23.0`?

My system's settings that work are:

```
%watermark -v -m -p numpy,scipy,tensorflow,tensorflow_probability,keras,causalimpact

Python implementation: CPython
Python version : 3.11.8
IPython version : 8.20.0

numpy : 1.26.4
scipy : 1.12.0
tensorflow : 2.15.0
tensorflow_probability: 0.23.0
keras : 2.15.0
causalimpact : 0.2.0

Compiler : Clang 16.0.6
OS : Darwin
Release : 22.6.0
Machine : x86_64
Processor : i386
CPU cores : 4
Architecture: 64bit
```

Hope you guys can look into it, and maybe this will help someone else stumbling on this same issue.

Contributor guide

Open the contributing guide

Research direction

Reproduce the fresh-install commands for Python 3.11.8 or 3.12.2 and inspect the package dependency metadata. Compare the failing TensorFlow 2.16.1 environment with the working TensorFlow 2.15.0 workaround. Done means a fresh install imports causalimpact and tensorflow_probability without the reported Keras attribute error.

Written by the indexing model from the issue text.

Assessment

Tech stack
keras, python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.