twosigma / twosigma/flint

TypeError: 'JavaPackage' object is not callable

Open
#77 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
1.2k
Forks
199
PR merge metrics
No merged PRs in 30d

Description

Whenever I try to use Flint here locally (no Hadoop/EMR involved), it keep barfing at me with the above error message in the subject. It's a setup on top of Python 3.7 with PySpark 2.4.4 and OpenJDK 8; an Ubuntu 19.04 install.

Note: As I'm running locally only, I'm getting this log message from Spark, but everything does run perfectly using vanilla PySpark:

```
19/10/23 09:59:50 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
```

It happens when I try to either read a PySpark dataframe into a `ts.flint.TimeSeriesDataFrame`. This example is adapted from the `Flint Example.ipynb`:

```python
import pyspark
import ts.flint
from ts.flint import FlintContext

sc = pyspark.SparkContext('local', 'Flint Example')
spark = pyspark.sql.SparkSession(sc)
flint_context = FlintContext(spark)

sp500 = (
spark.read
.option('header', True)
.option('inferSchema', True)
.csv('sp500.csv')
.withColumnRenamed('Date', 'time')
)
sp500 = flint_context.read.dataframe(sp500)
```

The last line causes the "boom", with this (first part of) the stack trace:

```
TypeError Traceback (most recent call last)
~/.virtualenvs/pyspark-test/lib/python3.7/site-packages/ts/flint/java.py in new_reader(self)
37 try:
---> 38 return utils.jvm(self.sc).com.twosigma.flint.timeseries.io.read.TSReadBuilder()
39 except TypeError:

TypeError: 'JavaPackage' object is not callable
```

Any ideas what may be going wrong and how the problem could be solved?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.