NYC Taxi Demo failing at get_offline_features with Spark error
- Dominant language
- Scala
- Stars
- 1.9k
- Forks
- 247
- PR merge metrics
- No merged PRs in 30d
Description
Attempting to run the latest version of the NYC taxi demo in my own environment, the Spark job for get_offline_features fails each time.
I have attached the logs from Synapse below. I have tried using both the LATEST .jar file and the 0.1.0-SNAPSHOT with the same result.
When troubleshooting, I also tried removing the UDF aggregating `fare_amount_cents`, but the same error arose. One thing changed: when the UDF was present, the error was only visible in the Driver stdout logs, but when I removed the UDF, it shows up in the Driver stderr logs.
```
22/05/17 20:28:13 ERROR ApplicationMaster: User class threw exception: com.typesafe.config.ConfigException$Parse: String: 3: Expecting end of input or a comma, got ':' (if you intended ':' to be part of a key or string value, try enclosing the key or value in double quotes)
com.typesafe.config.ConfigException$Parse: String: 3: Expecting end of input or a comma, got ':' (if you intended ':' to be part of a key or string value, try enclosing the key or value in double quotes)
at com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parseError(ConfigDocumentParser.java:201)
at com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parseError(ConfigDocumentParser.java:197)
at com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parseObject(ConfigDocumentParser.java:535)
at com.typesafe.config.impl.ConfigDocumentParser$ParseContext.parse(ConfigDocumentParser.java:648)
at com.typesafe.config.impl.ConfigDocumentParser.parse(ConfigDocumentParser.java:14)
at com.typesafe.config.impl.Parseable.rawParseValue(Parseable.java:260)
at com.typesafe.config.impl.Parseable.rawParseValue(Parseable.java:248)
at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:180)
at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:174)
at com.typesafe.config.impl.Parseable.parse(Parseable.java:299)
at com.typesafe.config.ConfigFactory.parseString(ConfigFactory.java:1001)
at com.typesafe.config.ConfigFactory.parseString(ConfigFactory.java:1011)
at com.linkedin.feathr.offline.config.datasource.DataSourceConfig.$anonfun$config$1(DataSourceConfig.scala:11)
at scala.Option.map(Option.scala:230)
at com.linkedin.feathr.offline.config.datasource.DataSourceConfig.(DataSourceConfig.scala:11)
at com.linkedin.feathr.offline.config.datasource.DataSourceConfigs.parseConfigStr(DataSourceConfigs.scala:29)
at com.linkedin.feathr.offline.config.datasource.DataSourceConfigs.(DataSourceConfigs.scala:26)
at com.linkedin.feathr.offline.config.datasource.DataSourceConfigUtils$.getConfigs(DataSourceConfigUtils.scala:37)
at com.linkedin.feathr.offline.job.FeatureJoinJob$.parseInputArgument(FeatureJoinJob.scala:247)
at com.linkedin.feathr.offline.job.FeatureJoinJob$.prepareSparkSession(FeatureJoinJob.scala:318)
at com.linkedin.feathr.offline.job.FeatureJoinJob$.main(FeatureJoinJob.scala:312)
at com.linkedin.feathr.offline.job.FeatureJoinJob.main(FeatureJoinJob.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:732)
```
Contributor guide
Assessment
This issue has not been assessed yet.