Support Hive MAP columns
- 主要言語
- Scala
- スター
- 147
- フォーク
- 32
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
I get this error when using HiveSink to write to a Hive table with a `MAP` column.
```
java.lang.RuntimeException: Unsupported hive type [map]
at scala.sys.package$.error(package.scala:27)
at io.eels.component.hive.HiveSchemaFns$.fromHiveType(HiveSchemaFns.scala:56)
at io.eels.component.hive.HiveSchemaFns$.fromHive(HiveSchemaFns.scala:25)
at io.eels.component.hive.HiveSchemaFns$.fromHiveField(HiveSchemaFns.scala:22)
at io.eels.component.hive.HiveOps$$anonfun$3.apply(HiveOps.scala:144)
at io.eels.component.hive.HiveOps$$anonfun$3.apply(HiveOps.scala:144)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at io.eels.component.hive.HiveOps.schema(HiveOps.scala:144)
at io.eels.component.hive.HiveSink.open(HiveSink.scala:124)
at io.eels.component.hive.HiveSink$$anonfun$open$1.apply(HiveSink.scala:88)
at io.eels.component.hive.HiveSink$$anonfun$open$1.apply(HiveSink.scala:88)
at scala.collection.generic.GenTraversableFactory.tabulate(GenTraversableFactory.scala:148)
at io.eels.component.hive.HiveSink.open(HiveSink.scala:88)
at io.eels.datastream.SinkAction.execute(SinkAction.scala:46)
at io.eels.datastream.DataStream$class.to(DataStream.scala:727)
at io.eels.datastream.DataStream$$anon$50.to(DataStream.scala:772)
at io.eels.datastream.DataStream$class.to(DataStream.scala:726)
at io.eels.datastream.DataStream$$anon$50.to(DataStream.scala:772)
...
```
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
調査の方向性
The error originates in HiveSchemaFns.scala at line 56, where the fromHiveType function does not handle the MAP type. Examine the Hive type system mapping in that file and the surrounding schema conversion logic. Look for existing support for other complex types like ARRAY or STRUCT to understand the pattern. Adding MAP support will likely involve updating the type matching and creating an appropriate EEL schema field.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- scala
- 領域
- data-engineering, databases
- issue の種類
- 機能追加
- 難易度
- 3/5
- 見積もり時間
- 1〜2日
- 活発さ
- 停滞
- 明瞭さ
- 明確に書かれている
- 初心者へのやさしさ
- 40/100