51zero / 51zero/eel-sdk

Support Hive MAP columns

Abierto
#388 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Scala
Estrellas
147
Forks
32
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

I get this error when using HiveSink to write to a Hive table with a `MAP` column.

```
java.lang.RuntimeException: Unsupported hive type [map]
at scala.sys.package$.error(package.scala:27)
at io.eels.component.hive.HiveSchemaFns$.fromHiveType(HiveSchemaFns.scala:56)
at io.eels.component.hive.HiveSchemaFns$.fromHive(HiveSchemaFns.scala:25)
at io.eels.component.hive.HiveSchemaFns$.fromHiveField(HiveSchemaFns.scala:22)
at io.eels.component.hive.HiveOps$$anonfun$3.apply(HiveOps.scala:144)
at io.eels.component.hive.HiveOps$$anonfun$3.apply(HiveOps.scala:144)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at io.eels.component.hive.HiveOps.schema(HiveOps.scala:144)
at io.eels.component.hive.HiveSink.open(HiveSink.scala:124)
at io.eels.component.hive.HiveSink$$anonfun$open$1.apply(HiveSink.scala:88)
at io.eels.component.hive.HiveSink$$anonfun$open$1.apply(HiveSink.scala:88)
at scala.collection.generic.GenTraversableFactory.tabulate(GenTraversableFactory.scala:148)
at io.eels.component.hive.HiveSink.open(HiveSink.scala:88)
at io.eels.datastream.SinkAction.execute(SinkAction.scala:46)
at io.eels.datastream.DataStream$class.to(DataStream.scala:727)
at io.eels.datastream.DataStream$$anon$50.to(DataStream.scala:772)
at io.eels.datastream.DataStream$class.to(DataStream.scala:726)
at io.eels.datastream.DataStream$$anon$50.to(DataStream.scala:772)
...
```

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Línea de trabajo

The error originates in HiveSchemaFns.scala at line 56, where the fromHiveType function does not handle the MAP type. Examine the Hive type system mapping in that file and the surrounding schema conversion logic. Look for existing support for other complex types like ARRAY or STRUCT to understand the pattern. Adding MAP support will likely involve updating the type matching and creating an appropriate EEL schema field.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
scala
Área
data-engineering, databases
Tipo de issue
Nueva funcionalidad
Dificultad
3/5
Tiempo estimado
1-2 días
Estado de actividad
Estancado
Claridad
Bien especificado
Aptitud para principiantes
40/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.