apache / apache/hop

[Feature Request]: Add AVRO Logical types to Avro File Output transform

Open
#3,804 0 comments 0 reactions 0 assignees View on GitHub
awaiting triage P2 Transforms
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
18h 33m
Merged PRs (30d)
204

Description

### What would you like to happen?

Currently HOP supports only Avro primitive types in transfrom.
For example Date is stored as INT, and only Logical type attribute identifies it as Date.
Field definition in schema, in this case, according Avro documentation:
"type" : [ "null", { "type": "int", "logicalType": "date" } ]

[_Avro_docs_](https://avro.apache.org/docs/1.8.0/spec.html#Logical+Types)

**One way of implementing:**
List all Primitive and Logical types in the same dropdown, and create fields in schema based on documentation.

**Another approach:**
1. Add column named "Avro Logical type", with dropdown selector.
Dropdown should list all available Logical types (7 entries).
Empty value should be on the list .
2. Rename "Avro type" to "Avro primitive type"

If Logical field is set, then:
1. Disable "Avro Primitive type" column editing for that field
2. Set primitive type to correct one according to Avro documentation.

Implementation could look:
![image](https://github.com/apache/hop/assets/210065/0aa56947-aa59-47c8-9bf2-9ebb9324fe46)

### Issue Priority

Priority: 2

### Issue Component

Component: Transforms

Contributor guide

Open the contributing guide

Research direction

The issue names the Avro File Output transform but no files or tests. Start by locating that transform and reviewing its existing Avro primitive-type field configuration alongside the linked Avro logical-types documentation. Done means logical types can be selected, their corresponding primitive types are handled correctly, and the generated schema matches the requested Avro representation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.