AbsaOSS / AbsaOSS/cobrix

Low Values in Binary Data

Open
#835 4 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Scala
Stars
170
Forks
96
Avg merge
57m
Merged PRs (30d)
2

Description

#Background

Currently we are using Cobrix with Databricks to read EBCDI files from Mainframe and convert them to Parquet. All is working fine. Some of the fields have Low values along with data in the Character/String fields. When we read the binary files they look like empty boxes. All is good till here.But when we load them to Postgres DB the load is failing as it is not able to recognize these characters.

## Question
We can definitely write a spark code to replace these as Spaces or something else. But will like to know if Cobrix has any function which we can apply so that it can convert these unrecogniized characters to a Default Value?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.