AbsaOSS / AbsaOSS/cobrix

Add an option to automatically detect record type

Open
#355 0 comments 0 reactions 1 assignee Claimed by @yruslan View on GitHub
enhancement
Dominant language
Scala
Stars
170
Forks
96
Avg merge
57m
Merged PRs (30d)
2

Description

## Background
Determining record type is critical for reading mainframe files. Yet, most issues raised are related to the confusion of specifying the type of records in a mainframe file. Simplifying this bit would help getting started with Cobrix quicker.

## Feature
Add an option, say,
```
.option("infer_record_type", "true")
```
that would read the input file and determine:
1. Presence and type of RDW (big-endian or little-endian)
2. True record size for fixed record-length files

Cobrix should log options used for reading such files so that the automatic inference could be replaced by these explicit options for future jobs.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.