apache / apache/beam

Default Coder/Reading Coder From File

Open
#18,676 0 comments 0 reactions 0 assignees View on GitHub
beam-model new feature P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

*Requirement*-: Need to Run Template With Same Logics on Different Tables Data.(Example is Given Below)

 

*Need*: Default Coder is Required So According to Data It Make All Fields as String and Read Data else Thier must be Dynamic Options to Read Coder From GCS as JSON FILE and Parse Data on Basis of That (But We can Pass Location Using ValueProvider) or SomeWhere Else so At Runtime Using ValueProvider.

 

 

*Examples*: I Have Two Tables 1 is Having Column (NAME, CLASS, ROLL, SUB_PRICE)

And 2 Table is (NAME, ROLL, SUB, TEST_MARKS)

 

On Both Tables, I am Just Sorting Table on Basis Of Roll Number so if We can Read Coder at Run Time The Same Template Can Be Used For Different Tables at Run Time.

 

Such Situations Make Our Work Easy and Make Our job Easy.

 

Imported from Jira [BEAM-3647](https://issues.apache.org/jira/browse/BEAM-3647). Original Jira may contain additional context.
Reported by: KishanK.

Contributor guide

Open the contributing guide

Research direction

The issue names no source file or test; start by locating Beam’s coder configuration and ValueProvider entry points. Clarify the required runtime behavior for a GCS-backed JSON coder and the default string-reading path, using the two differing table schemas as acceptance cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, java
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.