apache / apache/beam

filesToStage should allow for common ways in which people package their resources

Open
#18,099 0 comments 0 reactions 0 assignees View on GitHub
dataflow improvement P3 runners
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Different application environments launch and maintain their classpath resources in various ways. See these SO questions for examples of how people launch their pipelines that currently are unsupported:
http://stackoverflow.com/questions/31978566/detectclasspathresourcestostage-unable-to-convert-url
http://stackoverflow.com/questions/40099952/launching-dataflow-jobs-from-a-java-application

Add support for classpaths which:
* use URLs that are embedded within other jars
* allow for manifest files that specify their classpaths
* add support for rsrc:// URIs to support Eclipse jar packaging

Imported from Jira [BEAM-779](https://issues.apache.org/jira/browse/BEAM-779). Original Jira may contain additional context.
Reported by: lcwik.

Contributor guide

Open the contributing guide

Research direction

Start by locating the filesToStage implementation and reviewing the linked Stack Overflow examples and BEAM-779 context. Done means classpaths using URLs embedded in other jars, manifest-specified classpaths, and rsrc:// URIs are supported so their resources can be staged.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.