alibaba / alibaba/DataX

关于文件上传大小的限制

Open
#904 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

你好,我通过下载在这里提供的datax功能,执行python bin/datax.py job/xx.json这个命令上传文件到oss时,是不是会限制文件大小不能超过5G呢,现在我超过5G的文件就上传不了,我单独在linux服务器上执行这个命令,超过5G的文件是可以上传,但是我在java代码里面,通过这种调用方式,他就一直没有反应,请问是什么原因呢
String startCommand = "python " + dataxHomePath + "bin/datax.py " + xx.json;
Process pr = Runtime.getRuntime().exec(startCommand);

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the reported command, bin/datax.py job/xx.json, and the Java Runtime.exec call. Reproduce the difference between running the command directly on Linux and launching it from Java with a file larger than 5G, then inspect the process behavior and DataX output. Done means the upload-size behavior and the reason for the Java invocation hanging are identified and documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.