alibaba / alibaba/Alink

读本地文件或HDFS文件的区分

Open
#145 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
3.6k
Forks
780
PR merge metrics
No merged PRs in 30d

Description

您好:
我想读本地数据,在数据地址上加 file:// 或者 local:// 或者什么也不加,都报错:
`Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/fs/osshadoop/OSSFileSystemFactory
`
查询官方文档[https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/],确定读写local文件的前缀为“file://”,但是仅在linux的python环境尝试成功,但windows的java环境中始终失败,尝试如下:
`// 不成功
URL = "file://\\E:\\SVN_CODE\\alinkdemo\\data\\input\\ChnSentiCorp_htl_all.csv";`
`// 不成功
URL = "file:///E:/SVN_CODE//alinkdemo/data/input/ChnSentiCorp_htl_all.csv";`
`// 不成功
URL = "file://E:\\SVN_CODE\\alinkdemo\\data\\input\\ChnSentiCorp_htl_all.csv";`
`// 不成功
URL = "file:///E:\\SVN_CODE\\alinkdemo\\data\\input\\ChnSentiCorp_htl_all.csv";`
`// 不成功
URL = "file:\\E:\\SVN_CODE\\alinkdemo\\data\\input\\ChnSentiCorp_htl_all.csv";`
`// 不成功
URL = "file:///E:\\SVN_CODE\\alinkdemo\\data\\input\\ChnSentiCorp_htl_all.csv";`
`// 不成功
URL = "file:///\\E:\\SVN_CODE\\alinkdemo\\data\\input\\ChnSentiCorp_htl_all.csv";`
`// 不成功
URL = "file://\\SVN_CODE\\alinkdemo\\data\\input\\ChnSentiCorp_htl_all.csv";`
`// 不成功
URL = "./data/input/ChnSentiCorp_htl_all.csv";`
请问这个问题怎么解决?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Flink filesystem documentation and the Java Windows reproduction described in the issue, including the listed file:// paths and the OSSFileSystemFactory error. Check how local and HDFS paths are distinguished and what dependency or configuration is used; done means a local Windows file can be read without the NoClassDefFoundError.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.