alibaba / alibaba/DataX

HdfsWriter在Windows下清理临时文件夹问题

Open
#1,911 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

拼接文件路径时使用了IOUtils.DIR_SEPARATOR
```java
tmpFilePath = String.format("%s__%s%s", userPath, tmpSuffix, IOUtils.DIR_SEPARATOR)
```

在Windows下会通过"\\"拼接。但实际Hdfs上应该都是Linux的"/"。然后导致在HdfsHelper.renameFile方法中,获取父级目录时用的是hadoop-common的Path.getParent,会导致越过一级目录从而将整个数据库删除(如果是hive目录)
```java
tmpFilesParent = srcFilePah.getParent();
```

建议HdfsWriter文件中拼接路径统一采用Linux的分隔符

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in HdfsWriter at the temporary-file path construction using IOUtils.DIR_SEPARATOR, then inspect HdfsHelper.renameFile and its srcFilePah.getParent() handling. Confirm that the resulting HDFS path uses the Linux separator and that cleanup keeps the intended parent directory instead of crossing into a higher-level directory.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.