Unicode chars in local file system paths break dataflow runner tests.
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
toResourceIdentifier("/根目录/") failed with following error, and I think it fails with any unicode chars, probably related to:
http://stackoverflow.com/questions/37409379/invalidpathexception-for-chinese-filename
testGetCurrentDirectoryInUnix(org.apache.beam.sdk.io.LocalResourceIdTest) Time elapsed: 0.011 sec <<< ERROR!
java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: /???
at sun.nio.fs.UnixPath.encode(UnixPath.java:147)
at sun.nio.fs.UnixPath.(UnixPath.java:71)
at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281)
at java.nio.file.Paths.get(Paths.java:84)
at org.apache.beam.sdk.io.LocalResourceIdTest.toResourceIdentifier(LocalResourceIdTest.java:244)
at org.apache.beam.sdk.io.LocalResourceIdTest.testGetCurrentDirectoryInUnix(LocalResourceIdTest.java:204)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:239)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
Imported from Jira [BEAM-1453](https://issues.apache.org/jira/browse/BEAM-1453). Original Jira may contain additional context.
Reported by: peihe0@gmail.com.
Contributor guide
Research direction
Start with org.apache.beam.sdk.io.LocalResourceIdTest, especially toResourceIdentifier and testGetCurrentDirectoryInUnix, and run the failing test with the reported Unicode path. Trace the path handling used there and verify the test completes without InvalidPathException for Unicode characters.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- operating-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100