JuliaParallel / JuliaParallel/Elly.jl

Support for wasb:// protocol on Azure HDInsight

Open
#35 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
45
Forks
12
PR merge metrics
No merged PRs in 30d

Description

I can see the files using `hadoop fs -ls` but not using `readdir`. Trying to create a file reference for a file I know to exist using `HDSFFile` and then `stat` shows `Elly.HDFSException("Path not found")`

```
sshuser@hn0-myclust:~$ hadoop fs -ls /
Found 15 items
drwxr-xr-x - root supergroup 0 2018-02-07 14:25 /HdiSamples
drwxr-xr-x - hdfs supergroup 0 2018-02-07 14:15 /ams
drwxr-xr-x - hdfs supergroup 0 2018-02-07 14:15 /amshbase
drwxrwxrwx - yarn hadoop 0 2018-02-07 14:15 /app-logs
drwxr-xr-x - hdfs supergroup 0 2018-02-07 14:15 /apps
drwxr-xr-x - yarn hadoop 0 2018-02-07 14:15 /atshistory
drwxr-xr-x - root supergroup 0 2018-02-07 14:24 /custom-scriptaction-logs
drwxr-xr-x - root supergroup 0 2018-02-07 14:25 /example
drwxr-xr-x - hbase supergroup 0 2018-02-07 14:15 /hbase
drwxr-xr-x - hdfs supergroup 0 2018-02-07 14:15 /hdp
drwxr-xr-x - hdfs supergroup 0 2018-02-07 14:15 /hive
drwxr-xr-x - mapred supergroup 0 2018-02-07 14:15 /mapred
drwxrwxrwx - mapred hadoop 0 2018-02-07 14:15 /mr-history
drwxrwxrwx - hdfs supergroup 0 2018-02-07 14:15 /tmp
drwxr-xr-x - hdfs supergroup 0 2018-02-07 14:15 /user
sshuser@hn0-myclust:~$ julia
_
_ _ _(_)_ | A fresh approach to technical computing
(_) | (_) (_) | Documentation: https://docs.julialang.org
_ _ _| |_ __ _ | Type "?help" for help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 0.6.2 (2017-12-13 18:08 UTC)
_/ |\__'_|_|_|\__'_| | Official http://julialang.org/ release
|__/ | x86_64-pc-linux-gnu

julia> using Elly

julia> dfs = HDFSClient("hn0-myclust.3p0iyjauoc2e3faws152r5tm0e.cx.internal.cloudapp.net", 8020)
HDFSClient: sshuser@hn0-myclust.3p0iyjauoc2e3faws152r5tm0e.cx.internal.cloudapp.net:8020/
id: 76ba6c80-1ac9-45
connected: false
pwd: /

julia> readdir(dfs)
1-element Array{AbstractString,1}:
"tmp"
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.