ClickHouse / ClickHouse/libhdfs3

hdfsBuilderSetPrincipal doesn't work

Open
#11 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
43
Forks
62
PR merge metrics
No merged PRs in 30d

Description

Hi Team,
I got an error like this :
```
2021-03-14 14:10:20.339761, p16581, th139969607461248, INFO Retrying connect to server: "hadoopc1h5:25000". Already tried 9 time(s)
2021-03-14 14:10:22.359599, p16581, th139969607461248, ERROR Failed to setup RPC connection to "hadoopc1h5:25000" caused by:
RpcChannel.cpp: 747: Problem with callback handler
@ Hdfs::Internal::UnWrapper::unwrap(char const*, int)
@ Hdfs::Internal::UnWrapper::unwrap(char const*, int)
@ Hdfs::Internal::UnWrapper::unwrap(char const*, int)
@ Hdfs::Internal::UnWrapper::unwrap(char const*, int)
@ Hdfs::Internal::UnWrapper::unwrap(char const*, int)
@ Hdfs::Internal::HandlerRpcResponseException(std::__exception_ptr::exception_ptr)
@ Hdfs::Internal::RpcChannelImpl::readOneResponse(bool) [clone .cold]
@ Hdfs::Internal::RpcChannelImpl::setupSaslConnection()
@ Hdfs::Internal::RpcChannelImpl::connect()
@ Hdfs::Internal::RpcChannelImpl::invokeInternal(std::shared_ptr)
@ Hdfs::Internal::RpcChannelImpl::invoke(Hdfs::Internal::RpcCall const&)
@ Hdfs::Internal::NamenodeImpl::invoke(Hdfs::Internal::RpcCall const&)
@ Hdfs::Internal::NamenodeImpl::getFsStats()
@ Hdfs::Internal::NamenodeProxy::getFsStats()
@ Hdfs::Internal::FileSystemImpl::getFsStats()
@ Hdfs::Internal::FileSystemImpl::connect()
@ Hdfs::FileSystem::connect(char const*, char const*, char const*)
@ hdfsBuilderConnect
@ main
@ __libc_start_main
@ Unknown
```
and my code and hdfs conf :
```
int main() {
hdfsBuilder* builder = hdfsNewBuilder();
hdfsBuilderSetNameNode(builder, schemaAddress.c_str());
hdfsBuilderSetForceNewInstance(builder);

hdfsBuilderSetPrincipal(builder, "ossuser");

std::stringstream ss;
ss.imbue(std::locale::classic());
ss << "/tmp/krb5cc_0";
std::cout << ss.str() << std::endl;
const char * userCCpath = GetEnv("LIBHDFS3_TEST_USER_CCPATH", ss.str().c_str());
hdfsBuilderSetKerbTicketCachePath(builder, userCCpath);

hdfsFS hdfs = hdfsBuilderConnect(builder);
if (!hdfs) {
die("Could not connect to HDFS server \"" << schemaAddress << "\": " << thrill::vfs::HdfsGetLastError());
}

std::cout << "success connect to hdfs!!!!!!!!!!!" << std::endl;

return 1;
}
```
hdfs-client.xml
```

hadoop.security.authentication
kerberos

the RPC authentication method, valid values include "simple" or "kerberos". default is "simple"

```
klist shows:
```
Ticket cache: FILE:/tmp//krb5cc_0
Default principal: ossuser@HADOOP.COM

Valid starting Expires Service principal
03/14/21 14:09:56 03/15/21 14:09:56 krbtgt/HADOOP.COM@HADOOP.COM
03/14/21 14:10:05 03/15/21 14:09:56 hdfs/hadoop.hadoop.com@HADOOP.COM
```
Am I use the hdfsBuilderSetPrincipal in the wrong way?
Can you please give me some example or reference how to use libhdfs3 to connect kerberized cluster?

Appreciate your support!
Thanks!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.