KERBEROS auth hive error

[prompt-bot]

we connect hive with kerberos, kinit was done, but some wrong happend:

Failed to open Apache Hive connection: Bad SASL negotiation status: 3 (Final handshake step failed)

[beltran]

Either hive or gohive isn't set up correctly, what is the exception in the hive logs? What is the output of klist on the client?

[prompt-bot]

after running shell command kinit -kt ./xxx/xx/x.keytab [email protected] :

Ticket cache: FILE:/tmp/krb5cc_1002
Default principal: [email protected]

Valid starting Expires Service principal
2020-11-10T13:03:47 2020-11-11T13:03:47 krbtgt/[email protected]
renew until 2020-11-17T13:03:47

then running go run -tags kerberos example/main.go

Ticket cache: FILE:/tmp/krb5cc_1002
Default principal: [email protected]

Valid starting Expires Service principal
2020-11-10T13:03:47 2020-11-11T13:03:47 krbtgt/[email protected]
renew until 2020-11-17T13:03:47
2020-11-10T13:03:55 2020-11-11T13:03:47 hive/[email protected]
renew until 2020-11-15T13:03:55

[beltran]

Probably hive is not configured correctly. What is the stack trace you see in the hive logs when the connection fails? Also you can try to connect with beeline for example and see if it's working, you'd have to do something like beeline -u "jdbc:hive2://$HOST:10000/;principal=hive/[email protected];hive.server2.proxy.user=hive/[email protected]"

[prompt-bot]

just beeline -u "jdbc:hive2://db01.example.com:10000/sx_db;principal=hive/[email protected];hive.server2.proxy.user=hive"

work well, kdc just create principal with [email protected] see on

[testuser1@host-10-1-100-12 ~]$ klist -k /data/hive.keytab
Keytab name: FILE:/data/hive.keytab
KVNO Principal
---- --------------------------------------------------------------------------
   1 [email protected]
   1 [email protected]
   1 [email protected]
   1 [email protected]
   1 [email protected]
   1 [email protected]

not any principal like 'hive/$[email protected]' , Could that be the reason os this?

I see this value in klist after running example program like go run -tags kerberos example/main.go:
#128 (comment)

a new principal ticket create: hive/[email protected]

but I can't see the hive logs of the current problem right now

[prompt-bot]

@beltran hive-server2 logs found：

2020-11-11 11:03:30,927 ERROR org.apache.thrift.server.TThreadPoolServer: [HiveServer2-Handler-Pool: Thread-59]: Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream
	at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) ~[hive-exec-2.1.1-cdh6.1.1.jar:2.1.1-cdh6.1.1]
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:652) ~[hive-exec-2.1.1-cdh6.1.1.jar:2.1.1-cdh6.1.1]
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge.java:649) ~[hive-exec-2.1.1-cdh6.1.1.jar:2.1.1-cdh6.1.1]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_181]
	at javax.security.auth.Subject.doAs(Subject.java:360) ~[?:1.8.0_181]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1711) ~[hadoop-common-3.0.0-cdh6.1.1.jar:?]
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge.java:649) ~[hive-exec-2.1.1-cdh6.1.1.jar:2.1.1-cdh6.1.1]
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) [hive-exec-2.1.1-cdh6.1.1.jar:2.1.1-cdh6.1.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream
	at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:328) ~[hive-exec-2.1.1-cdh6.1.1.jar:2.1.1-cdh6.1.1]
	at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) ~[hive-exec-2.1.1-cdh6.1.1.jar:2.1.1-cdh6.1.1]
	at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) ~[hive-exec-2.1.1-cdh6.1.1.jar:2.1.1-cdh6.1.1]
	... 10 more

[beltran]

Are these or similar parmeters set in hive: https://github.com/beltran/gohive#connect-with-sasl-kerberos?

The error I see is what I'd expect if hive.server2.authentication = NONE, what is the value for hive.server2.authentication, you should be able to see it connecting with beeline and then SET hive.server2.authentication;

[prompt-bot]

I found hive.server2.authentication=KERBEROS in hive-sites.xml，but connected by beeline and SET hive.server2.authentication; get： hive.server2.authentication=LDAP..., but hive command using this hive-sites.xml and beeline work well.

I found dual authentication set， https://docs.cloudera.com/documentation/enterprise/latest/topics/cdh_sg_hiveserver2_security.html#topic_9_1_1
so hive.server2.authentication=LDAP,
why beeeline connect just use principal，we must depend on kinit tool .

sorry, I'm not very familiar with hive and his certification, please

[beltran]

No problem, then to connect you should do something like this:

configuration := gohive.NewConnectConfiguration()
configuration.Username = "hive" // your username
configuration.Service = "password" // your password
=connection, errConn := gohive.Connect("hs2.example.com", 10000, "LDAP", configuration)

If you don't have a password you can leave it empty or sett it to "x"

[prompt-bot]

The problem is still.

all service authorization with KERBEROS, So I don't have a LDAP password, with CDH dual authentication set.

Why did it create a new one Service principal, host for hive-server2?
hive/[email protected]

after kinit -kt ./xxx/xx/x.keytab [email protected], command line hive works normally.

[beltran]

hmmm, what error are you getting now? Maybe you're using both kerberos and ldap auth, like in the note in this link: https://docs.cloudera.com/documentation/enterprise/latest/topics/cdh_sg_hiveserver2_security.html, in that case you should set KERBEROS and not LDAP in gohive.Connect(, I've never tried this but by the error you were getting(Failed to open Apache Hive connection: Bad SASL negotiation status: 3 (Final handshake step failed)) it looks like something is misconfigured

[prompt-bot]

yes, I set KERBEROS for gohive.Connect( ,but not any valuable error messages in krb5kdc.log and hive log, so I don't know threre Any error configuration for gssapi

and some others suggestion, why don't we use the same go library jcmturner/gokrb5 like calcite-avatica-go, maybe we don't need to rely on C library libgssapi_krb5.so.