window 环境下jdbc访问启用kerberos的impala

时间 2019-12-12

标签 window 环境 jdbc 访问启用 kerberos impala 栏目 Windows 繁體版

原文原文链接

最近，公司生产集群添加kerberos安全认证后，访问集群的任何组件都须要进行认证，这样问题来了，对于impala，未配置kerberos安全认证以前经过impala的jdbc驱动(impala-jdbc)，配置链接字符串java

jdbc:impala://192.168.1.10:21050/default;AuthMech=3;UID=user;PWD=pwd;UseSasl=0

是能够正常访问的，可是开启了kerberos后，impala访问报错，通过阅读impala jdbc使用文档(https://www.cloudera.com/documentation/other/connectors/impala-jdbc/2-5-5/Cloudera-JDBC-Driver-for-Impala-Install-Guide-2-5-5.pdf)，能够发现，对于启用kerberos的impala链接字符串须要调整为以下：sql

jdbc:impala://192.168.1.10:21050/default;AuthMech=1;KrbHostFQDN=hostalias;KrbServiceName=impala

其中：KrbHostFQDN须要指定链接哪台服务器的impalad，须要使用服务器的别名。apache

调整jdbc配置，链接impala失败，异常信息以下api

java.sql.SQLException: [Simba][ImpalaJDBCDriver](500168) Error creating login context using ticket cache: Unable to obtain Principal Name for authentication .
 ......
Caused by: com.cloudera.support.exceptions.GeneralException: [Simba][ImpalaJDBCDriver](500168) Error creating login context using ticket cache: Unable to obtain Principal Name for authentication .
    ... 39 more
Caused by: javax.security.auth.login.LoginException: Unable to obtain Principal Name for authentication 
    at com.sun.security.auth.module.Krb5LoginModule.promptForName(Krb5LoginModule.java:841)
    at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:704)
    at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
    at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
    at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
    at com.cloudera.jdbc.kerberos.Kerberos.getSubjectViaTicketCache(Unknown Source)
    at com.cloudera.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
    at com.cloudera.hivecommon.api.HiveServer2ClientFactory.createClient(Unknown Source)
    at com.cloudera.hivecommon.core.HiveJDBCCommonConnection.connect(Unknown Source)
    at com.cloudera.impala.core.ImpalaJDBCConnection.connect(Unknown Source)
    at com.cloudera.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
    at com.cloudera.jdbc.common.AbstractDriver.connect(Unknown Source)
    at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1421)
    at com.alibaba.druid.pool.DruidAbstractDataSource.createPhysicalConnection(DruidAbstractDataSource.java:1477)
    at com.alibaba.druid.pool.DruidDataSource.init(DruidDataSource.java:677)

View Code

　　经过上述异常信息可知，kerberos认证未经过。因为是window环境，不属于hadoop安全范围，因此认证失败，为了认证经过，须要以下的代码：安全

@Test
    /**
     * 链接Impala查询
     */
    public void testImpala() throws SQLException, IOException {
        Configuration conf = new Configuration();
        conf.set("hadoop.security.authentication", "Kerberos");
        UserGroupInformation.setConfiguration(conf);
        UserGroupInformation.loginUserFromKeytab(user, keyTabPath);
        UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
        String query = "select * from table limit 1";
        loginUser.doAs((PrivilegedAction<Void>) () -> {
            try {
                try (Connection connection = DriverManager.getConnection(impalaUrl)) {
                    try (Statement statement = connection.createStatement()) {
                        ResultSet resultSet = statement.executeQuery(query);
                        while (resultSet.next()) {
                            System.out.println(resultSet.getObject(1));
                        }
                    }
                }
            } catch (SQLException e) {
                e.printStackTrace();
            }
            return null;
        });
    }

View Code

首先，设置hadoop.security.authentication为kerberos，其次，使用UserGroupInformation来进行操做。这样虽然kerberos认证是经过了，可是仍是有其余异常：服务器

java.lang.IllegalArgumentException: Can't get Kerberos realm
	....
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:84)
	at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63)
	... 30 more
Caused by: KrbException: Cannot locate default realm
	at sun.security.krb5.Config.getDefaultRealm(Config.java:1029)
	... 36 more

　　没法获取Kerberos realm，具体出现这个问题的地方在sun.security.krb5.Config文件中，阅读该Config类的构造函数，会发现以下代码：ide

Config(){
.....
 try {
                String var3 = this.getJavaFileName();
                List var2;
                if(var3 != null) {
                    var2 = this.loadConfigFile(var3);
                    this.stanzaTable = this.parseStanzaTable(var2);
                    if(DEBUG) {
                        System.out.println("Loaded from Java config");
                    }
                } else {
                    boolean var4 = false;
                    if(isMacosLionOrBetter()) {
                        try {
                            this.stanzaTable = SCDynamicStoreConfig.getConfig();
                            if(DEBUG) {
                                System.out.println("Loaded from SCDynamicStoreConfig");
                            }

                            var4 = true;
                        } catch (IOException var6) {
                            ;
                        }
                    }

                    if(!var4) {
                        var3 = this.getNativeFileName();
                        var2 = this.loadConfigFile(var3);
                        this.stanzaTable = this.parseStanzaTable(var2);
                        if(DEBUG) {
                            System.out.println("Loaded from native config");
                        }
                    }
                }
            } catch (IOException var7) {
                ;
            }
}

private String getJavaFileName() {
        String var1 = getProperty("java.security.krb5.conf");
        if(var1 == null) {
            var1 = getProperty("java.home") + File.separator + "lib" + File.separator + "security" + File.separator + "krb5.conf";
            if(!this.fileExists(var1)) {
                var1 = null;
            }
        }

        if(DEBUG) {
            System.out.println("Java config name: " + var1);
        }

        return var1;
    }

 private String getNativeFileName() {
        String var1 = null;
        String var2 = getProperty("os.name");
        if(var2.startsWith("Windows")) {
            try {
                Credentials.ensureLoaded();
            } catch (Exception var4) {
                ;
            }

            if(Credentials.alreadyLoaded) {
                String var3 = getWindowsDirectory(false);
                if(var3 != null) {
                    if(var3.endsWith("\\")) {
                        var3 = var3 + "krb5.ini";
                    } else {
                        var3 = var3 + "\\krb5.ini";
                    }

                    if(this.fileExists(var3)) {
                        var1 = var3;
                    }
                }

                if(var1 == null) {
                    var3 = getWindowsDirectory(true);
                    if(var3 != null) {
                        if(var3.endsWith("\\")) {
                            var3 = var3 + "krb5.ini";
                        } else {
                            var3 = var3 + "\\krb5.ini";
                        }

                        var1 = var3;
                    }
                }
            }

            if(var1 == null) {
                var1 = "c:\\winnt\\krb5.ini";
            }
        } else if(var2.startsWith("SunOS")) {
            var1 = "/etc/krb5/krb5.conf";
        } else if(var2.contains("OS X")) {
            var1 = this.findMacosConfigFile();
        } else {
            var1 = "/etc/krb5.conf";
        }

        if(DEBUG) {
            System.out.println("Native config name: " + var1);
        }

        return var1;
    }

View Code

　　其中重要的就是getJavaFileName和getNativeFileName两个方法，getJavaFileName中会查找java.security.krb5.conf属性，而getNativeFileName会查找系统目录(对于window就是C:\Windows目录)。函数

因此，在window环境下，能够在系统属性中配置上java.security.krb5.conf属性来指定krb5.ini文件，或将krb5.conf 保存到%JAVA_HOME%\lib\security\目录下，或将krb5.ini文件保存到c:\window目录下便可正常访问kerberos安全认证的Impala。oop

　　程序调试kerberos异常信息，启动的时候设置“-Dsun.security.krb5.debug=true” ，再次启动就会输出kerberos的详细信息，根据详细信息进行排查。 测试

截图的异常信息是开启了kerberos的调试信息后打印的信息。其中“Clock skew too great”说明系统时间与kerberos系统的时间差距超出了5分钟。

测试环境：

客户端操做系统： window 7

JDK：1.8

Impala-jdbc：

 <dependency>
       <groupId>com.cloudera.impala</groupId>
       <artifactId>impala-jdbc</artifactId>
       <version>2.5.34-cdh5.7.2</version>
 </dependency>

服务器环境：

操做系统：Centos 6.5

CDH：5.7.2