笔者好久以前就有个想法:参考现有的主流ORM
框架的设计,造一个ORM
轮子,在基本不改变使用体验的前提下把框架依赖的大量的反射设计去掉,这些反射API
构筑的组件使用动态编译加载的实例去替代,从而能够获得接近于直接使用原生JDBC
的性能。因而带着这样的想法,深刻学习Java
的动态编译。编写本文的时候使用的是JDK11
。前端
下面这个很眼熟的图来源于《深刻理解Java虚拟机》前端编译与优化的章节,主要描述编译的过程:java
上图看起来只有三步,其实每一步都有大量的步骤,下图尝试相对详细地描述具体的步骤(图比较大难以分割,直接放原图):mysql
实际上,仅仅对于编译这个过程来讲,开发者或者使用者没必要要彻底掌握其中的细节,JDK
提供了一个工具包javax.tools
让使用者能够用简易的API
进行编译(其实在大多数请下,开发者是面向业务功能开发,像编译和打包这些细节通常直接由开发工具、Maven
、Gradle
等工具完成):sql
具体的使用过程包括:shell
javax.tools.JavaCompiler
实例。Java
文件对象初始化一个编译任务javax.tools.JavaCompiler$CompilationTask
实例。CompilationTask
实例执行结果表明着编译过程的成功与否。咱们熟知的
javac
编译器其实就是JavaCompiler
接口的实现,在JDK11
中,对应的实现类为com.sun.tools.javac.api.JavacTool
。在JDK8
中不存在JavaCompiler
接口,具体的编译入口类为com.sun.tools.javac.main.JavaCompiler
。编程
由于JVM
里面的Class
是基于ClassLoader
隔离的,因此编译成功以后能够经过自定义的类加载器加载对应的类实例,而后就能够应用反射API
进行实例化和后续的调用。api
JDK
动态编译的步骤在上一节已经清楚地说明,这里造一个简单的场景。假设存在一个接口以下:数组
package club.throwable.compile; public interface HelloService { void sayHello(String name); } // 默认实现 package club.throwable.compile; public class DefaultHelloService implements HelloService { @Override public void sayHello(String name) { System.out.println(String.format("%s say hello [by default]", name)); } }
咱们能够经过字符串SOURCE_CODE
定义一个类:缓存
static String SOURCE_CODE = "package club.throwable.compile;\n" + "\n" + "public class JdkDynamicCompileHelloService implements HelloService{\n" + "\n" + " @Override\n" + " public void sayHello(String name) {\n" + " System.out.println(String.format(\"%s say hello [by jdk dynamic compile]\", name));\n" + " }\n" + "}"; // 这里不须要定义类文件,还原类文件内容以下 package club.throwable.compile; public class JdkDynamicCompileHelloService implements HelloService{ @Override public void sayHello(String name) { System.out.println(String.format("%s say hello [by jdk dynamic compile]", name)); } }
在组装编译任务实例以前,还有几项工做须要完成:session
JavaFileObject
标准实现SimpleJavaFileObject
是面向类源码文件,因为动态编译时候输入的是类源码文件的内容字符串,须要自行实现JavaFileObject
。JavaFileManager
是面向类路径下的Java
源码文件进行加载,这里也须要自行实现JavaFileManager
。ClassLoader
实例去加载编译出来的动态类。自行实现一个JavaFileObject
,其实能够简单点直接继承SimpleJavaFileObject
,覆盖须要用到的方法便可:
public class CharSequenceJavaFileObject extends SimpleJavaFileObject { public static final String CLASS_EXTENSION = ".class"; public static final String JAVA_EXTENSION = ".java"; private static URI fromClassName(String className) { try { return new URI(className); } catch (URISyntaxException e) { throw new IllegalArgumentException(className, e); } } private ByteArrayOutputStream byteCode; private final CharSequence sourceCode; public CharSequenceJavaFileObject(String className, CharSequence sourceCode) { super(fromClassName(className + JAVA_EXTENSION), Kind.SOURCE); this.sourceCode = sourceCode; } public CharSequenceJavaFileObject(String fullClassName, Kind kind) { super(fromClassName(fullClassName), kind); this.sourceCode = null; } public CharSequenceJavaFileObject(URI uri, Kind kind) { super(uri, kind); this.sourceCode = null; } @Override public CharSequence getCharContent(boolean ignoreEncodingErrors) throws IOException { return sourceCode; } @Override public InputStream openInputStream() { return new ByteArrayInputStream(getByteCode()); } // 注意这个方法是编译结果回调的OutputStream,回调成功后就能经过下面的getByteCode()方法获取目标类编译后的字节码字节数组 @Override public OutputStream openOutputStream() { return byteCode = new ByteArrayOutputStream(); } public byte[] getByteCode() { return byteCode.toByteArray(); } }
若是编译成功以后,直接经过自行添加的CharSequenceJavaFileObject#getByteCode()
方法便可获取目标类编译后的字节码对应的字节数组(二进制内容)。这里的CharSequenceJavaFileObject
预留了多个构造函数用于兼容原有的编译方式。
只要简单继承ClassLoader
便可,关键是要覆盖原来的ClassLoader#findClass()
方法,用于搜索自定义的JavaFileObject
实例,从而提取对应的字节码字节数组进行装载,为了实现这一点能够添加一个哈希表做为缓存,键-值分别是全类名的别名(xx.yy.MyClass
形式,而非URI
模式)和目标类对应的JavaFileObject
实例。
public class JdkDynamicCompileClassLoader extends ClassLoader { public static final String CLASS_EXTENSION = ".class"; private final Map<String, JavaFileObject> javaFileObjectMap = Maps.newConcurrentMap(); public JdkDynamicCompileClassLoader(ClassLoader parentClassLoader) { super(parentClassLoader); } @Override protected Class<?> findClass(String name) throws ClassNotFoundException { JavaFileObject javaFileObject = javaFileObjectMap.get(name); if (null != javaFileObject) { CharSequenceJavaFileObject charSequenceJavaFileObject = (CharSequenceJavaFileObject) javaFileObject; byte[] byteCode = charSequenceJavaFileObject.getByteCode(); return defineClass(name, byteCode, 0, byteCode.length); } return super.findClass(name); } @Nullable @Override public InputStream getResourceAsStream(String name) { if (name.endsWith(CLASS_EXTENSION)) { String qualifiedClassName = name.substring(0, name.length() - CLASS_EXTENSION.length()).replace('/', '.'); CharSequenceJavaFileObject javaFileObject = (CharSequenceJavaFileObject) javaFileObjectMap.get(qualifiedClassName); if (null != javaFileObject && null != javaFileObject.getByteCode()) { return new ByteArrayInputStream(javaFileObject.getByteCode()); } } return super.getResourceAsStream(name); } /** * 暂时存放编译的源文件对象,key为全类名的别名(非URI模式),如club.throwable.compile.HelloService */ void addJavaFileObject(String qualifiedClassName, JavaFileObject javaFileObject) { javaFileObjectMap.put(qualifiedClassName, javaFileObject); } Collection<JavaFileObject> listJavaFileObject() { return Collections.unmodifiableCollection(javaFileObjectMap.values()); } }
JavaFileManager
是Java
文件的抽象管理器,它用于管理常规的Java
文件,可是不局限于文件,也能够管理其余来源的Java
类文件数据。下面就经过实现一个自定义的JavaFileManager
用于管理字符串类型的源代码。为了简单起见,能够直接继承已经存在的ForwardingJavaFileManager
:
public class JdkDynamicCompileJavaFileManager extends ForwardingJavaFileManager<JavaFileManager> { private final JdkDynamicCompileClassLoader classLoader; private final Map<URI, JavaFileObject> javaFileObjectMap = Maps.newConcurrentMap(); public JdkDynamicCompileJavaFileManager(JavaFileManager fileManager, JdkDynamicCompileClassLoader classLoader) { super(fileManager); this.classLoader = classLoader; } private static URI fromLocation(Location location, String packageName, String relativeName) { try { return new URI(location.getName() + '/' + packageName + '/' + relativeName); } catch (URISyntaxException e) { throw new IllegalArgumentException(e); } } @Override public FileObject getFileForInput(Location location, String packageName, String relativeName) throws IOException { JavaFileObject javaFileObject = javaFileObjectMap.get(fromLocation(location, packageName, relativeName)); if (null != javaFileObject) { return javaFileObject; } return super.getFileForInput(location, packageName, relativeName); } /** * 这里是编译器返回的同(源)Java文件对象,替换为CharSequenceJavaFileObject实现 */ @Override public JavaFileObject getJavaFileForOutput(Location location, String className, JavaFileObject.Kind kind, FileObject sibling) throws IOException { JavaFileObject javaFileObject = new CharSequenceJavaFileObject(className, kind); classLoader.addJavaFileObject(className, javaFileObject); return javaFileObject; } /** * 这里覆盖原来的类加载器 */ @Override public ClassLoader getClassLoader(Location location) { return classLoader; } @Override public String inferBinaryName(Location location, JavaFileObject file) { if (file instanceof CharSequenceJavaFileObject) { return file.getName(); } return super.inferBinaryName(location, file); } @Override public Iterable<JavaFileObject> list(Location location, String packageName, Set<JavaFileObject.Kind> kinds, boolean recurse) throws IOException { Iterable<JavaFileObject> superResult = super.list(location, packageName, kinds, recurse); List<JavaFileObject> result = Lists.newArrayList(); // 这里要区分编译的Location以及编译的Kind if (location == StandardLocation.CLASS_PATH && kinds.contains(JavaFileObject.Kind.CLASS)) { // .class文件以及classPath下 for (JavaFileObject file : javaFileObjectMap.values()) { if (file.getKind() == JavaFileObject.Kind.CLASS && file.getName().startsWith(packageName)) { result.add(file); } } // 这里须要额外添加类加载器加载的全部Java文件对象 result.addAll(classLoader.listJavaFileObject()); } else if (location == StandardLocation.SOURCE_PATH && kinds.contains(JavaFileObject.Kind.SOURCE)) { // .java文件以及编译路径下 for (JavaFileObject file : javaFileObjectMap.values()) { if (file.getKind() == JavaFileObject.Kind.SOURCE && file.getName().startsWith(packageName)) { result.add(file); } } } for (JavaFileObject javaFileObject : superResult) { result.add(javaFileObject); } return result; } /** * 自定义方法,用于添加和缓存待编译的源文件对象 */ public void addJavaFileObject(Location location, String packageName, String relativeName, JavaFileObject javaFileObject) { javaFileObjectMap.put(fromLocation(location, packageName, relativeName), javaFileObject); } }
注意在这个类中引入了自定义类加载器JdkDynamicCompileClassLoader
,目的是为了实现JavaFileObject
实例的共享以及为文件管理器提供类加载器实例。
前置准备工做完成,咱们能够经过JavaCompiler
去编译这个前面提到的字符串,为了字节码的兼容性更好,编译的时候能够指定稍低的JDK
版本例如1.6
:
public class Client { static String SOURCE_CODE = "package club.throwable.compile;\n" + "\n" + "public class JdkDynamicCompileHelloService implements HelloService{\n" + "\n" + " @Override\n" + " public void sayHello(String name) {\n" + " System.out.println(String.format(\"%s say hello [by jdk dynamic compile]\", name));\n" + " }\n" + "}"; /** * 编译诊断收集器 */ static DiagnosticCollector<JavaFileObject> DIAGNOSTIC_COLLECTOR = new DiagnosticCollector<>(); public static void main(String[] args) throws Exception { // 获取系统编译器实例 JavaCompiler compiler = ToolProvider.getSystemJavaCompiler(); // 设置编译参数 - 指定编译版本为JDK1.6以提升兼容性 List<String> options = new ArrayList<>(); options.add("-source"); options.add("1.6"); options.add("-target"); options.add("1.6"); // 获取标准的Java文件管理器实例 StandardJavaFileManager manager = compiler.getStandardFileManager(DIAGNOSTIC_COLLECTOR, null, null); // 初始化自定义类加载器 JdkDynamicCompileClassLoader classLoader = new JdkDynamicCompileClassLoader(Thread.currentThread().getContextClassLoader()); // 初始化自定义Java文件管理器实例 JdkDynamicCompileJavaFileManager fileManager = new JdkDynamicCompileJavaFileManager(manager, classLoader); String packageName = "club.throwable.compile"; String className = "JdkDynamicCompileHelloService"; String qualifiedName = packageName + "." + className; // 构建Java源文件实例 CharSequenceJavaFileObject javaFileObject = new CharSequenceJavaFileObject(className, SOURCE_CODE); // 添加Java源文件实例到自定义Java文件管理器实例中 fileManager.addJavaFileObject( StandardLocation.SOURCE_PATH, packageName, className + CharSequenceJavaFileObject.JAVA_EXTENSION, javaFileObject ); // 初始化一个编译任务实例 JavaCompiler.CompilationTask compilationTask = compiler.getTask( null, fileManager, DIAGNOSTIC_COLLECTOR, options, null, Lists.newArrayList(javaFileObject) ); // 执行编译任务 Boolean result = compilationTask.call(); System.out.println(String.format("编译[%s]结果:%s", qualifiedName, result)); Class<?> klass = classLoader.loadClass(qualifiedName); HelloService instance = (HelloService) klass.getDeclaredConstructor().newInstance(); instance.sayHello("throwable"); } }
输出结果以下:
编译[club.throwable.compile.JdkDynamicCompileHelloService]结果:true throwable say hello [by jdk dynamic compile]
可见经过了字符串的类源码,实现了动态编译、类加载、反射实例化以及最终的方法调用。另外,编译过程的诊断信息能够经过DiagnosticCollector
实例获取。为了复用,这里能够把JDK
动态编译的过程抽取到一个方法中:
public final class JdkCompiler { static DiagnosticCollector<JavaFileObject> DIAGNOSTIC_COLLECTOR = new DiagnosticCollector<>(); @SuppressWarnings("unchecked") public static <T> T compile(String packageName, String className, String sourceCode, Class<?>[] constructorParamTypes, Object[] constructorParams) throws Exception { // 获取系统编译器实例 JavaCompiler compiler = ToolProvider.getSystemJavaCompiler(); // 设置编译参数 List<String> options = new ArrayList<>(); options.add("-source"); options.add("1.6"); options.add("-target"); options.add("1.6"); // 获取标准的Java文件管理器实例 StandardJavaFileManager manager = compiler.getStandardFileManager(DIAGNOSTIC_COLLECTOR, null, null); // 初始化自定义类加载器 JdkDynamicCompileClassLoader classLoader = new JdkDynamicCompileClassLoader(Thread.currentThread().getContextClassLoader()); // 初始化自定义Java文件管理器实例 JdkDynamicCompileJavaFileManager fileManager = new JdkDynamicCompileJavaFileManager(manager, classLoader); String qualifiedName = packageName + "." + className; // 构建Java源文件实例 CharSequenceJavaFileObject javaFileObject = new CharSequenceJavaFileObject(className, sourceCode); // 添加Java源文件实例到自定义Java文件管理器实例中 fileManager.addJavaFileObject( StandardLocation.SOURCE_PATH, packageName, className + CharSequenceJavaFileObject.JAVA_EXTENSION, javaFileObject ); // 初始化一个编译任务实例 JavaCompiler.CompilationTask compilationTask = compiler.getTask( null, fileManager, DIAGNOSTIC_COLLECTOR, options, null, Lists.newArrayList(javaFileObject) ); Boolean result = compilationTask.call(); System.out.println(String.format("编译[%s]结果:%s", qualifiedName, result)); Class<?> klass = classLoader.loadClass(qualifiedName); return (T) klass.getDeclaredConstructor(constructorParamTypes).newInstance(constructorParams); } }
既然有JDK
的动态编译,为何还存在Javassist
这样的字节码加强工具?撇开性能或者效率层面,JDK
动态编译存在比较大的局限性,比较明显的一点就是没法完成字节码插桩,换言之就是没法基于原有的类和方法进行修饰或者加强,可是Javassist
能够作到。再者,Javassist
提供的API
和JDK
反射的API
十分相近,若是反射平时用得比较熟练,Javassist
的上手也就变得比较简单。这里仅仅列举一个加强前面提到的DefaultHelloService
的例子,先引入依赖:
<dependency> <groupId>org.javassist</groupId> <artifactId>javassist</artifactId> <version>3.27.0-GA</version> </dependency>
编码以下:
public class JavassistClient { public static void main(String[] args) throws Exception { ClassPool pool = ClassPool.getDefault(); CtClass cc = pool.get("club.throwable.compile.DefaultHelloService"); CtMethod ctMethod = cc.getDeclaredMethod("sayHello", new CtClass[]{pool.get("java.lang.String")}); ctMethod.insertBefore("System.out.println(\"insert before by Javassist\");"); ctMethod.insertAfter("System.out.println(\"insert after by Javassist\");"); Class<?> klass = cc.toClass(); System.out.println(klass.getName()); HelloService helloService = (HelloService) klass.getDeclaredConstructor().newInstance(); helloService.sayHello("throwable"); } }
输出结果以下:
club.throwable.compile.DefaultHelloService insert before by Javassist throwable say hello [by default] insert after by Javassist
Javaassist
这个单词实际上是Java
和Assist
两个单词拼接在一块儿,意为Java
助手,是一个Java
字节码加强类库:
不像ASM
(ASM
的学习曲线比较陡峭,属于相对底层的字节码操做类库,固然从性能上来看ASM
对字节码加强的效率远高于其余高层次封装的框架)那样须要对字节码编程十分了解,Javaassist
下降了字节码加强功能的入门难度。
如今定义一个接口MysqlInfoMapper
,用于动态执行一条已知的SQL
,很简单,就是查询MySQL
的系统表mysql
里面的用户信息SELECT Host,User FROM mysql.user
:
@Data public class MysqlUser { private String host; private String user; } public interface MysqlInfoMapper { List<MysqlUser> selectAllMysqlUsers(); }
假设如今只提供一个MySQL
的驱动包(mysql:mysql-connector-java:jar:8.0.20
),暂时不能依赖任何高层次的框架,要动态实现MysqlInfoMapper
接口,优先整理须要的组件:
MySQL
的链接。SQL
执行器用于执行查询SQL
。为了简单起见,笔者在定义这三个组件接口的时候顺便在接口中经过单例进行实现(部分配置彻底写死):
// 链接管理器 public interface ConnectionManager { String USER_NAME = "root"; String PASS_WORD = "root"; String URL = "jdbc:mysql://localhost:3306/mysql?useUnicode=true&characterEncoding=utf-8&serverTimezone=GMT%2B8&useSSL=false"; Connection newConnection() throws SQLException; void closeConnection(Connection connection); ConnectionManager X = new ConnectionManager() { @Override public Connection newConnection() throws SQLException { return DriverManager.getConnection(URL, USER_NAME, PASS_WORD); } @Override public void closeConnection(Connection connection) { try { connection.close(); } catch (Exception ignore) { } } }; } // 执行器 public interface SqlExecutor { ResultSet execute(Connection connection, String sql) throws SQLException; SqlExecutor X = new SqlExecutor() { @Override public ResultSet execute(Connection connection, String sql) throws SQLException { Statement statement = connection.createStatement(); statement.execute(sql); return statement.getResultSet(); } }; } // 结果处理器 public interface ResultHandler<T> { T handleResultSet(ResultSet resultSet) throws SQLException; ResultHandler<List<MysqlUser>> X = new ResultHandler<List<MysqlUser>>() { @Override public List<MysqlUser> handleResultSet(ResultSet resultSet) throws SQLException { try { List<MysqlUser> result = Lists.newArrayList(); while (resultSet.next()) { MysqlUser item = new MysqlUser(); item.setHost(resultSet.getString("Host")); item.setUser(resultSet.getString("User")); result.add(item); } return result; } finally { resultSet.close(); } } }; }
接着须要动态编译MysqlInfoMapper
的实现类,它的源文件的字符串内容以下(注意不要在类路径下新建这个DefaultMysqlInfoMapper
类):
package club.throwable.compile; import java.sql.Connection; import java.sql.ResultSet; import java.util.List; public class DefaultMysqlInfoMapper implements MysqlInfoMapper { private final ConnectionManager connectionManager; private final SqlExecutor sqlExecutor; private final ResultHandler resultHandler; private final String sql; public DefaultMysqlInfoMapper(ConnectionManager connectionManager, SqlExecutor sqlExecutor, ResultHandler resultHandler, String sql) { this.connectionManager = connectionManager; this.sqlExecutor = sqlExecutor; this.resultHandler = resultHandler; this.sql = sql; } @Override public List<MysqlUser> selectAllMysqlUsers() { try { Connection connection = connectionManager.newConnection(); try { ResultSet resultSet = sqlExecutor.execute(connection, sql); return (List<MysqlUser>) resultHandler.handleResultSet(resultSet); } finally { connectionManager.closeConnection(connection); } } catch (Exception e) { // 暂时忽略异常处理,统一封装为IllegalStateException throw new IllegalStateException(e); } } }
而后编写一个客户端进行动态编译和执行:
public class MysqlInfoClient { static String SOURCE_CODE = "package club.throwable.compile;\n" + "import java.sql.Connection;\n" + "import java.sql.ResultSet;\n" + "import java.util.List;\n" + "\n" + "public class DefaultMysqlInfoMapper implements MysqlInfoMapper {\n" + "\n" + " private final ConnectionManager connectionManager;\n" + " private final SqlExecutor sqlExecutor;\n" + " private final ResultHandler resultHandler;\n" + " private final String sql;\n" + "\n" + " public DefaultMysqlInfoMapper(ConnectionManager connectionManager,\n" + " SqlExecutor sqlExecutor,\n" + " ResultHandler resultHandler,\n" + " String sql) {\n" + " this.connectionManager = connectionManager;\n" + " this.sqlExecutor = sqlExecutor;\n" + " this.resultHandler = resultHandler;\n" + " this.sql = sql;\n" + " }\n" + "\n" + " @Override\n" + " public List<MysqlUser> selectAllMysqlUsers() {\n" + " try {\n" + " Connection connection = connectionManager.newConnection();\n" + " try {\n" + " ResultSet resultSet = sqlExecutor.execute(connection, sql);\n" + " return (List<MysqlUser>) resultHandler.handleResultSet(resultSet);\n" + " } finally {\n" + " connectionManager.closeConnection(connection);\n" + " }\n" + " } catch (Exception e) {\n" + " // 暂时忽略异常处理,统一封装为IllegalStateException\n" + " throw new IllegalStateException(e);\n" + " }\n" + " }\n" + "}\n"; static String SQL = "SELECT Host,User FROM mysql.user"; public static void main(String[] args) throws Exception { MysqlInfoMapper mysqlInfoMapper = JdkCompiler.compile( "club.throwable.compile", "DefaultMysqlInfoMapper", SOURCE_CODE, new Class[]{ConnectionManager.class, SqlExecutor.class, ResultHandler.class, String.class}, new Object[]{ConnectionManager.X, SqlExecutor.X, ResultHandler.X, SQL}); System.out.println(JSON.toJSONString(mysqlInfoMapper.selectAllMysqlUsers())); } }
最终的输出结果是:
编译[club.throwable.compile.DefaultMysqlInfoMapper]结果:true [{"host":"%","user":"canal"},{"host":"%","user":"doge"},{"host":"localhost","user":"mysql.infoschema"},{"host":"localhost","user":"mysql.session"},{"host":"localhost","user":"mysql.sys"},{"host":"localhost","user":"root"}]
而后笔者查看本地安装的MySQL
中的结果,验证该查询结果是正确的。
这里笔者为了简化整个例子,没有在MysqlInfoMapper#selectAllMysqlUsers()
方法中添加查询参数,能够尝试一下查询的SQL
是SELECT Host,User FROM mysql.user WHERE User = 'xxx'
场景下的编码实现。
若是把动态实现的
DefaultMysqlInfoMapper
注册到IOC
容器中,就能够实现MysqlInfoMapper
按照类型自动装配。
若是把SQL
和参数处理能够抽离到单独的文件中,而且实现一个对应的文件解析器,那么就能够把类文件和SQL
隔离,Mybatis
和Hibernate
都是这样作的。
动态编译或者更底层的面向字节码层面的编程,实际上是一个十分有挑战性可是能够创造无限可能的领域,本文只是简单分析了一下Java
源码编译的过程,而且经过一些简单的例子进行动态编译的模拟,离使用于实际应用中还有很多距离,后面须要花更多的时间去分析一下相关领域的知识。
参考资料:
JDK11
部分源码(本文完 c-4-d e-a-20200606 0:23)