转载请务必注明原创地址为:http://www.54tianzhisheng.cn/2018/08/11/es-code02/java
上篇文章写了 ElasticSearch 源码解析 —— 环境搭建 ,其中里面说了启动 打开 server 模块下的 Elasticsearch 类:org.elasticsearch.bootstrap.Elasticsearch,运行里面的 main 函数就能够启动 ElasticSearch 了,这篇文章讲讲启动流程,由于篇幅会不少,因此分了两篇来写。node
能够看到入口实际上是一个 main 方法,方法里面先是检查权限,而后是一个错误日志监听器(确保在日志配置以前状态日志没有出现 error),而后是 Elasticsearch 对象的建立,而后调用了静态方法 main 方法(18 行),并把建立的对象和参数以及 Terminal 默认值传进去。静态的 main 方法里面调用 elasticsearch.main 方法。json
public static void main(final String[] args) throws Exception { //一、入口 // we want the JVM to think there is a security manager installed so that if internal policy decisions that would be based on the // presence of a security manager or lack thereof act as if there is a security manager present (e.g., DNS cache policy) System.setSecurityManager(new SecurityManager() { @Override public void checkPermission(Permission perm) { // grant all permissions so that we can later set the security manager to the one that we want } }); LogConfigurator.registerErrorListener(); // final Elasticsearch elasticsearch = new Elasticsearch(); int status = main(args, elasticsearch, Terminal.DEFAULT); //二、调用Elasticsearch.main方法 if (status != ExitCodes.OK) { exit(status); } } static int main(final String[] args, final Elasticsearch elasticsearch, final Terminal terminal) throws Exception { return elasticsearch.main(args, terminal); //三、command main }
由于 Elasticsearch 类是继承了 EnvironmentAwareCommand 类,EnvironmentAwareCommand 类继承了 Command 类,可是 Elasticsearch 类并无重写 main 方法,因此上面调用的 elasticsearch.main 实际上是调用了 Command 的 main 方法,代码以下:bootstrap
/** Parses options for this command from args and executes it. */ public final int main(String[] args, Terminal terminal) throws Exception { if (addShutdownHook()) { //利用Runtime.getRuntime().addShutdownHook方法加入一个Hook,在程序退出时触发该Hook shutdownHookThread = new Thread(() -> { try { this.close(); } catch (final IOException e) { try ( StringWriter sw = new StringWriter(); PrintWriter pw = new PrintWriter(sw)) { e.printStackTrace(pw); terminal.println(sw.toString()); } catch (final IOException impossible) { // StringWriter#close declares a checked IOException from the Closeable interface but the Javadocs for StringWriter // say that an exception here is impossible throw new AssertionError(impossible); } } }); Runtime.getRuntime().addShutdownHook(shutdownHookThread); } beforeMain.run(); try { mainWithoutErrorHandling(args, terminal);//四、mainWithoutErrorHandling } catch (OptionException e) { printHelp(terminal); terminal.println(Terminal.Verbosity.SILENT, "ERROR: " + e.getMessage()); return ExitCodes.USAGE; } catch (UserException e) { if (e.exitCode == ExitCodes.USAGE) { printHelp(terminal); } terminal.println(Terminal.Verbosity.SILENT, "ERROR: " + e.getMessage()); return e.exitCode; } return ExitCodes.OK; }
上面代码一开始利用一个勾子函数,在程序退出时触发该 Hook,该方法主要代码是 mainWithoutErrorHandling() 方法,而后下面的是 catch 住方法抛出的异常,方法代码以下:安全
/*** Executes the command, but all errors are thrown. */ void mainWithoutErrorHandling(String[] args, Terminal terminal) throws Exception { final OptionSet options = parser.parse(args); if (options.has(helpOption)) { printHelp(terminal); return; } if (options.has(silentOption)) { terminal.setVerbosity(Terminal.Verbosity.SILENT); } else if (options.has(verboseOption)) { terminal.setVerbosity(Terminal.Verbosity.VERBOSE); } else { terminal.setVerbosity(Terminal.Verbosity.NORMAL); } execute(terminal, options);//五、执行 EnvironmentAwareCommand 中的 execute(),(重写了command里面抽象的execute方法) }
上面的代码从 3 ~ 14 行是解析传进来的参数并配置 terminal,重要的 execute() 方法,执行的是 EnvironmentAwareCommand 中的 execute() (重写了 Command 类里面的抽象 execute 方法),从上面那个继承图能够看到 EnvironmentAwareCommand 继承了 Command,重写的 execute 方法代码以下:app
@Override protected void execute(Terminal terminal, OptionSet options) throws Exception { final Map<String, String> settings = new HashMap<>(); for (final KeyValuePair kvp : settingOption.values(options)) { if (kvp.value.isEmpty()) { throw new UserException(ExitCodes.USAGE, "setting [" + kvp.key + "] must not be empty"); } if (settings.containsKey(kvp.key)) { final String message = String.format( Locale.ROOT, "setting [%s] already set, saw [%s] and [%s]", kvp.key, settings.get(kvp.key), kvp.value); throw new UserException(ExitCodes.USAGE, message); } settings.put(kvp.key, kvp.value); } //六、根据咱们ide配置的 vm options 进行设置path.data、path.home、path.logs putSystemPropertyIfSettingIsMissing(settings, "path.data", "es.path.data"); putSystemPropertyIfSettingIsMissing(settings, "path.home", "es.path.home"); putSystemPropertyIfSettingIsMissing(settings, "path.logs", "es.path.logs"); execute(terminal, options, createEnv(terminal, settings));//七、先调用 createEnv 建立环境 //九、执行elasticsearch的execute方法,elasticsearch中重写了EnvironmentAwareCommand中的抽象execute方法 }
方法前面是根据传参去判断配置的,若是配置为空,就会直接跳到执行 putSystemPropertyIfSettingIsMissing 方法,这里会配置三个属性:path.data、path.home、path.logs 设置 es 的 data、home、logs 目录,它这里是根据咱们 ide 配置的 vm options 进行设置的,这也是为何咱们上篇文章说的配置信息,若是不配置的话就会直接报错。下面看看 putSystemPropertyIfSettingIsMissing 方法代码里面怎么作到的:jvm
/** Ensure the given setting exists, reading it from system properties if not already set. */ private static void putSystemPropertyIfSettingIsMissing(final Map<String, String> settings, final String setting, final String key) { final String value = System.getProperty(key);//获取key(es.path.data)找系统设置 if (value != null) { if (settings.containsKey(setting)) { final String message = String.format( Locale.ROOT, "duplicate setting [%s] found via command-line [%s] and system property [%s]", setting, settings.get(setting), value); throw new IllegalArgumentException(message); } else { settings.put(setting, value); } } }
执行这三个方法后:elasticsearch
跳出此方法,继续看会发现 execute 方法调用了方法,ide
execute(terminal, options, createEnv(terminal, settings));
这里咱们先看看 createEnv(terminal, settings)
方法:函数
protected Environment createEnv(final Terminal terminal, final Map<String, String> settings) throws UserException { final String esPathConf = System.getProperty("es.path.conf");//八、读取咱们 vm options 中配置的 es.path.conf if (esPathConf == null) { throw new UserException(ExitCodes.CONFIG, "the system property [es.path.conf] must be set"); } return InternalSettingsPreparer.prepareEnvironment(Settings.EMPTY, terminal, settings, getConfigPath(esPathConf)); //八、准备环境 prepareEnvironment }
读取咱们 ide vm options 中配置的 es.path.conf,同上篇文章也讲了这个必定要配置的,由于 es 启动的时候会加载咱们的配置和一些插件。这里继续看下上面代码第 6 行的 prepareEnvironment 方法:
public static Environment prepareEnvironment(Settings input, Terminal terminal, Map<String, String> properties, Path configPath) { // just create enough settings to build the environment, to get the config dir Settings.Builder output = Settings.builder(); initializeSettings(output, input, properties); Environment environment = new Environment(output.build(), configPath); //查看 es.path.conf 目录下的配置文件是否是 yml 格式的,若是不是则抛出一个异常 if (Files.exists(environment.configFile().resolve("elasticsearch.yaml"))) { throw new SettingsException("elasticsearch.yaml was deprecated in 5.5.0 and must be renamed to elasticsearch.yml"); } if (Files.exists(environment.configFile().resolve("elasticsearch.json"))) { throw new SettingsException("elasticsearch.json was deprecated in 5.5.0 and must be converted to elasticsearch.yml"); } output = Settings.builder(); // start with a fresh output Path path = environment.configFile().resolve("elasticsearch.yml"); if (Files.exists(path)) { try { output.loadFromPath(path); //加载文件并读取配置文件内容 } catch (IOException e) { throw new SettingsException("Failed to load settings from " + path.toString(), e); } } // re-initialize settings now that the config file has been loaded initializeSettings(output, input, properties); //再一次初始化设置 finalizeSettings(output, terminal); environment = new Environment(output.build(), configPath); // we put back the path.logs so we can use it in the logging configuration file output.put(Environment.PATH_LOGS_SETTING.getKey(), environment.logsFile().toAbsolutePath().normalize().toString()); return new Environment(output.build(), configPath); }
准备的环境如上图,经过构建的环境查看配置文件 elasticsearch.yml 是否是以 yml 结尾,若是是 yaml 或者 json 结尾的则抛出异常(在 5.5.0 版本其余两种格式过时了,只能使用 yml 格式),而后加载该配置文件并读取里面的内容(KV结构)。
跳出 createEnv 方法,咱们继续看 execute 方法吧。
EnvironmentAwareCommand 类的 execute 方法代码以下:
protected abstract void execute(Terminal terminal, OptionSet options, Environment env) throws Exception;
这是个抽象方法,那么它的实现方法在 Elasticsearch 类中,代码以下:
@Override protected void execute(Terminal terminal, OptionSet options, Environment env) throws UserException { if (options.nonOptionArguments().isEmpty() == false) { throw new UserException(ExitCodes.USAGE, "Positional arguments not allowed, found " + options.nonOptionArguments()); } if (options.has(versionOption)) { final String versionOutput = String.format( Locale.ROOT, "Version: %s, Build: %s/%s/%s/%s, JVM: %s", Version.displayVersion(Version.CURRENT, Build.CURRENT.isSnapshot()), Build.CURRENT.flavor().displayName(), Build.CURRENT.type().displayName(), Build.CURRENT.shortHash(), Build.CURRENT.date(), JvmInfo.jvmInfo().version()); terminal.println(versionOutput); return; } final boolean daemonize = options.has(daemonizeOption); final Path pidFile = pidfileOption.value(options); final boolean quiet = options.has(quietOption); // a misconfigured java.io.tmpdir can cause hard-to-diagnose problems later, so reject it immediately try { env.validateTmpFile(); } catch (IOException e) { throw new UserException(ExitCodes.CONFIG, e.getMessage()); } try { init(daemonize, pidFile, quiet, env); //十、初始化 } catch (NodeValidationException e) { throw new UserException(ExitCodes.CONFIG, e.getMessage()); } }
上面代码里主要仍是看看 init(daemonize, pidFile, quiet, env);
初始化方法吧。
void init(final boolean daemonize, final Path pidFile, final boolean quiet, Environment initialEnv) throws NodeValidationException, UserException { try { Bootstrap.init(!daemonize, pidFile, quiet, initialEnv); //十一、执行 Bootstrap 中的 init 方法 } catch (BootstrapException | RuntimeException e) { // format exceptions to the console in a special way // to avoid 2MB stacktraces from guice, etc. throw new StartupException(e); } }
Bootstrap 中的静态 init 方法以下:
static void init( final boolean foreground, final Path pidFile, final boolean quiet, final Environment initialEnv) throws BootstrapException, NodeValidationException, UserException { // force the class initializer for BootstrapInfo to run before // the security manager is installed BootstrapInfo.init(); INSTANCE = new Bootstrap(); //十二、建立一个 Bootstrap 实例 final SecureSettings keystore = loadSecureSettings(initialEnv);//若是注册了安全模块则将相关配置加载进来 final Environment environment = createEnvironment(foreground, pidFile, keystore, initialEnv.settings(), initialEnv.configFile()); //干以前干过的事情 try { LogConfigurator.configure(environment); //1三、log 配置环境 } catch (IOException e) { throw new BootstrapException(e); } if (environment.pidFile() != null) { try { PidFile.create(environment.pidFile(), true); } catch (IOException e) { throw new BootstrapException(e); } } final boolean closeStandardStreams = (foreground == false) || quiet; try { if (closeStandardStreams) { final Logger rootLogger = ESLoggerFactory.getRootLogger(); final Appender maybeConsoleAppender = Loggers.findAppender(rootLogger, ConsoleAppender.class); if (maybeConsoleAppender != null) { Loggers.removeAppender(rootLogger, maybeConsoleAppender); } closeSystOut(); } // fail if somebody replaced the lucene jars checkLucene(); //1四、检查Lucene版本 // install the default uncaught exception handler; must be done before security is initialized as we do not want to grant the runtime permission setDefaultUncaughtExceptionHandler Thread.setDefaultUncaughtExceptionHandler( new ElasticsearchUncaughtExceptionHandler(() -> Node.NODE_NAME_SETTING.get(environment.settings()))); INSTANCE.setup(true, environment); //1五、调用 setup 方法 try { // any secure settings must be read during node construction IOUtils.close(keystore); } catch (IOException e) { throw new BootstrapException(e); } INSTANCE.start(); //2六、调用 start 方法 if (closeStandardStreams) { closeSysError(); } } catch (NodeValidationException | RuntimeException e) { // disable console logging, so user does not see the exception twice (jvm will show it already) final Logger rootLogger = ESLoggerFactory.getRootLogger(); final Appender maybeConsoleAppender = Loggers.findAppender(rootLogger, ConsoleAppender.class); if (foreground && maybeConsoleAppender != null) { Loggers.removeAppender(rootLogger, maybeConsoleAppender); } Logger logger = Loggers.getLogger(Bootstrap.class); if (INSTANCE.node != null) { logger = Loggers.getLogger(Bootstrap.class, Node.NODE_NAME_SETTING.get(INSTANCE.node.settings())); } // HACK, it sucks to do this, but we will run users out of disk space otherwise if (e instanceof CreationException) { // guice: log the shortened exc to the log file ByteArrayOutputStream os = new ByteArrayOutputStream(); PrintStream ps = null; try { ps = new PrintStream(os, false, "UTF-8"); } catch (UnsupportedEncodingException uee) { assert false; e.addSuppressed(uee); } new StartupException(e).printStackTrace(ps); ps.flush(); try { logger.error("Guice Exception: {}", os.toString("UTF-8")); } catch (UnsupportedEncodingException uee) { assert false; e.addSuppressed(uee); } } else if (e instanceof NodeValidationException) { logger.error("node validation exception\n{}", e.getMessage()); } else { // full exception logger.error("Exception", e); } // re-enable it if appropriate, so they can see any logging during the shutdown process if (foreground && maybeConsoleAppender != null) { Loggers.addAppender(rootLogger, maybeConsoleAppender); } throw e; } }
该方法主要有:
一、建立 Bootstrap 实例
二、若是注册了安全模块则将相关配置加载进来
三、建立 Elasticsearch 运行的必须环境以及相关配置, 如将 config、scripts、plugins、modules、logs、lib、bin 等配置目录加载到运行环境中
四、log 配置环境,建立日志上下文
五、检查是否存在 PID 文件,若是不存在,建立 PID 文件
六、检查 Lucene 版本
七、调用 setup 方法(用当前环境来建立一个节点)
private void setup(boolean addShutdownHook, Environment environment) throws BootstrapException { Settings settings = environment.settings();//根据环境获得配置 try { spawner.spawnNativeControllers(environment); } catch (IOException e) { throw new BootstrapException(e); } initializeNatives( environment.tmpFile(), BootstrapSettings.MEMORY_LOCK_SETTING.get(settings), BootstrapSettings.SYSTEM_CALL_FILTER_SETTING.get(settings), BootstrapSettings.CTRLHANDLER_SETTING.get(settings)); // initialize probes before the security manager is installed initializeProbes(); if (addShutdownHook) { Runtime.getRuntime().addShutdownHook(new Thread() { @Override public void run() { try { IOUtils.close(node, spawner); LoggerContext context = (LoggerContext) LogManager.getContext(false); Configurator.shutdown(context); } catch (IOException ex) { throw new ElasticsearchException("failed to stop node", ex); } } }); } try { // look for jar hell final Logger logger = ESLoggerFactory.getLogger(JarHell.class); JarHell.checkJarHell(logger::debug); } catch (IOException | URISyntaxException e) { throw new BootstrapException(e); } // Log ifconfig output before SecurityManager is installed IfConfig.logIfNecessary(); // install SM after natives, shutdown hooks, etc. try { Security.configure(environment, BootstrapSettings.SECURITY_FILTER_BAD_DEFAULTS_SETTING.get(settings)); } catch (IOException | NoSuchAlgorithmException e) { throw new BootstrapException(e); } node = new Node(environment) { //1六、新建节点 @Override protected void validateNodeBeforeAcceptingRequests( final BootstrapContext context, final BoundTransportAddress boundTransportAddress, List<BootstrapCheck> checks) throws NodeValidationException { BootstrapChecks.check(context, boundTransportAddress, checks); } }; }
上面代码最后就是 Node 节点的建立,这篇文章就不讲 Node 的建立了,下篇文章会好好讲一下 Node 节点的建立和正式启动 ES 节点的。
这篇文章主要先把大概启动流程串通,由于篇幅较多因此拆开成两篇,先不扣细节了,后面流程启动文章写完后咱们再单一的扣细节。
二、渣渣菜鸡的 ElasticSearch 源码解析 —— 环境搭建
三、渣渣菜鸡的 ElasticSearch 源码解析 —— 启动流程(上)
四、渣渣菜鸡的 ElasticSearch 源码解析 —— 启动流程(下)
五、Elasticsearch 系列文章(一):Elasticsearch 默认分词器和中分分词器之间的比较及使用方法
六、Elasticsearch 系列文章(二):全文搜索引擎 Elasticsearch 集群搭建入门教程
七、Elasticsearch 系列文章(三):ElasticSearch 集群监控
八、Elasticsearch 系列文章(四):ElasticSearch 单个节点监控