elasticsearch-6.2.4集群安装以及开启集群X-pack密码认证

时间  2021-04-16
标签 html java node linux git github json bootstrap centos api 栏目 日志分析 繁體版
原文   原文链接

1、rpm包方式安装elasticsearch

环境是：html

[root@sdk-25 run]# cat /etc/redhat-release 
CentOS Linux release 7.6.1810 (Core)
关闭selinx,

下面是安装和启动命令
[root@local-216 soft]# rpm -ivh elasticsearch-6.2.4.rpm 
warning: elasticsearch-6.2.4.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY
Preparing...                          ################################# [100%]
Creating elasticsearch group... OK
Creating elasticsearch user... OK
Updating / installing...
   1:elasticsearch-0:6.2.4-1          ################################# [100%]

下面是 systemctl启动es命令:java

NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service

2、采用二进制包elasticsearch-6.2.4.tar.gz 直接解压方式安装

提早对elasticsearch服务器的系统进行下面的参数优化，这样在安装完es启动过程当中能够避免好多报错node

ES服务器系统环境优化:linux

最少使用swap内存交换分区，
 关于优化，能够参考
 https://www.jianshu.com/p/7c163d7e9ecb

[root@sdk-25 ~]# tail -2 /etc/sysctl.conf 
vm.swappiness=1  ##禁止用交换内存
vm.max_map_count=262144   ##设置虚拟内存

 [root@sdk-25 ~]# cat /etc/security/limits.conf 
##文件句柄数
* soft nofile 131072
* hard nofile 131072

##进程线程数
* soft nproc 131072
* hard nproc 131072

##内存锁定交换
* soft memlock unlimited
* hard memlock unlimited

安装系统环境说明:git

[root@sdk-25 run]# cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)github

具体安装步骤以下：json

提早安装好jdk1.8环境：
[root@sdk-25 config]# source /etc/profile
export JAVA_HOME=/usr/local/jdk
export PATH=$JAVA_HOME/bin:$PATHbootstrap

centos7.6.单实例二进制包安装ES:
下载elasticsearch-6.2.4.tar.gz 二进制安装包：
wget -P /data/soft https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.tar.gzcentos

下面准备的是单台机器启动三个elasticsearch实例：api

cd /data/soft;tar xf elasticsearch-6.2.4.tar.gz -C /usr/local/;cp -rp elasticsearch-6.2.4 elasticsearch01;cp -rp elasticsearch-6.2.4 elasticsearch02;cp -rp elasticsearch-6.2.4 elasticsearch03;
useradd  elasticsearch;passwd elasticsearch 
 mkdir /data/elasticsearch{01,02,03}/{data,logs,run} -p
 cd /data
 chown -R elasticsearch.elasticsearch elasticsearch0*

2.一、启动第一个单实例elasticsearch01

单实例elasticsearch01的配置文件以下:

[root@sdk-25 config]# cat /usr/local/elasticsearch01/config/elasticsearch.yml
node.name: node25
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
path.data: /data/elasticsearch01/data
path.logs: /data/elasticsearch01/logs
#
bootstrap.memory_lock: true
#
network.host: 127.0.0.1
#network.host: 192.168.1.25
#
http.port: 9200
transport.tcp.port: 9300
##下面是es7版本的参数
#discovery.seed_hosts: ["192.168.1.25:9300"]
#cluster.initial_master_nodes: ["192.168.1.25:9300"]

提示：以前在centos7.6 机器上是rpm包方式安装的elasticsearch，因此会有systemctl启动脚本,或者service elasticsearch01 start/status/restart/ 

[root@sdk-25 ~]# cp /usr/lib/systemd/system/elasticsearch.service /usr/lib/systemd/system/elasticsearch01.service
[root@sdk-25 ~]# systemctl enable elasticsearch01.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch01.service to /usr/lib/systemd/system/elasticsearch01.service.

elasticsearch01启动脚本配置文件elasticsearch 配置文件以下:

[root@sdk-25 run]# cat /usr/lib/systemd/system/elasticsearch01.service
[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target

[Service]
RuntimeDirectory=elasticsearch
Environment=ES_HOME=/usr/local/elasticsearch01
Environment=ES_PATH_CONF=/usr/local/elasticsearch01/config
Environment=PID_DIR=/data/elasticsearch01/run
EnvironmentFile=-/etc/sysconfig/elasticsearch01

WorkingDirectory=/usr/local/elasticsearch01

User=elasticsearch
Group=elasticsearch

ExecStart=/usr/local/elasticsearch01/bin/elasticsearch  -p ${PID_DIR}/elasticsearch.pid --quiet

# StandardOutput is configured to redirect to journalctl since
# some error messages may be logged in standard output before
# elasticsearch logging system is initialized. Elasticsearch
# stores its logs in /var/log/elasticsearch and does not use
# journalctl by default. If you also want to enable journalctl
# logging, you can simply remove the "quiet" option from ExecStart.
StandardOutput=journal
StandardError=inherit

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536

# Specifies the maximum number of processes
LimitNPROC=4096

# Specifies the maximum size of virtual memory
LimitAS=infinity

# Specifies the maximum file size
LimitFSIZE=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=0

# SIGTERM signal is used to stop the Java process
KillSignal=SIGTERM

# Send the signal only to the JVM rather than its control group
KillMode=process

# Java process is never killed
SendSIGKILL=no

# When a JVM receives a SIGTERM signal it exits with code 143
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

# Built for distribution-6.2.4 (distribution)

elasticsearch01环境变量 配置文件以下:

[root@sdk-25 local]# cp /etc/sysconfig/elasticsearch /etc/sysconfig/elasticsearch01
 [root@sdk-25 run]# cat /etc/sysconfig/elasticsearch01
################################
#Elasticsearch
################################
#Elasticsearch home directory
ES_HOME=/usr/local/elasticsearch01
#Elasticsearch Java path
JAVA_HOME=/usr/local/jdk
 Elasticsearch configuration directory
ES_PATH_CONF=/usr/local/elasticsearch01/config
#Elasticsearch PID directory
PID_DIR=/data/elasticsearch01/run
#Additional Java OPTS
#ES_JAVA_OPTS=

#Configure restart on package upgrade (true, every other setting will lead to not restarting)
#RESTART_ON_UPGRADE=true

################################
#Elasticsearch service
################################
#SysV init.d
#The number of seconds to wait before checking if Elasticsearch started successfully as a daemon process
ES_STARTUP_SLEEP_TIME=5

################################
#System properties
################################
#Specifies the maximum file descriptor number that can be opened by this process
#When using Systemd, this setting is ignored and the LimitNOFILE defined in
#/usr/lib/systemd/system/elasticsearch.service takes precedence
#MAX_OPEN_FILES=65536
#The maximum number of bytes of memory that may be locked into RAM
#Set to "unlimited" if you use the 'bootstrap.memory_lock: true' option
#in elasticsearch.yml.
#When using systemd, LimitMEMLOCK must be set in a unit file such as
#/etc/systemd/system/elasticsearch.service.d/override.conf.
#MAX_LOCKED_MEMORY=unlimited

#Maximum number of VMA (Virtual Memory Areas) a process can own
#When using Systemd, this setting is ignored and the 'vm.max_map_count'
#property is set at boot time in /usr/lib/sysctl.d/elasticsearch.conf
#MAX_MAP_COUNT=262144
[root@sdk-25 run]# systemctl status elasticsearch.service
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: active (running) since 五 2020-07-24 23:44:12 CST; 3min 32s ago
     Docs: http://www.elastic.co
 Main PID: 18141 (java)
   CGroup: /system.slice/elasticsearch.service
           └─18141 /usr/local/jdk/bin/java -Xms4g -Xmx4g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava....

7月 24 23:44:12 sdk-25 systemd[1]: Started Elasticsearch.

[root@sdk-25 local]# ss -lntup|grep java
tcp    LISTEN     0      128    ::ffff:127.0.0.1:9200                 :::*                   users:(("java",pid=7245,fd=750))
tcp    LISTEN     0      128    ::ffff:127.0.0.1:9300                 :::*                   users:(("java",pid=7245,fd=556))
[root@sdk-25 local]# service elasticsearch01 stop
Redirecting to /bin/systemctl stop elasticsearch01.service

[root@sdk-25 local]# service elasticsearch01 start
Redirecting to /bin/systemctl start elasticsearch01.service

[root@sdk-25 local]# ss -lntup|grep java
tcp    LISTEN     0      128    ::ffff:127.0.0.1:9200                 :::*                   users:(("java",pid=8591,fd=750))
tcp    LISTEN     0      128    ::ffff:127.0.0.1:9300                 :::*                   users:(("java",pid=8591,fd=556))

[root@sdk-25 local]# service elasticsearch01 status
Redirecting to /bin/systemctl status elasticsearch01.service
● elasticsearch01.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch01.service; enabled; vendor preset: disabled)
   Active: active (running) since 日 2020-07-26 12:03:03 CST; 54s ago
     Docs: http://www.elastic.co
 Main PID: 7245 (java)
   CGroup: /system.slice/elasticsearch01.service
           ├─7245 /usr/local/jdk/bin/java -Xms4g -Xmx4g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.a...
           └─7409 /usr/local/elasticsearch01/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller

7月 26 12:03:03 sdk-25 systemd[1]: Started Elasticsearch.

2.二、启动第二个单实例elasticsearch02

操做过程同第一个单实例elasticsearch01同样

单实例elasticsearch02配置文件内容以下:

[root@sdk-25 local]# cat /usr/local/elasticsearch02/config/elasticsearch.yml
node.name: node25-1

#Add custom attributes to the node:
#node.attr.rack: r1
path.data: /data/elasticsearch02/data
path.logs: /data/elasticsearch02/logs
bootstrap.memory_lock: true
network.host: 127.0.0.1
#network.host: 192.168.1.25
http.port: 9201
transport.tcp.port: 9301
##下面是es7版本的参数
#discovery.seed_hosts: ["192.168.1.25:9300"]
#cluster.initial_master_nodes: ["192.168.1.25:9300"]

准备systemctl启动的配置文件和启动加载的环境变量文件：

[root@sdk-25 local]# cat /usr/lib/systemd/system/elasticsearch02.service
[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target

[Service]
RuntimeDirectory=elasticsearch
Environment=ES_HOME=/usr/local/elasticsearch02
Environment=ES_PATH_CONF=/usr/local/elasticsearch02/config
Environment=PID_DIR=/data/elasticsearch02/run
EnvironmentFile=-/etc/sysconfig/elasticsearch02

WorkingDirectory=/usr/local/elasticsearch02

User=elasticsearch
Group=elasticsearch

ExecStart=/usr/local/elasticsearch02/bin/elasticsearch  -p ${PID_DIR}/elasticsearch.pid --quiet

#StandardOutput is configured to redirect to journalctl since
#some error messages may be logged in standard output before
#elasticsearch logging system is initialized. Elasticsearch
#stores its logs in /var/log/elasticsearch and does not use
#journalctl by default. If you also want to enable journalctl
#logging, you can simply remove the "quiet" option from ExecStart.
StandardOutput=journal
StandardError=inherit

#Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536

#Specifies the maximum number of processes
LimitNPROC=4096

#Specifies the maximum size of virtual memory
LimitAS=infinity

#Specifies the maximum file size
LimitFSIZE=infinity

#Disable timeout logic and wait until process is stopped
TimeoutStopSec=0

#SIGTERM signal is used to stop the Java process
KillSignal=SIGTERM

#Send the signal only to the JVM rather than its control group
KillMode=process

#Java process is never killed
SendSIGKILL=no
#When a JVM receives a SIGTERM signal it exits with code 143
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target
#Built for distribution-6.2.4 (distribution)
[root@sdk-25 local]# cat /etc/sysconfig/elasticsearch02 
################################
#Elasticsearch
################################

#Elasticsearch home directory
ES_HOME=/usr/local/elasticsearch02

#Elasticsearch Java path
JAVA_HOME=/usr/local/jdk

#Elasticsearch configuration directory
ES_PATH_CONF=/usr/local/elasticsearch02/config

#Elasticsearch PID directory
PID_DIR=/data/elasticsearch02/run

#Additional Java OPTS
#ES_JAVA_OPTS=

#Configure restart on package upgrade (true, every other setting will lead to not restarting)
#RESTART_ON_UPGRADE=true

################################
#Elasticsearch service
################################

#SysV init.d

#The number of seconds to wait before checking if Elasticsearch started successfully as a daemon process
ES_STARTUP_SLEEP_TIME=5

################################
#System properties
################################

#Specifies the maximum file descriptor number that can be opened by this process
#When using Systemd, this setting is ignored and the LimitNOFILE defined in
#/usr/lib/systemd/system/elasticsearch.service takes precedence
#MAX_OPEN_FILES=65536

#The maximum number of bytes of memory that may be locked into RAM
#Set to "unlimited" if you use the 'bootstrap.memory_lock: true' option
#in elasticsearch.yml.
#When using systemd, LimitMEMLOCK must be set in a unit file such as
#/etc/systemd/system/elasticsearch.service.d/override.conf.
#MAX_LOCKED_MEMORY=unlimited

#Maximum number of VMA (Virtual Memory Areas) a process can own
#When using Systemd, this setting is ignored and the 'vm.max_map_count'
#property is set at boot time in /usr/lib/sysctl.d/elasticsearch.conf
#MAX_MAP_COUNT=262144

systemctl 相关的es启动命令以下:

保证es开机自启动:
[root@sdk-25 system]# systemctl enable elasticsearch02.service
Created symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch02.service to /usr/lib/systemd/system/elasticsearch02.service.

[root@sdk-25 local]# service elasticsearch02 status/stop/restart
[root@sdk-25 local]# systemctl status elasticsearch02.service 
● elasticsearch02.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch02.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since 日 2020-07-26 12:21:49 CST; 6s ago
     Docs: http://www.elastic.co
  Process: 4255 ExecStart=/usr/local/elasticsearch02/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=143)
 Main PID: 4255 (code=exited, status=143)

7月 25 00:02:22 sdk-25 systemd[1]: Started Elasticsearch.
7月 26 12:21:49 sdk-25 systemd[1]: Stopping Elasticsearch...
7月 26 12:21:49 sdk-25 systemd[1]: Stopped Elasticsearch.

[root@sdk-25 local]# service elasticsearch02 start
Redirecting to /bin/systemctl start elasticsearch02.service
[root@sdk-25 local]# ss -lntup|egrep "9201|9301"
tcp    LISTEN     0      128    ::ffff:127.0.0.1:9201                 :::*                   users:(("java",pid=11387,fd=685))
tcp    LISTEN     0      128    ::ffff:127.0.0.1:9301                 :::*                   users:(("java",pid=11387,fd=491))
[root@sdk-25 system]# curl http://127.0.0.1:9201
{
  "name" : "node25-1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "6qPnIoTCRn2fACH4CENyPA",
  "version" : {
    "number" : "6.2.4",
    "build_hash" : "ccec39f",
    "build_date" : "2018-04-12T20:37:28.497551Z",
    "build_snapshot" : false,
    "lucene_version" : "7.2.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

2.3启动第三个单实例elasticsearch03

操做步骤和上面的方法同样，此处再也不过多的描述了

3、elasticsearch01 实例一上安装IK插件和pinyin插件,以及x-pack插件

这些插件的版本要和实例elasticsearch01的版本严格保持一致，不然会出错（不兼容）
安装IK插件和pinyin插件 直接在github上下下载，解压到/usr/local/elasticsearch01/plugins ,同时必定要注意插件的权限必须为elasticsearch，不然到期es重启失败。同时安装完插件要重启下es服务

[root@sdk-25 plugins]# pwd
/usr/local/elasticsearch01/plugins
[root@sdk-25 plugins]# ls
ik  pinyin  x-pack
[root@sdk-25 plugins]# ll
总用量 0
drwxrwxrwx  3 elasticsearch elasticsearch 213 5月   6 2018 ik
drwxrwxrwx  2 elasticsearch elasticsearch 113 5月   6 2018 pinyin
drwxr-xr-x 11 elasticsearch elasticsearch 244 7月  25 12:07 x-pack

下面详细的介绍下x-pack插件的安装和简单的应用：

安装x-pack插件
参考文档:
https://www.jianshu.com/p/802c5d803a95

查看已经安装的插件:

[root@sdk-25 vhost]# /usr/local/elasticsearch01/bin/elasticsearch-plugin  list
ik
pinyi

注册x-pack插件，两种安装方式：

在线安装方式：（g国内的话，基于网络环境，，很是慢，基本安装不上）

下面是在国外的服务器进行如今安装的，很是的快
[root@192-200-102-74 plugins]# /usr/share/elasticsearch/bin/elasticsearch-plugin install x-pack 
-> Downloading x-pack from elastic
[=================================================] 100%?? 
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.io.FilePermission \\.\pipe\* read,write
* java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
* java.net.SocketPermission * connect,accept,resolve
* java.security.SecurityPermission createPolicy.JavaPolicy
* java.security.SecurityPermission getPolicy
* java.security.SecurityPermission putProviderProperty.BC
* java.security.SecurityPermission setPolicy
* java.util.PropertyPermission * read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@        WARNING: plugin forks a native controller        @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
This plugin launches a native controller that is not subject to the Java
security manager nor to system call filters.

Continue with installation? [y/N]y
Elasticsearch keystore is required by plugin [x-pack-security], creating...
-> Installed x-pack with: x-pack-core,x-pack-deprecation,x-pack-graph,x-pack-logstash,x-pack-ml,x-pack-monitoring,x-pack-security,x-pack-upgrade,x-pack-watcher
[root@192-200-102-74 plugins]# echo $?
0

第二种就是离线安装方式:国内的服务器建议就离线安装:

**提早下载多对应的x-pack插件的版本。
我线上用的是elasticsearch.6.2.4.tar.gz 二进制包安装的，因此要下载对应的离线插件版本x-pack.2.6.4.zip包
官方的下载地址，固然也得搭*下载
https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-6.2.4.zip

下载完，上传到香港的阿里ECS服务器
du -sh x-pack-6.2.4.zip
296M x-pack-6.2.4.zip

从HK的es服务器推送到阿里的OSS,而后经过阿里CDN域名来下载：

[root@hk-cj01 ~]# /usr/local/sbin/ossutil64 --config-file=/data/soft/ossconfig cp x-pack-6.2.4.zip oss://lanhu-static/zy01baodown/ --update 
Succeed: Total num: 1, size: 309,419,696. OK num: 1(upload 1 files).                                        
79.701253(s) elapsed

wget https://va1.j7lf.cn/zy01baodown/x-pack-6.2.4.zip

[root@sdk-25 vhost]# /usr/local/elasticsearch01/bin/elasticsearch-plugin install file:///data/soft/x-pack-6.2.4.zip 
-> Downloading file:///data/soft/x-pack-6.2.4.zip
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.io.FilePermission \\.\pipe\* read,write
* java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
* java.net.SocketPermission * connect,accept,resolve
* java.security.SecurityPermission createPolicy.JavaPolicy
* java.security.SecurityPermission getPolicy
* java.security.SecurityPermission putProviderProperty.BC
* java.security.SecurityPermission setPolicy
* java.util.PropertyPermission * read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@        WARNING: plugin forks a native controller        @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
This plugin launches a native controller that is not subject to the Java
security manager nor to system call filters.

Continue with installation? [y/N]y
Elasticsearch keystore is required by plugin [x-pack-security], creating...
-> Installed x-pack with: x-pack-core,x-pack-deprecation,x-pack-graph,x-pack-logstash,x-pack-ml,x-pack-monitoring,x-pack-security,x-pack-upgrade,x-pack-watcher

肯定插件是否安装成功:

[root@sdk-25 vhost]# /usr/local/elasticsearch01/bin/elasticsearch-plugin list
ik
pinyin
x-pack
    x-pack-core
    x-pack-deprecation
    x-pack-graph
    x-pack-logstash
    x-pack-ml
    x-pack-monitoring
    x-pack-security
    x-pack-upgrade
    x-pack-watcher

卸载x-pack插件

bin/elasticsearch-plugin remove x-pack

es默认生成的密码以下:

/usr/local/elasticsearch01/bin/x-pack/setup-passwords auto
[root@sdk-25 vhost]# /usr/local/elasticsearch01/bin/x-pack/setup-passwords auto 

Unexpected response code [404] from calling GET http://127.0.0.1:9200/_xpack/security/_authenticate?pretty
Possible causes include:
 * The relative path of the URL is incorrect. Is there a proxy in-between?
 * The protocol (http/https) does not match the port.
 * Is this really an Elasticsearch server?

ERROR: Uknown error

报错，由于安装完插件须要重启下es,

[root@sdk-25 vhost]# systemctl restart elasticsearch.service
**重启完接着报错：**
[root@sdk-25 vhost]# systemctl status elasticsearch.service
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since 六 2020-07-25 12:27:59 CST; 7s ago
     Docs: http://www.elastic.co
  Process: 32419 ExecStart=/usr/local/elasticsearch01/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
 Main PID: 32419 (code=exited, status=1/FAILURE)

7月 25 12:27:57 sdk-25 systemd[1]: Started Elasticsearch.
7月 25 12:27:59 sdk-25 elasticsearch[32419]: Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /usr/local/elastic...rch.keystore
7月 25 12:27:59 sdk-25 systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
7月 25 12:27:59 sdk-25 systemd[1]: Unit elasticsearch.service entered failed state.
7月 25 12:27:59 sdk-25 systemd[1]: elasticsearch.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

查看安装上的插件权限，发现是权限不对致使的

[root@sdk-25 plugins]# ll
总用量 0
drwxrwxrwx  3 root root 213 5月   6 2018 ik
drwxrwxrwx  2 root root 113 5月   6 2018 pinyin
drwxr-xr-x 11 root root 244 7月  25 12:07 x-pack

受权elasticsearch：

[root@sdk-25 elasticsearch01]# chown -R elasticsearch.elasticsearch *
启动成功：
[root@sdk-25 elasticsearch01]# systemctl status  elasticsearch.service
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: active (running) since 六 2020-07-25 12:33:11 CST; 43min ago
     Docs: http://www.elastic.co
 Main PID: 1266 (java)
   CGroup: /system.slice/elasticsearch.service
           ├─1266 /usr/local/jdk/bin/java -Xms4g -Xmx4g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.a...
           └─1431 /usr/local/elasticsearch01/plugins/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller

7月 25 12:33:11 sdk-25 systemd[1]: Started Elasticsearch.

es默认生成的密码以下:

/usr/local/elasticsearch01/bin/x-pack/setup-passwords auto 
[root@sdk-25 elasticsearch01]# /usr/local/elasticsearch01/bin/x-pack/setup-passwords auto 
Initiating the setup of passwords for reserved users elastic,kibana,logstash_system.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y

Changed password for user kibana
PASSWORD kibana = FvdqDOUkXvEijZKjfB8p

Changed password for user logstash_system
PASSWORD logstash_system = VUFZn9iL4AEJrH3Owkdq

Changed password for user elastic
PASSWORD elastic = 5BheRCDLKSvT1ZP1zhHf
[root@sdk-25 elasticsearch01]# curl http://127.0.0.1:9200 
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}[root@sdk-25 elasticsearch01]#

经过帐户和密码访问:

[root@sdk-25 elasticsearch01]# curl --user elastic:5BheRCDLKSvT1ZP1zhHf http://127.0.0.1:9200
{
  "name" : "node25",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "YSRMrxOBTZW7hicZqZ-Dhg",
  "version" : {
    "number" : "6.2.4",
    "build_hash" : "ccec39f",
    "build_date" : "2018-04-12T20:37:28.497551Z",
    "build_snapshot" : false,
    "lucene_version" : "7.2.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

忘记ES密码找回方法:

https://www.cnblogs.com/mere/p/12165637.html

使用命令ES_HOME/bin/x-pack/users建立一个基于本地问价认证的超级管理员:

[root@sdk-25 config]# /usr/local/elasticsearch01/bin/x-pack/users useradd my_admin -p 5BheRCDLK12389Sv -r superuser
[root@sdk-25 config]#

经过api重置elastic超级管理员的密码:

curl -u my_admin -XPUT 'http://localhost:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'{"password" : "5BheRCDLK12389Sv"}'

[root@sdk-25 config]# curl -u my_admin -XPUT 'http://localhost:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'{"password" : "5BheRCDLK12389Sv"}'
Enter host password for user 'my_admin': 5BheRCDLK12389Sv
{ }
[2020-07-25T14:19:50,117][INFO ][o.e.x.s.a.f.FileUserPasswdStore] [node25] users file [/usr/local/elasticsearch01/config/x-pack/users] changed. updating users... )
[2020-07-25T14:19:50,124][INFO ][o.e.x.s.a.f.FileUserRolesStore] [node25] users roles file [/usr/local/elasticsearch01/config/x-pack/users_roles] changed. updating users roles...

使用原来的密码登陆失败:

[root@sdk-25 ~]# curl --user elastic:5BheRCDLKSvT1ZP1zhHf http://127.0.0.1:9200
{"error":{"root_cause":[{"type":"security_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"failed to authenticate user [elastic]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}[root@sdk-25 ~]# 

[2020-07-25T14:23:51,638][INFO ][o.e.x.s.a.AuthenticationService] [node25] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]

使用新的密码是成功的:

[root@sdk-25 ~]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9200
{
  "name" : "node25",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "YSRMrxOBTZW7hicZqZ-Dhg",
  "version" : {
    "number" : "6.2.4",
    "build_hash" : "ccec39f",
    "build_date" : "2018-04-12T20:37:28.497551Z",
    "build_snapshot" : false,
    "lucene_version" : "7.2.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

校验下密码是否重置成功:

curl -u elastic 'http://127.0.0.1:9200/_xpack/security/_authenticate?pretty'

[root@sdk-25 ~]# curl -u elastic 'http://localhost:9200/_xpack/security/_authenticate?pretty'
Enter host password for user 'elastic':  5BheRCDLK12389Sv
{
  "username" : "elastic",
  "roles" : [
    "superuser"
  ],
  "full_name" : null,
  "email" : null,
  "metadata" : {
    "_reserved" : true
  },
  "enabled" : true
}

ElasticSearch之CURL操做:
https://blog.csdn.net/diyiday/article/details/83927744

[root@local-216 ~]# /usr/share/elasticsearch/bin/x-pack/users useradd my_admin -p admin123987 -r superuser

ES学习参考资料:
http://www.javashuo.com/article/p-seoiweer-mh.html

4、单台服务器安装3个ES实例，配置基于X-pack密码认证的ES集群

配置文件内容以下:

[root@sdk-25 logs]# cat /usr/local/elasticsearch01/config/elasticsearch.yml
cluster.name: escluster
node.name: es1
node.master: true
node.data: true
path.data: /data/elasticsearch01/data
path.logs: /data/elasticsearch01/logs
bootstrap.memory_lock: true
bootstrap.system_call_filter: false
http.port: 9200
transport.tcp.port: 9300
network.host: 127.0.0.1
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 3s
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9301","127.0.0.1:9302"]
[root@sdk-25 logs]# cat /usr/local/elasticsearch02/config/elasticsearch.yml
cluster.name: escluster
node.name: es2
node.master: true
node.data: true
path.data: /data/elasticsearch02/data
path.logs: /data/elasticsearch02/logs
bootstrap.memory_lock: true
bootstrap.system_call_filter: false
http.port: 9201
transport.tcp.port: 9301
network.host: 127.0.0.1
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 3s
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300","127.0.0.1:9302"]
[root@sdk-25 logs]# cat /usr/local/elasticsearch03/config/elasticsearch.yml
cluster.name: escluster
node.name: es3
node.master: true
node.data: true
path.data: /data/elasticsearch03/data
path.logs: /data/elasticsearch03/logs
bootstrap.memory_lock: true
bootstrap.system_call_filter: false
http.port: 9202
transport.tcp.port: 9302
network.host: 127.0.0.1
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 3s
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300","127.0.0.1:9301"]

三台ES实例都启动，而且三个实例都不要提早安装x-pack密码认证插件.三台实例会自动识别加入到集群中。

正确给ES集群设置密码的方法是：
一开始3个实例都不要安装x-pack插件设置密码。而是先配置好三个实例，都启动，从是master节点的实例上来安装x-pack插件来来设置密码。
而后其余的节点也都安装x-pack插件，而后重启其余的2个ES实例，可是其余的2个ES实例不须要设置密码

下面是三个ES实例都未安装x-pack密码插件时，查看集群的状态:

[root@sdk-25 plugins]# curl  http://127.0.0.1:9200/_cat/nodes
127.0.0.1 28 32 0 0.06 0.09 0.12 mdi - es3
127.0.0.1 42 32 0 0.06 0.09 0.12 mdi * es1
127.0.0.1 44 32 0 0.06 0.09 0.12 mdi - es2
[root@sdk-25 plugins]# curl  http://127.0.0.1:9201/_cat/nodes
127.0.0.1 42 32 0 0.00 0.04 0.10 mdi * es1
127.0.0.1 44 32 0 0.00 0.04 0.10 mdi - es2
127.0.0.1 28 32 0 0.00 0.04 0.10 mdi - es3
[root@sdk-25 plugins]# curl  http://127.0.0.1:9202/_cat/nodes
127.0.0.1 28 32 0 0.00 0.04 0.10 mdi - es3
127.0.0.1 42 32 0 0.00 0.04 0.10 mdi * es1
127.0.0.1 44 32 0 0.00 0.04 0.10 mdi - es2

master节点是ES1:

[root@sdk-25 plugins]# curl 'http://127.0.0.1:9200/_cat/master?v'
id                     host      ip        node
VojUpPevTV2tH56AwJN03g 127.0.0.1 127.0.0.1 es1

能够看到ES3节点是集群中的master，从是ES-master节点ES1的实例上来安装x-pack插件来给次实例设置密码。而后再给其余的2个ES实例也安装x-pack插件，而且重启其余的2个ES服务。
可是都不须要给这2个ES实例设置密码

[root@sdk-25 bin]# /usr/local/elasticsearch01/bin/elasticsearch-plugin install file:///data/soft/x-pack-6.2.4.zip
-> Downloading file:///data/soft/x-pack-6.2.4.zip
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.io.FilePermission \\.\pipe\* read,write
* java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
* java.net.SocketPermission * connect,accept,resolve
* java.security.SecurityPermission createPolicy.JavaPolicy
* java.security.SecurityPermission getPolicy
* java.security.SecurityPermission putProviderProperty.BC
* java.security.SecurityPermission setPolicy
* java.util.PropertyPermission * read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@        WARNING: plugin forks a native controller        @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
This plugin launches a native controller that is not subject to the Java
security manager nor to system call filters.

Continue with installation? [y/N]y
Elasticsearch keystore is required by plugin [x-pack-security], creating...
-> Installed x-pack with: x-pack-core,x-pack-deprecation,x-pack-graph,x-pack-logstash,x-pack-ml,x-pack-monitoring,x-pack-security,x-pack-upgrade,x-pack-watcher
[root@sdk-25 bin]# 
[root@sdk-25 bin]# cd /usr/local/elasticsearch01/plugins/
[root@sdk-25 plugins]# ll
总用量 0
drwxrwxrwx  3 root root 213 5月   6 2018 ik
drwxrwxrwx  2 root root 113 5月   6 2018 pinyin
drwxr-xr-x 11 root root 244 7月  26 16:15 x-pack
 [root@sdk-25 bin]#cd /usr/local/elasticsearch03
 [root@sdk-25 bin]# chown -R elasticsearch.elasticsearch *
[root@sdk-25 plugins]# ll
总用量 0
drwxrwxrwx  3 elasticsearch elasticsearch 213 5月   6 2018 ik
drwxrwxrwx  2 elasticsearch elasticsearch 113 5月   6 2018 pinyin
drwxr-xr-x 11 elasticsearch elasticsearch 244 7月  26 16:15 x-pack

[root@sdk-25 config]# systemctl restart elasticsearch01
[root@sdk-25 config]#  /usr/local/elasticsearch01/bin/x-pack/users useradd my_admin -p 5BheRCDLK12389Sv -r superuser
[root@sdk-25 config]# curl -u my_admin -XPUT 'http://localhost:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'{"password" : "5BheRCDLK12389Sv"}'
Enter host password for user 'my_admin':
{ }

给剩余的ES2 ES3实例安装x-pack插件，而后受权elasticsearch.elasticsearch，最后重启这个2个es实例：

/usr/local/elasticsearch03/bin/elasticsearch-plugin install file:///data/soft/x-pack-6.2.4.zip
/usr/local/elasticsearch02/bin/elasticsearch-plugin install file:///data/soft/x-pack-6.2.4.zip
[root@sdk-25 config]# cd /usr/local/
[root@sdk-25 local]# chown -R elasticsearch.elasticsearch elasticsearch0*

[root@sdk-25 ~]# systemctl restart elasticsearch02; systemctl restart elasticsearch03

此时只能经过用户和密码来查看集群的状态:

[root@sdk-25 local]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9200/_cat/nodes
127.0.0.1 22 32 5 3.07 1.00 0.47 mdi - es2
127.0.0.1 38 32 5 3.07 1.00 0.47 mdi * es1
127.0.0.1 31 32 5 3.07 1.00 0.47 mdi - es3
[root@sdk-25 local]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9201/_cat/nodes
127.0.0.1 38 32 0 2.83 0.98 0.47 mdi * es1
127.0.0.1 22 32 0 2.83 0.98 0.47 mdi - es2
127.0.0.1 31 32 0 2.83 0.98 0.47 mdi - es3
[root@sdk-25 local]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9202/_cat/nodes
127.0.0.1 24 32 1 2.76 1.00 0.48 mdi - es2
127.0.0.1 41 32 1 2.76 1.00 0.48 mdi * es1
127.0.0.1 31 32 1 2.76 1.00 0.48 mdi - es3

不输入帐户和密码访问节点报错：

[root@sdk-25 local]# curl http://127.0.0.1:9200/_cat/nodes
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/_cat/nodes]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication token for REST request [/_cat/nodes]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}[root@sdk-25 local]#

5、ES集群部署过程当中遇到的问题

案例一：

[root@sdk-25 plugins]# /usr/local/elasticsearch01/bin/x-pack/setup-passwords auto
Unexpected response code [404] from calling GET http://127.0.0.1:9202/_xpack/security/_authenticate?pretty
Possible causes include:
 * The relative path of the URL is incorrect. Is there a proxy in-between?
 * The protocol (http/https) does not match the port.
 * Is this really an Elasticsearch server?

ERROR: Uknown error

缘由是安装完x-pack插件没有从新受权elasticsearch.elasticsearch.而且重启es服务，才致使的报错

案例二：

elasticsearch01 实例安装了x-pack 插件的，而且设置了ES的登陆密码 其余的都没有安装，致使在配置集群时，实例elasticsearch01加入到集群中失败
实例es1 输出错误日志：

[zen-disco-node-failed({es1}{jbc_qu6ZQteoD1uH_o6eEg}{vmt_wvYPQwaHCPnBQOonzw}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=67336515584, 
ml.max_open_jobs=20, ml.enabled=true}), reason(failed to ping, tried [3] times, each with maximum [30s] timeout)[{es1}{jbc_qu6ZQteoD1uH_o6eEg}
{vmt_wvYPQwaHCPnBQOonzw}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=67336515584, ml.max_open_jobs=20, ml.enabled=true} 
failed to ping, tried [3] times, each with maximum [30s] timeout, {es1}{jbc_qu6ZQteoD1uH_o6eEg}{vmt_wvYPQwaHCPnBQOonzw}
{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=67336515584, ml.max_open_jobs=20, ml.enabled=true} failed to ping, tried [3] times, each with maximum [30s] timeout]]])

查看集群中各个节点只能查看到实例2和实例3的，看不到实例1的：

[root@sdk-25 logs]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9202/_cat/nodes 
127.0.0.1 29 32 0 0.23 0.26 0.19 mdi * es3
127.0.0.1 29 32 1 0.23 0.26 0.19 mdi - es2
[root@sdk-25 logs]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9201/_cat/nodes 
127.0.0.1 29 32 0 0.23 0.26 0.19 mdi * es3
127.0.0.1 29 32 0 0.23 0.26 0.19 mdi - es2
[root@sdk-25 logs]# curl --user elastic:5BheRCDLK12389Sv http://127.0.0.1:9200/_cat/nodes 
127.0.0.1 29 32 0 0.21 0.25 0.19 mdi * es3
127.0.0.1 29 32 0 0.21 0.25 0.19 mdi - es2

正确给ES集群设置密码的方法是：
一开始3个实例都不要安装x-pack插件设置密码。而是先配置好三个实例，都启动，从是master节点的实例上来安装x-pack插件来来设置密码。
而后其余的节点也都安装x-pack插件，而后重启其余的2个ES实例，可是其余的2个ES实例不须要设置密码

下面是未安装x-pack密码插件时，查看集群的状态:

[root@sdk-25 plugins]# curl  http://127.0.0.1:9200/_cat/nodes
127.0.0.1 28 32 0 0.06 0.09 0.12 mdi - es1
127.0.0.1 42 32 0 0.06 0.09 0.12 mdi * es3
127.0.0.1 44 32 0 0.06 0.09 0.12 mdi - es2
[root@sdk-25 plugins]# curl  http://127.0.0.1:9201/_cat/nodes
127.0.0.1 42 32 0 0.00 0.04 0.10 mdi * es3
127.0.0.1 44 32 0 0.00 0.04 0.10 mdi - es2
127.0.0.1 28 32 0 0.00 0.04 0.10 mdi - es1
[root@sdk-25 plugins]# curl  http://127.0.0.1:9202/_cat/nodes
127.0.0.1 28 32 0 0.00 0.04 0.10 mdi - es1
127.0.0.1 42 32 0 0.00 0.04 0.10 mdi * es3
127.0.0.1 44 32 0 0.00 0.04 0.10 mdi - es2

[root@sdk-25 plugins]# curl 'http://127.0.0.1:9200/_cat/master?v'
id                     host      ip        node
VojUpPevTV2tH56AwJN03g 127.0.0.1 127.0.0.1 es3

能够看到ES3节点是集群中的master，从是ES-master节点的实例上来安装x-pack插件来给次实例设置密码。而后再给其余的2个ES实例也安装x-pack插件，而且重启其余的2个ES服务。
可是都不须要给这2个ES实例设置密码

此处不太理解：一旦重启ES实例，原先是maser节点的ES会切换到其余的节点上，那在原先的master节点上重启后设置的密码数据如何才能被复制到其余的节点上呢？？？？

相关文章
相关标签/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公众号
   欢迎关注本站公众号,获取更多信息
联系我们 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程