elk

ELK之一:ELK基础及安装部署

 

一：什么是ELK？

1.1：ELK又称为ELK Stack,是 Elasticsearch、Logstash、Kibana 三个开源软件的组合,每一个完成不一样的功能，Elasticsearch 可实现数据的实时全文搜索搜索、支持分布式可实现高可用、提供API接口，能够处理大规模日志数据，好比Nginx、Tomcat、系统日志等功能，官方地址：https://www.elastic.co/

1.2：Logstash:经过插件实现日志收集，支持日志过滤，支持普通log、自定义json格式的日志解析:

1.3：kibana主要是调用elasticsearch的数据，并进行前端数据可视化的展示：

二：安装部署

2.1：安装环境准备：

2.1.1：系统环境部分

 

两台服务器: Server1:主机名：elkserver1 IP地址：192.168.0.4 Server2：主机名：elkserver2 IP地址：192.168.0.31 操做系统：Centos 7.2.11 x86_64 Server1： systemctl disable firewalld #开机关闭防火墙 sed -i '/SELINUX/s/enforcing/disabled/' /etc/selinux/config #开机关闭selinux echo "* soft nofile 65536" >> /etc/security/limits.conf #修改进程打开最大文件描述符限制 echo "* hard nofile 65536" >> /etc/security/limits.conf 

 

2.1.2：两台服务器分别安装java运行环境，能够安装二进制(须要配置profile环境变量)也能够安装rpm包，本文采用下载好的jdk-8u92：

java下载地址：http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

[root@elkserver2 tianqi]# ll  jdk-8u92-linux-x64.rpm

[root@elkserver1 tianqi]# yum install jdk-8u92-linux-x64.rpm

2.1.3：软件包准备,能够配置yum源安装也能够使用rpm包或二进制包，推荐在官网下载rpm包下载：

elasticsearch:官网下载地址：https://www.elastic.co/downloads/elasticsearch ，当前最新版本2.3.5

logstash：官网下载地址：https://www.elastic.co/downloads/logstash，当前最新版本2.3.4

kibana:官网下载地址:https://www.elastic.co/downloads/kibana，当前最新版本4.5.4

2.2：安装部署Eelasticsearch及集群：

2.2.1：两台服务器分别都执行安装Eelasticsearch：

2.2.2:编辑elasticsearch配置文件：

Server1：

 

[root@elkserver1 tianqi]# grep "^[a-Z]" /etc/elasticsearch/elasticsearch.yml cluster.name: hfelk #集群名称，名称相同即属于同一个集群 node.name: elkserver1 #本机在集群的内的名称 path.data: /els/data #保存数据的目录，此目录空间要大IO要高 path.logs: /els/logs #保存日志的目录 bootstrap.mlockall: true #服务启动的时候锁定内存，防止写入swap network.host: 0.0.0.0 #监听地址 http.port: 9200 #监听端口 discovery.zen.ping.unicast.hosts: ["192.168.0.4", "192.168.0.31"] #建立集群的时候组播地址，用于广播无效的状况下

Server2：

[root@elkserver2 tianqi]# grep "^[a-Z]" /etc/elasticsearch/elasticsearch.yml cluster.name: hfelk node.name: elkserver2 #最大的不一样就是node名称不同，其余都一致 path.data: /els/data path.logs: /els/logs bootstrap.mlockall: true network.host: 0.0.0.0 http.port: 9200 discovery.zen.ping.unicast.hosts: ["192.168.0.4", "192.168.0.31"]

2.2.3：分别在两台服务器建立保存数据和日志的目录并对elastic用户受权：

Server1：

[root@elkserver1 tianqi]# mkdir /els/{data,logs} -pv mkdir: created directory ‘/els’ mkdir: created directory ‘/els/data’ mkdir: created directory ‘/els/logs’ [root@elkserver2 tianqi]# chown elasticsearch.elasticsearch /els/ -R

Server2：

[root@elkserver2 tianqi]# mkdir /els/{data,logs} -pv mkdir: created directory ‘/els’ mkdir: created directory ‘/els/data’ mkdir: created directory ‘/els/logs’ [root@elkserver2 tianqi]# chown elasticsearch.elasticsearch /els/ -R

2.2.4：分别启动elasticsearch服务：

[root@elkserver1 tianqi]# systemctl start elasticsearch [root@elkserver2 tianqi]# systemctl start elasticsearch

#Server1启动成功的日志信息，若是启动不成功则根据日志进行排错，检查是否java不知足环境仍是elasticsearch用户对数据目录没有写入权限：

[2016-08-22 05:06:28,601][INFO ][node ] [elkserver1] initialized [2016-08-22 05:06:28,601][INFO ][node ] [elkserver1] starting ... [2016-08-22 05:06:28,802][INFO ][transport ] [elkserver1] publish_address {192.168.0.4:9300}, bound_addresses {[::]:9300} [2016-08-22 05:06:28,813][INFO ][discovery ] [elkserver1] hfelk/9gKGXIUfThC6iixl8kXXFQ [2016-08-22 05:06:31,924][INFO ][cluster.service #master选举为elkserver2 ] [elkserver1] detected_master {elkserver2}{mTutqI1JTaeqS8QLVngQ1A}{192.168.0.31}{192.168.0.31:9300}, added {{elkserver2}{mTutqI1JTaeqS8QLVngQ1A}{192.168.0.31}{192.168.0.31:9300},}, reason: zen-disco-receive(from master [{elkserver2}{mTutqI1JTaeqS8QLVngQ1A}{192.168.0.31}{192.168.0.31:9300}]) [2016-08-22 05:06:32,040][INFO ][http ] [elkserver1] publish_address {192.168.0.4:9200}, bound_addresses {[::]:9200} [2016-08-22 05:06:32,040][INFO ][node ] [elkserver1] started

#查看端口状态：

2.2.5：访问elasticsearch的web界面：

2.3：elasticsearch的插件们：

#插件是为了完成不一样的功能，官方提供了一些插件可是是收费的，另外也有一些开发爱好者提供的插件，能够实现对elasticsearch集群的状态监控与管理配置等功能，以下：

2.3.1：安装head插件：

[root@elkserver1 tianqi]# /usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head

2.3.2:访问插件：

http://hfelk.chinacloudapp.cn:9200/_plugin/head/

#https://github.com/mobz/elasticsearch-head

#集群状态：

#界面操做

#在插件提交数据：

#在插件获取数据：

2.4：安装kopf插件：

[root@elkserver1 tianqi]# /usr/share/elasticsearch/bin/plugin  install lmenezes/elasticsearch-kopf

#github地址：https://github.com/lmenezes/elasticsearch-kopf

2.5：监控集群状态，能够经过访问集群状态接口的返回信息对集群状态进行监控，以下:

2.5.1：获取集群状态的命令：

[root@elkserver1 tianqi]# curl  -XGET “http://192.168.0.4:9200/_cluster/health?pretty=true”

#获取到的是一个字典格式的返回值，那就能够经过python对其中的信息进行分析，例如对status进行分析，若是等于green(绿色)就是运行在正常，等于yellow(黄色)表示副本分片丢失，red(红色)表示主分片丢失

{ "cluster_name" : "hfelk", "status" : "green", "timed_out" : false, "number_of_nodes" : 2, "number_of_data_nodes" : 2, "active_primary_shards" : 5, "active_shards" : 10, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0 }

2.5.2：脚本内容以下：

 

#!/usr/bin/env python #coding:utf-8 #Author Zhang Shijie import smtplib from email.mime.text import MIMEText from email.utils import formataddr import subprocess body = "" def mail(user,mbody): ret = True msg = MIMEText(mbody, 'plain', 'utf-8') msg['From'] = formataddr(["张杰",'yy@126.com']) msg['To'] = formataddr(["ELS报警邮件",'xx@qq.com']) msg['Subject'] = "主题" server = smtplib.SMTP("smtp.126.com", 25) server.login("yy@126.com", "本身的密码") server.sendmail('yy@126.com', user, msg.as_string()) server.quit() return ret false="false" obj = subprocess.Popen(("curl -sXGET http://本身的服务器地址:9200/_cluster/health?pretty=true"),shell=True, stdout=subprocess.PIPE) data = obj.stdout.read() data1 = eval(data) status = data1.get("status") if status == "green": mail("xx@qq.com","ELS 服务器绿色") pass elif status == "yellow": mail("xx@qq.com","ELS 服务器黄色") elif status == "yellow": mail("xx@qq.com","ELS 服务器红色") else: mail("xx@qq.com","ELS服务器可能不在运行")

2.5.3：测试一下脚本：

#打开邮件内容以下：

#脚本内容能够根据实际状况修改便可！

三：安装kibana：

3.1：安装及配置部分：

3.1.1：安装：

3.1.2：配置：

[root@elkserver1 tianqi]# vim /opt/kibana/config/kibana.yml

[root@elkserver1 tianqi]# grep “^[a-Z]” /opt/kibana/config/kibana.yml 

server.port: 5601 #监听的端口 server.host: "0.0.0.0" #监听的地址 elasticsearch.url: "http://192.168.0.4:9200" #elasticsearch服务器的地址，即kibana和elasticsearch能够不在一个服务器

3.1.3：启动服务：

[root@elkserver1 tianqi]# systemctl  start kibana
[root@elkserver1 tianqi]# systemctl  enable kibana

3.1.4：访问web页面：

http://ELS服务器地址:端口   #这是能够访问了，可是不能通过认证，因此谁均可以未经认证访问，所以将端口关闭改成nginx代理

3.2：使用nginx代理kibana：

3.2.1：编译安装一个nginx吧，yum的版本比较低：

[root@elkserver1 tianqi]# rpm -ivh http://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm [root@elkserver1 yum.repos.d]# cd /usr/local/src/ [root@elkserver1 src]# wget http://nginx.org/download/nginx-1.8.1.tar.gz [root@elkserver1 src]# tar xvf nginx-1.8.1.tar.gz [root@elkserver1 src]# mv nginx-1.8.1 /usr/local/ [root@elkserver1 src]# cd /usr/local/nginx-1.8.1/ [root@elkserver1 nginx-1.8.1]# ./configure --prefix=/usr/local/nginx --sbin-path=/usr/local/nginx/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx/nginx.pid --lock-path=/var/lock/nginx.lock --user=nginx --group=nginx --with-http_ssl_module --with-http_stub_status_module --with-http_gzip_static_module --http-client-body-temp-path=/var/tmp/nginx/client/ --http-proxy-temp-path=/var/tmp/nginx/proxy/ --http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi --http-scgi-temp-path=/var/tmp/nginx/scgi --with-pcre root@elkserver1 nginx-1.8.1]# make && make install [root@elkserver1 nginx-1.8.1]# useradd nginx -s /sbin/nologin [root@elkserver1 nginx-1.8.1]# mkdir -pv /var/tmp/nginx/client [root@elkserver1 nginx-1.8.1]# /usr/local/nginx/sbin/nginx #启动nginx，若是没法启动看日志，缺什么补什么

3.2.2：增长一个nginx配置文件：

[root@elkserver1 ~]# vim /usr/local/nginx/conf/conf.d/hfelk_server.conf

server { listen 80; server_name hfelk.chinacloudapp.cn; #当前主机名 auth_basic "Input User and Password"; auth_basic_user_file /usr/local/nginx/conf/htpasswd.users; #登陆认证 location / { proxy_pass http://localhost:5601; #kibana端口 proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; } }

3.2.3：编辑nginx配置主文件导入上一步的配置文件：

[root@elkserver1 ~]# vim /usr/local/nginx/conf/nginx.conf

include /usr/local/nginx/conf/conf.d/*.conf;

3.2.4:生成用户认证文件：

[root@elkserver1 ~]# htpasswd  -b -c /usr/local/nginx/conf/htpasswd.users zhangjie  123456
Adding password for user zhangjie

[root@elkserver1 ~]# chown  nginx.nginx  /usr/local/nginx/conf/htpasswd.users

[root@elkserver1 ~]# chmod  600 /usr/local/nginx/conf/htpasswd.users

 

3.2.4：重启nginx：

[root@elkserver1 ~]# /usr/local/nginx/sbin/nginx -t nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful [root@elkserver1 ~]# /usr/local/nginx/sbin/nginx -s reload

3.2.5：再访问试试：

3.2.5：输入密码登陆:

#输入正确的密码便可登陆

四：实现https访问：

4.1：自制CA证书：

4.1.1：生成CA key，这是至关于CA机构的根证书:

[root@elkserver1 ~]# cd /usr/local/nginx [root@elkserver1 nginx]# mkdir key [root@elkserver1 nginx]# cd key/ [root@elkserver1 key]# openssl genrsa -out ca.key 2048 Generating RSA private key, 2048 bit long modulus ....................................................................................+++ ................................................+++ e is 65537 (0x10001) [root@elkserver1 key]# openssl genrsa -des3 -out server.key 2048 Generating RSA private key, 2048 bit long modulus ...+++ ......................+++ e is 65537 (0x10001) Enter pass phrase for server.key: #输入密码 Verifying - Enter pass phrase for server.key: #重复输入一次密码

4.1.2:生成签名证书，#这是在nginx服务器执行，是生成一个向CA服务器申请签名证书的csr证书，CA服务器根据此csr证书发给一个签名的证书，生成后会是一对儿，即一个公钥一个私钥，公钥用于加密，私钥用于签名：

[root@elkserver1 key]# openssl req -new -key server.key -out server.csr  Enter pass phrase for server.key: #必须输入密码 You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:China string is too long, it needs to be less than  2 bytes long Country Name (2 letter code) [XX]:CN State or Province Name (full name) []:Beijing     Locality Name (eg, city) [Default City]:Beijing Organization Name (eg, company) [Default Company Ltd]:HFAW Organizational Unit Name (eg, section) []:HFAW Common Name (eg, your name or your server's hostname) []:HFELK Email Address []:zhangshijie@weathercn.com Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []:

4.1.3：#删除服务器私钥的密码，其实不删除也能够，只是之后重启nginx都要密码，其实要密码更安全：

[root@elkserver1 key]# openssl rsa -in server.key -out server_new.key Enter pass phrase for server.key: #输入一次建立key的时候的密码 writing RSA key

#当前目录的key文件：

4.1.4：对csr文件签名，会生成一个crt格式的证书：

[root@elkserver1 key]# openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt Signature ok subject=/C=CN/ST=Beijing/L=Beijing/O=HFAW/OU=HFAW/CN=HFELK/emailAddress=zhangshijie@weathercn.com Getting Private key Enter pass phrase for server.key: #CA证书的密码 [root@elkserver1 key]# ll total 20 -rw-r--r-- 1 root root 1675 Aug 23 06:15 ca.key -rw-r--r-- 1 root root 1298 Aug 23 06:36 server.crt #通过CA服务器签名后的crt证书，就能够光明正大的在nginx服务器进行使用了 -rw-r--r-- 1 root root 1050 Aug 23 06:21 server.csr -rw-r--r-- 1 root root 1743 Aug 23 06:16 server.key -rw-r--r-- 1 root root 1675 Aug 23 06:32 server_new.key

4.1.5：配置nginx使用证书：

server { listen 443 ssl; ssl_certificate /usr/local/nginx/key/server.crt; #公钥 ssl_certificate_key /usr/local/nginx/key/server_new.key; #私钥 server_name hfelk.chinacloudapp.cn; #当前主机名 auth_basic "Input User and Passwowd"; auth_basic_user_file /usr/local/nginx/conf/htpasswd.users; #登陆认证 location / { proxy_pass http://localhost:5601; #kibana端口 proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; } }

4.1.6：访问验证：

4.1.6.1：https://服务器地址  #默认就是443端口 

4.1.6.2：添加例外：

4.1.6.3：确认安全例外

4.1.6.4:输入密码登陆：

4.1.6.5：登陆后的界面：

4.2:配置http访问转发至https：

4.2.1:编辑nginx配置文件：

server { listen 80; server_name hfelk.chinacloudapp.cn; rewrite ^(.*)$ https://$server_name$1 permanent; #经过write重写为https访问 } server { listen 443 ssl; ssl_certificate /usr/local/nginx/key/server.crt; ssl_certificate_key /usr/local/nginx/key/server_new.key; server_name hfelk.chinacloudapp.cn; #当前主机名 auth_basic "Input User and Passwowd"; auth_basic_user_file /usr/local/nginx/conf/htpasswd.users; #登陆认证 location / { proxy_pass http://localhost:5601; #kibana端口 proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; } }

4.2.2:测试访问http可否转发至https:

#浏览器访问：http://hfelk.chinacloudapp.cn/