006-ELK的二进制安装

二进制安装ELKstack

本次搭建属于单点，在同一台机器上进行安装

基础组件部署

curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
 wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
 yum makecache
 yum install wget vim lsof net-tools lrzsz -y
 yum -y install ntp
 systemctl enable ntpd
 systemctl start ntpd
 ntpdate -u cn.pool.ntp.org
 hwclock --systohc
 timedatectl set-timezone Asia/Shanghai
 systemctl stop firewalld
 systemctl disable firewalld
 setenforce 0
 sed -i '/SELINUX/s/enforcing/disabled/' /etc/selinux/config
 echo '
* hard nofile 65536
* soft nofile 65536
* soft nproc  65536
* hard nproc  65536
'>>/etc/security/limit.conf
echo '
vm.max_map_count = 262144
net.core.somaxconn=65535
net.ipv4.ip_forward = 1
'>>/etc/sysctl.conf
sysctl -p
yum install -y java
mkdir /root/elk -p;cd /root/elk 
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.5.0.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.5.0-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/logstash/logstash-5.5.0.tar.gz

安装Elasticsearch

mkdir /data/{es-data,es-logs} -p
tar xf elasticsearch-5.5.0.tar.gz -C /usr/local/
ln -s /usr/local/elasticsearch-5.5.0 /usr/local/elasticsearch
cat>>/usr/local/elasticsearch-5.5.0/config/elasticsearch.yml<<EOF
node.name: my-es
path.data: /data/es-data
path.logs: /data/es-logs
http.host: "192.168.83.60"
http.cors.enabled: true    
http.cors.allow-origin: "*"
EOF
useradd elk
chown -R elk /data/es-*
chown -R elk /usr/local/elasticsearch-5.5.0/

启动es

/usr/local/elasticsearch/bin/elasticsearch -d -d 后台运行 第一次不加-d参数，则能够看到启动的报错信息

[root@60 data]# lsof -i:9200
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    20586  elk  132u  IPv6  61210      0t0  TCP 60:wap-wsp (LISTEN)     
[root@60 data]# curl 192.168.83.60:9200
{
  "name" : "my-es",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "cYF13nSqQAi91gHnn9I_bg",
  "version" : {
    "number" : "5.5.0",
    "build_hash" : "260387d",
    "build_date" : "2017-06-30T23:16:05.735Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}

相关配置信息解说文章： https://blog.csdn.net/laoyang360/article/details/72850834

ES经常使用命令

curl -XDELETE 'http://host.IP.address:9200/logstash-*'  删除索引(后面为索引名称)
curl -XGET 'host.IP.address:9200/_cat/health?v&pretty'  查看集群状态
curl -XGET 'host.IP.address:9200/_cat/indices?v&pretty'    查看索引

logstash安装

tar xf logstash-5.5.0.tar.gz -C /usr/local/
ln -s /usr/local/logstash-5.5.0 /usr/local/logstash
通常状况下，咱们能够不配置logstash直接就能够启动，logstash下有一个叫logstash.yml的文件，里面能够对logstash作一些简单的优化
vim /usr/local/logstash/config/logstash.yml
config.reload.automatic: true    #开启配置文件自动加载
config.reload.interval: 10    #定义配置文件重载时间周期

建立配置文件

mkdir /usr/local/logstash/conf/
vim /usr/local/logstash/conf/nginx.conf
input {
    file {
        path => " /usr/local/nginx/logs/access.log"
        start_position => "beginning"
        }
}
filter {
}
output {
        #stdout { codec=> rubydebug }
    elasticsearch {
        hosts => ["192.168.83.60:9200"]
        index => "static_nginx_access-%{+YYYY.MM.dd}"
 }
}

#这里面实际只是定义了一个nginx log路径
#stdout { codec=> rubydebug }  将内容打印出来，方便测试
#host 填写es主机
#index 索引名称

nginx中日志配置

log_format json '{"@timestamp":"$time_iso8601",'
                  '"host":"$server_addr",'
                  '"clientip":"$remote_addr",'
                  '"remote_user":"$remote_user",'
                  '"request":"$request",'
                  '"http_user_agent":"$http_user_agent",'
                  '"size":$body_bytes_sent,'
                  '"responsetime":$request_time,'
                  '"upstreamtime":"$upstream_response_time",'
                  '"upstreamhost":"$upstream_addr",'
                  '"http_host":"$host",'
                  '"requesturi":"$request_uri",'
                  '"url":"$uri",'
                  '"domain":"$host",'
                  '"xff":"$http_x_forwarded_for",'
                  '"referer":"$http_referer",'
                  '"status":"$status"}';
    access_log /usr/local/nginx/logs/access.log json;

测试logstash配置文件是否正常

尽可能使用logstash chown -RHL elk.elk /usr/local/logstash su - elk

[elk@60 ~]$ /usr/local/logstash/bin/logstash -f /usr/local/logstash/conf/nginx.conf  -t
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Sending Logstash's logs to /usr/local/logstash/logs which is now configured via log4j2.properties
[2019-12-12T02:39:24,954][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/usr/local/logstash/data/queue"}
[2019-12-12T02:39:24,956][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/local/logstash/data/dead_letter_queue"}
Configuration OK
[2019-12-12T02:39:25,050][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

kibana安装

cd /root/elk
tar xf kibana-5.5.0-linux-x86_64.tar.gz -C /usr/local/
cd /usr/local
ln -nfsv kibana-5.5.0-linux-x86_64 kibana
chown -RHL elk.elk kibana-5.5.0-linux-x86_64
vim /usr/local/kibana/config/kibana.yml
server.port: 5601
server.host: "192.168.83.60"
elasticsearch.url: "http://192.168.83.60:9200"

启动kibana

su - elk
前台启动查看是否异常
/usr/local/kibana/bin/kibana
后台启动
nohup /usr/local/kibana/bin/kibana &
查看索引
[root[@60](https://my.oschina.net/bingo60) ~]# curl -XGET '192.168.83.60:9200/_cat/indices?v&pretty'
health status index                          uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   static_nginx_access-2019.12.11 7mbLeNcfSM-_Z-G3uW3_2Q   5   1          3            0     14.8kb         14.8kb
yellow open   .kibana                        cCBXRM7dREO4imbiIaUsww   1   1          1            0      3.2kb          3.2kb
其中static_nginx_access-2019.12.11为索引名称

kibana建立索引

Elasticsearch Head插件安装

说明：界面化的集群操做和管理工具，能够对集群进行傻瓜式操做 显示集群的拓扑,而且可以执行索引和节点级别操做 搜索接口可以查询集群中原始json或表格格式的检索数据 可以快速访问并显示集群的状态 有一个输入窗口,容许任意调用RESTful API。这个接口包含几个选项,能够组合在一块儿以产生有趣的结果

官方文档：https://github.com/mobz/elasticsearch-head

rpm -vih https://rpm.nodesource.com/pub_10.x/el/7/x86_64/nodejs-10.15.3-1nodesource.x86_64.rpm
npm install -g pm2
cd /usr/local
git clone https://github.com/mobz/elasticsearch-head.git
cd elasticsearch-head/
#须要注意的是执行如下命令须要在elasticsearch-head目录下执行
npm install    
npm run start      
#后台运行nohup npm run start &，或者能够针对npm的启停作一个shell启停脚本
vim /usr/local/elasticsearch/elasticsearch.yml
http.cors.enabled: true        # elasticsearch中启用CORS
http.cors.allow-origin: "*"    #容许访问的IP地址段，* 为全部IP均可以访问

修改启动默认端口 vim /usr/local/elasticsearch-head/Gruntfile.js

修改链接的ip地址 vim /usr/local/elasticsearch-head/_site/app.js

修改完成后直接要npm run start &便可