搭建ELK

时间 2020-07-19

标签搭建 elk 繁體版

原文原文链接

CentOS 6.5搭建ELK环境

ELK工做流程

多个独立的Agent(Shipper)负责收集不一样来源的数据，一个中心Agent(Indexer)负责汇总和分析数据，在中心Agent前的Broker(使用Redis实现)做为缓冲区，中心Agent后的ElasticSearch用于存储和搜索数据，前端的Kibana提供丰富的图表展现。html

Shipper表示日志收集，使用LogStash收集各类来源的日志数据，能够是系统日志、文件、Redis、mq等等；前端

Broker做为远程Agent与中心Agent之间的缓冲区，使用Redis实现，一是能够提升系统的性能，二是能够提升系统的可靠性，当中心Agent提取数据失败时，数据保存在Redis中，而不至于丢失；java

中心Agent(Indexer)也是LogStash，从Broker中提取数据，能够执行相关的分析和处理(Filter)；node

ElasticSearch用于存储最终的数据，并提供搜索功能；mysql

Kibana提供一个简单、丰富的Web界面，数据来自于ElasticSearch，支持各类查询、统计和展现linux

机器部署

系统ios	IPnginx	配置git
CentOS 7github	192.168.18.171	Logstash
CentOS 6.5	192.168.18.186	ES+Kibana

（Logstash部署在IP为192.168.18.171的机器上。）

数据流

input|decode|filter|encode|output

一.ElasticSearch

安装和配置

若是是在不一样机器上安装，则须要像Logstash的步骤1同样配置好Java环境。

（本文在不一样机器上部署，如下配置在IP为192.168.123.3的机器上进行。）

1.安装Java环境

[root@hxy ~]# yum install java-1.8.0-openjdk

2.下载并安装GPG key

[root@hxy ~]# rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch

3.yum源配置

[root@hxy ~]# cat >/etc/yum.repos.d/elasticsearch.repo<<EOF

[elasticsearch-5.x]

name=Elasticsearch repository for 5.x packages

baseurl=https://artifacts.elastic.co/packages/5.x/yum

gpgcheck=1

gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch

enabled=1

autorefresh=1

type=rpm-md

EOF

4.安装ElasticSearch

[root@hxy ~]# yum install elasticsearch -y

5.修改内核参数 limits.conf

须要修改几个参数，否则启动会报错
vim /etc/security/limits.conf
在末尾追加如下内容（*为启动用户，固然也能够指定为某个用户名）
* soft nofile 65536
* hard nofile 65536
* soft nproc 2048
* hard nproc 2048
* soft memlock unlimited
* hard memlock unlimited

继续再修改一个参数
vim /etc/security/limits.d/90-nproc.conf
将里面的1024改成2048（ES最少要求为2048）
* soft nproc 2048

注:这些是须要重启后生效的,若是启动报错,能够试着重启下虚拟机

6.建立目录并受权

[root@hxy ~]# mkdir -p /data/es-data

[root@hxy ~]# chown -R elasticsearch.elasticsearch /data/es-data/

7.配置elasticsearch.yml

[root@localhost bin]#vim /etc/elasticsearch/elasticsearch.yml

cluster.name: demon # 集群的名称

node.name: elk-1 # 节点的名称

path.data: /data/es-data # 数据存储的目录（多个目录使用逗号分隔）

path.logs: /var/log/elasticsearch # 日志路径

bootstrap.memory_lock: false # 锁住内存，使内存不会分配至交换区（swap）(个人是关闭的,true的话es会没法启动,centos也没有日志或者是报这个错memory locking requested for elasticsearch process but memory is not locked,这个问题我查了好长时间才发现的)

bootstrap.system_call_filter: false #(这是在由于Centos6不支持SecComp，而ES5.2.0默认bootstrap.system_call_filter为true进行检测，因此致使检测失败，失败后直接致使ES不能启动)

network.host: 192.168.18.186 # 本机IP地址

http.port: 9200 # 端口默认9200

http.cors.enabled: true

http.cors.allow-origin: "*"

#查看配置文件

[root@hxy ~]# grep -Ev "^#|^$" /etc/elasticsearch/elasticsearch.yml

cluster.name: demon

node.name: elk-1

path.data: /data/es-data

path.logs: /var/log/elasticsearch/

bootstrap.memory_lock: false

bootstrap.system_call_filter: false

network.host: 192.168.18.186

http.port: 9200

http.cors.enabled: true

http.cors.allow-origin: "*"

8.配置java虚拟机内存

把2g改成512m(系统默认是2g,咱们作实验,虚拟机内存达不到2g会报错)

vim /etc/elasticsearch/jvm.options

-Xms2g

-Xmx2g

#改成

-Xms512m

-Xmx512m

9.启动ElasticSearch

[root@hxy ~]# /etc/init.d/elasticsearch restart

Stopping elasticsearch: [FAILED]

Starting elasticsearch: [ OK ]

10.检查启动

查看进程

[root@hxy ~]# ps -ef|grep ela

496 2458 1 7 14:49 ? 00:00:46 /usr/bin/java -Xms512m -Xmx512m -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid -d -Edefault.path.logs=/var/log/elasticsearch -Edefault.path.data=/var/lib/elasticsearch -Edefault.path.conf=/etc/elasticsearch

root 2835 1774 0 14:59 pts/0 00:00:00 grep ela

查看端口

[root@hxy ~]# netstat -natp |grep 9200

tcp 0 0 :::9200 :::* LISTEN 2458/java

11.访问测试(经过浏览器请求下9200的端口，看下是否成功)

#Linux下访问：

[root@hxy ~]# curl http://127.0.0.1:9200/

{

"name" : "elk-1",

"cluster_name" : "demon",

"cluster_uuid" : "0oT4R0FgSNuymd7KrAF8tw",

"version" : {

"number" : "5.6.8",

"build_hash" : "688ecce",

"build_date" : "2018-02-16T16:46:30.010Z",

"build_snapshot" : false,

"lucene_version" : "6.6.1"

"tagline" : "You Know, for Search"

}

12.windows下访问:

13.如何和elasticsearch交互

JavaAPI

RESTful API
Javascript,.Net,PHP,Perl,Python

利用API查看状态

[root@hxy ~]# curl -i -XGET 'localhost:9200/_count?pretty'
HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 95

{
"count" : 0,
"_shards" : {
"total" : 0,
"successful" : 0,
"failed" : 0
}
}

14.安装插件

安装elasticsearch-head插件
Elasticsearch Head Plugin: 对ES进行各类操做，如查询、删除、浏览索引等。

安装elasticsearch-head插件

安装docker镜像或者经过github下载elasticsearch-head项目都是能够的，1或者2两种方式选择一种安装使用便可

1. 使用docker的集成好的elasticsearch-head

# docker run -p 9100:9100 mobz/elasticsearch-head:5

docker容器下载成功并启动之后，运行浏览器打开http://localhost:9100/

2. 使用git安装elasticsearch-head

# yum install -y npm

# git clone git://github.com/mobz/elasticsearch-head.git

# cd elasticsearch-head

# npm install

# npm run start

检查端口是否起来

netstat -antp |grep 9100

浏览器访问测试是否正常

http://IP:9100/

15.Windows网页

二.Logstash

1.安装logstash

官方安装手册:

https://www.elastic.co/guide/en/logstash/current/installing-logstash.html

下载yum源的密钥认证：

# rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

利用yum安装logstash

# yum install -y logstash

查看下logstash的安装目录

# rpm -ql logstash

建立一个软链接，每次执行命令的时候不用在写安装路劲（默认安装在/usr/share下）

ln -s /usr/share/logstash/bin/logstash /bin/

执行logstash的命令

# logstash -e 'input { stdin { } } output { stdout {} }'

运行成功之后输入:

nihao

stdout返回的结果:

将日志存储到ES中的配置：

注:

-e 执行操做

input 标准输入

{ input } 插件

output 标准输出

{ stdout } 插件

经过rubydebug来输出下更详细的信息

# logstash -e 'input { stdin { } } output { stdout {codec => rubydebug} }'

执行成功输入:

nihao

stdout输出的结果:

6. 运行测试

若是标准输出还有elasticsearch中都须要保留应该怎么玩，看下面
[root@hxy conf.d]# /usr/share/logstash/bin/logstash -e 'input { stdin { } } output { elasticsearch { hosts => ["192.168.18.186:9200"] } stdout { codec => rubydebug }}'

运行成功之后输入:
hello

太慢了

返回的结果(标准输出中的结果):

7.logstash使用配置文件：

官方指南:

https://www.elastic.co/guide/en/logstash/current/configuration.html

建立配置文件01-logstash.conf

这样是指定文件启动,结果同样的

# vim /etc/logstash/conf.d/test.conf

文件中添加如下内容

input { stdin { } }

output {

elasticsearch { hosts => ["192.168.18.186:9200"] }

stdout { codec => rubydebug }

}

使用配置文件运行logstash

# logstash -f ./test.conf

运行成功之后输入以及标准输出结果

logstash的数据库类型

1. Input插件
权威指南:https://www.elastic.co/guide/en/logstash/current/input-plugins.html

file插件的使用
# vim /etc/logstash/conf.d/elk.conf
[root@hxy ~]# cat /etc/logstash/conf.d/elk.conf

input {

file {

path => "/var/log/messages"

type => "system"

start_position => "beginning"

}

output {

elasticsearch {

hosts => ["192.168.18.186:9200"]

index => "system-%{+YYY.MM.dd}"

}

}

运行logstash指定elk.conf配置文件，进行过滤匹配

#注:若是发现配置文件错误的话,最好本身手动的去输入,不要复制,应为你不知道错误在哪里,我这个配置文件就是应为复制的时候错了,查了好半天也没找到缘由,结果本身手动输入就行了,因此不要偷懒
#logstash -f /etc/logstash/conf.d/elk.conf

[root@hxy conf.d]# logstash -f /etc/logstash/conf.d/elk.conf

配置安全日志的而且把日志的索引按类型作存放，继续编辑elk.conf文件

input {

file {

path => "/var/log/messages"

type => "system"

start_position => "beginning"

}

file {

path => "/var/log/secure"

type => "secure"

start_position => "beginning"

}

output {

if [type] == "system" {

elasticsearch {

hosts => ["192.168.18.186:9200"]

index => "zabbix-system-%{+YYY.MM.dd}"

}

if [type] == "secure" {

elasticsearch {

hosts => ["192.168.18.186:9200"]

index => "zabbix-secure-%{+YYY.MM.dd}"

}

logstaash安装完成

三.Kibana

安装和配置

这些设置都没有问题以后，接下来安装下kibana，可让在前台展现

Kibana的安装及使用

安装kibana环境

官方安装手册:https://www.elastic.co/guide/en/kibana/current/install.html

下载kibana的tar.gz的软件包

# wget https://artifacts.elastic.co/downloads/kibana/kibana-5.4.0-linux-x86_64.tar.gz

解压kibana的tar包

# tar -xzf kibana-5.4.0-linux-x86_64.tar.gz

进入解压好的kibana

# mv kibana-5.4.0-linux-x86_64 /usr/local

建立kibana的软链接

# ln -s /usr/local/kibana-5.4.0-linux-x86_64/ /usr/local/kibana

编辑kibana的配置文件

# vim /usr/local/kibana/config/kibana.yml

修改配置文件以下，开启如下的配置

server.port: 5601

server.host: "0.0.0.0"

elasticsearch.url: "http://192.168.8.186:9200"

kibana.index: ".kibana"

安装screen,以便于kibana在后台运行（固然也能够不用安装，用其余方式进行后台启动）

# yum -y install screen

# screen

[root@hxy ~]# grep -Ev '^$|^#' /usr/local/kibana/config/kibana.yml

server.port: 5601

server.host: "0.0.0.0"

elasticsearch.url: "http://192.168.18.186:9200"

kibana.index: ".kibana"

# /usr/local/kibana/bin/kibana

netstat -antp |grep 5601

tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 37134/node

打开浏览器并设置对应的index

http://192.168.18.186:5601

名字本身写上去就OK了

2、ELK实战篇

好，如今索引也能够建立了，如今能够来输出nginx、apache、message、secrue的日志到前台展现1.Nginx有的话直接修改，没有自行安装

编辑nginx配置文件，修改如下内容（在http模块下添加）

log_format json '{"@timestamp":"$time_iso8601",'

'"@version":"1",'

'"client":"$remote_addr",'

'"url":"$uri",'

'"status":"$status",'

'"domian":"$host",'

'"host":"$server_addr",'

'"size":"$body_bytes_sent",'

'"responsetime":"$request_time",'

'"referer":"$http_referer",'

'"ua":"$http_user_agent"'

'}';

修改access_log的输出格式为刚才定义的json

access_log logs/elk.access.log json;

编辑logstash配置文件，进行日志收集

vim /etc/logstash/conf.d/full.conf

input {

file {

path => "/var/log/messages"

type => "system"

start_position => "beginning"

}

file {

path => "/var/log/secure"

type => "secure"

start_position => "beginning"

}

file {

path => "/var/log/nginx/elk.access.log"

type => "nginx"

start_position => "beginning"

}

output {

if [type] == "system" {

elasticsearch {

hosts => ["192.168.18.186:9200"]

index => "nagios-system-%{+YYYY.MM.dd}"

}

if [type] == "secure" {

elasticsearch {

hosts => ["192.168.18.186:9200"]

index => "nagios-secure-%{+YYYY.MM.dd}"

}

if [type] == "nginx" {

elasticsearch {

hosts => ["192.168.18.186:9200"]

index => "nagios-nginx-%{+YYYY.MM.dd}"

}

在页面上查看输入结果,

2. 在centos7安装完成logstash(安装方法和6.5同样的),apche有的话直接修改，没有自行安装

配置apache

修改apache的配置文件

vim /etc/httpd/conf/httpd.conf

LogFormat "{ \

\"@timestamp\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \

\"@version\": \"1\", \

\"tags\":[\"apache\"], \

\"message\": \"%h %l %u %t \\\"%r\\\" %>s %b\", \

\"clientip\": \"%a\", \

\"duration\": %D, \

\"status\": %>s, \

\"request\": \"%U%q\", \

\"urlpath\": \"%U\", \

\"urlquery\": \"%q\", \

\"bytes\": %B, \

\"method\": \"%m\", \

\"site\": \"%{Host}i\", \

\"referer\": \"%{Referer}i\", \

\"useragent\": \"%{User-agent}i\" \

}" ls_apache_json

同样修改输出格式为上面定义的json格式

CustomLog logs/access_log ls_apache_json

重启apache

httpd

启动logstash

logstash -f /etc/logstash/conf.d/apa.conf

注:因为个人centos7是新装的,因此防火墙没有关闭,我这里须要关闭防火墙

systemctl stop firewalld.service

到页面上查看就有结果了

能够发现全部建立日志的索引都已存在，接下来就去Kibana建立日志索引，进行展现（按照上面的方法进行建立索引便可），看下展现的效果

Redis的简单使用方法

https://www.cnblogs.com/idiotgroup/p/5575236.html

下面的我都还没作或是没作成功,而是从原博客上直接复制过来的,就不说了,感兴趣的,能够继续往下看

接下来再来一发MySQL慢日志的展现

因为MySQL的慢日志查询格式比较特殊，因此须要用正则进行匹配，并使用multiline可以进行多行匹配（看具体配置）
input {
file {
path => "/var/log/messages"
type => "system"
start_position => "beginning"
}

file {
path => "/var/log/secure"
type => "secure"
start_position => "beginning"
}

file {
path => "/var/log/httpd/access_log"
type => "http"
start_position => "beginning"
}

file {
path => "/usr/local/nginx/logs/elk.access.log"
type => "nginx"
start_position => "beginning"
}

file {
path => "/var/log/mysql/mysql.slow.log"
type => "mysql"
start_position => "beginning"
codec => multiline {
pattern => "^# User@Host:"
negate => true
what => "previous"
}
}
}

filter {

grok {
match => { "message" => "SELECT SLEEP" }
add_tag => [ "sleep_drop" ]
tag_on_failure => []
}

if "sleep_drop" in [tags] {
drop {}
}

grok {
match => { "message" => "(?m)^# User@Host: %{USER:User}\[[^\]]+\] @ (?:(?<clienthost>\S*) )?\[(?:%{IP:Client_IP})?\]\s.*# Query_time: %{NUMBER:Query_Time:float}\s+Lock_time: %{NUMBER:Lock_Time:float}\s+Rows_sent: %{NUMBER:Rows_Sent:int}\s+Rows_examined: %{NUMBER:Rows_Examined:int}\s*(?:use %{DATA:Database};\s*)?SET timestamp=%{NUMBER:timestamp};\s*(?<Query>(?<Action>\w+)\s+.*)\n# Time:.*$" }
}

date {
match => [ "timestamp", "UNIX" ]
remove_field => [ "timestamp" ]
}

}


output {

if [type] == "system" {

elasticsearch {
hosts => ["192.168.1.202:9200"]
index => "nagios-system-%{+YYYY.MM.dd}"
}
}

if [type] == "secure" {

elasticsearch {
hosts => ["192.168.1.202:9200"]
index => "nagios-secure-%{+YYYY.MM.dd}"
}
}

if [type] == "http" {

elasticsearch {
hosts => ["192.168.1.202:9200"]
index => "nagios-http-%{+YYYY.MM.dd}"
}
}

if [type] == "nginx" {

elasticsearch {
hosts => ["192.168.1.202:9200"]
index => "nagios-nginx-%{+YYYY.MM.dd}"
}
}

if [type] == "mysql" {

elasticsearch {
hosts => ["192.168.1.202:9200"]
index => "nagios-mysql-slow-%{+YYYY.MM.dd}"
}
}
}

查看效果(一条慢日志查询会显示一条，若是不进行正则匹配，那么一行就会显示一条)

具体的日志输出需求，进行具体的分析

三：ELK终极篇

安装reids
# yum install -y redis

修改redis的配置文件
# vim /etc/redis.conf

修改内容以下
daemonize yes

bind 192.168.1.202

启动redis服务
# /etc/init.d/redis restart

测试redis的是否启用成功
# redis-cli -h 192.168.1.202

输入info若是有不报错便可
redis 192.168.1.202:6379> info

redis_version:2.4.10
....

编辑配置redis-out.conf配置文件，把标准输入的数据存储到redis中
# vim /etc/logstash/conf.d/redis-out.conf

添加以下内容

input {
stdin {}
}

output {

redis {
host => "192.168.1.202"
port => "6379"
password => 'test'
db => '1'
data_type => "list"
key => 'elk-test'
}
}

运行logstash指定redis-out.conf的配置文件
# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/redis-out.conf

运行成功之后，在logstash中输入内容（查看下效果）

编辑配置redis-in.conf配置文件，把reids的存储的数据输出到elasticsearch中
# vim /etc/logstash/conf.d/redis-out.conf

添加以下内容
input{
redis {
host => "192.168.1.202"
port => "6379"
password => 'test'
db => '1'
data_type => "list"
key => 'elk-test'
batch_count => 1 #这个值是指从队列中读取数据时，一次性取出多少条，默认125条（若是redis中没有125条，就会报错，因此在测试期间加上这个值）
}

}

output {
elasticsearch {
hosts => ['192.168.1.202:9200']
index => 'redis-test-%{+YYYY.MM.dd}'
}
}

运行logstash指定redis-in.conf的配置文件
# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/redis-out.conf

把以前的配置文件修改一下，变成全部的日志监控的来源文件都存放到redis中，而后经过redis在输出到elasticsearch中

更改成以下，编辑full.conf
input {
file {
path => "/var/log/httpd/access_log"
type => "http"
start_position => "beginning"
}

file {
path => "/usr/local/nginx/logs/elk.access.log"
type => "nginx"
start_position => "beginning"
}

file {
path => "/var/log/secure"
type => "secure"
start_position => "beginning"
}

file {
path => "/var/log/messages"
type => "system"
start_position => "beginning"
}
}

output {
if [type] == "http" {
redis {
host => "192.168.1.202"
password => 'test'
port => "6379"
db => "6"
data_type => "list"
key => 'nagios_http'
}
}

if [type] == "nginx" {
redis {
host => "192.168.1.202"
password => 'test'
port => "6379"
db => "6"
data_type => "list"
key => 'nagios_nginx'
}
}

if [type] == "secure" {
redis {
host => "192.168.1.202"
password => 'test'
port => "6379"
db => "6"
data_type => "list"
key => 'nagios_secure'
}
}

if [type] == "system" {
redis {
host => "192.168.1.202"
password => 'test'
port => "6379"
db => "6"
data_type => "list"
key => 'nagios_system'
}
}
}

运行logstash指定shipper.conf的配置文件
# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/full.conf

在redis中查看是否已经将数据写到里面(有时候输入的日志文件不产生日志，会致使redis里面也没有写入日志)

把redis中的数据读取出来，写入到elasticsearch中(须要另一台主机作实验)

编辑配置文件
# vim /etc/logstash/conf.d/redis-out.conf

添加以下内容
input {
redis {
type => "system"
host => "192.168.1.202"
password => 'test'
port => "6379"
db => "6"
data_type => "list"
key => 'nagios_system'
batch_count => 1
}

redis {
type => "http"
host => "192.168.1.202"
password => 'test'
port => "6379"
db => "6"
data_type => "list"
key => 'nagios_http'
batch_count => 1
}

redis {
type => "nginx"
host => "192.168.1.202"
password => 'test'
port => "6379"
db => "6"
data_type => "list"
key => 'nagios_nginx'
batch_count => 1
}

redis {
type => "secure"
host => "192.168.1.202"
password => 'test'
port => "6379"
db => "6"
data_type => "list"
key => 'nagios_secure'
batch_count => 1
}
}

output {

if [type] == "system" {
elasticsearch {
hosts => ["192.168.1.202:9200"]
index => "nagios-system-%{+YYYY.MM.dd}"
}
}

if [type] == "http" {
elasticsearch {
hosts => ["192.168.1.202:9200"]
index => "nagios-http-%{+YYYY.MM.dd}"
}
}

if [type] == "nginx" {
elasticsearch {
hosts => ["192.168.1.202:9200"]
index => "nagios-nginx-%{+YYYY.MM.dd}"
}
}

if [type] == "secure" {
elasticsearch {
hosts => ["192.168.1.202:9200"]
index => "nagios-secure-%{+YYYY.MM.dd}"
}
}
}

注意:
input是从客户端收集的
output是一样也保存到192.168.1.202中的elasticsearch中，若是要保存到当前的主机上，能够把output中的hosts修改为localhost，若是还须要在kibana中显示，须要在本机上部署kabana，为什么要这样作，起到一个松耦合的目的
说白了，就是在客户端收集日志，写到服务端的redis里或是本地的redis里面，输出的时候对接ES服务器便可

运行命令看看效果
# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/redis-out.conf

效果是和直接往ES服务器输出同样的（这样是先将日志存到redis数据库，而后再从redis数据库里取出日志）

上线ELK

１.　日志分类
系统日志 rsyslog logstash syslog插件
访问日志 nginx logstash codec json
错误日志 file   logstash mulitline
运行日志 file   logstash codec json
设备日志 syslog logstash syslog插件
Debug日志 file   logstash json 或者 mulitline

2. 日志标准化
路径固定
格式尽可能json

3. 系统个日志开始-->错误日志-->运行日志-->访问日志

由于ES保存日志是永久保存，因此须要按期删除一下日志，下面命令为删除指定时间前的日志

curl -X DELETE http://xx.xx.com:9200/logstash-*-`date +%Y-%m-%d -d "-$n days"`

原文来自

http://mp.weixin.qq.com/s?__biz=MzAwNTM5Njk3Mw==&mid=2247484489&idx=1&sn=5dddb82dec8c544b7b64c0b9488c9132&chksm=9b1c04cbac6b8ddde7007d74a605273148fac3d7997c98c59d60855aa78d10149302b555adc5&scene=0#rd

最后再加上安装使用过程当中的问题及解决方法:

问题

1.memory locking requested for elasticsearch process but memory is not locked

[1]: memory locking requested for elasticsearch process but memory is not locked

[2]: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

[2018-04-16T16:50:25,427][INFO ][o.e.n.Node ] [elk-1] stopping ...

[2018-04-16T16:50:25,457][INFO ][o.e.n.Node ] [elk-1] stopped

[2018-04-16T16:50:25,457][INFO ][o.e.n.Node ] [elk-1] closing ...

[2018-04-16T16:50:25,481][INFO ][o.e.n.Node ] [elk-1] closed

若是你遇到上面的错误，说明你还须要配置/etc/security/limits.conf
增长下面行到文件末尾.*表示全部用户
* soft memlock unlimited
* hard memlock unlimited

2.system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

解决：
Centos6不支持SecComp，而ES5.2.0默认bootstrap.system_call_filter为true

禁用：在elasticsearch.yml中配置bootstrap.system_call_filter为false，注意要在Memory下面:
bootstrap.memory_lock: true
bootstrap.system_call_filter: false

2.1无法分配内存

[2018-04-16T16:50:02,348][WARN ][o.e.b.JNANatives ] Unable to lock JVM Memory: error=12, reason=没法分配内存

[2018-04-16T16:50:02,348][WARN ][o.e.b.JNANatives ] This can result in part of the JVM being swapped out.

[2018-04-16T16:50:02,348][WARN ][o.e.b.JNANatives ] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536

[2018-04-16T16:50:02,349][WARN ][o.e.b.JNANatives ] These can be adjusted by modifying /etc/security/limits.conf, for example:

# allow user 'elasticsearch' mlockall

elasticsearch soft memlock unlimited

elasticsearch hard memlock unlimited

Unable to lock JVM Memory: error=12, reason=没法分配内存

解决方案：

vim /etc/security/limits.conf //添加

* soft memlock unlimited

* hard memlock unlimited

3.max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]

解决方案：

vim /etc/sysctl.conf //添加

fs.file-max = 1645037

vm.max_map_count=655360

4.max number of threads [1024] for user [es] likely too low, increase to at least [2048]

缘由：没法建立本地线程问题,用户最大可建立线程数过小

解决方案：切换到root用户，进入limits.d目录下，修改90-nproc.conf 配置文件。

vi /etc/security/limits.d/90-nproc.conf

找到以下内容：

* soft nproc 1024

#修改成

* soft nproc 2048

5.max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]

缘由：最大虚拟内存过小

解决方案：切换到root用户下，修改配置文件sysctl.conf

vi /etc/sysctl.conf

添加下面配置：

vm.max_map_count=655360

并执行命令：

sysctl -p

而后从新启动elasticsearch，便可启动成功。

6.ElasticSearch启动找不到主机或路由

缘由：ElasticSearch 单播配置有问题

解决方案：

检查ElasticSearch中的配置文件

vi config/elasticsearch.yml

找到以下配置：

discovery.zen.ping.unicast.hosts:[“192.168.**.**:9300″,”192.168.**.**:9300”]

通常状况下，是这里配置有问题，注意书写格式

7.org.elasticsearch.transport.RemoteTransportException: Failed to deserialize exception response from stream

缘由:ElasticSearch节点之间的jdk版本不一致

解决方案：ElasticSearch集群统一jdk环境

8.Unsupported major.minor version 52.0

缘由：jdk版本问题过低

解决方案：更换jdk版本，ElasticSearch5.0.0支持jdk1.8.0

9.bin/elasticsearch-plugin install license

ERROR: Unknown plugin license

缘由：ElasticSearch5.0.0之后插件命令已经改变

解决方案：使用最新命令安装全部插件
bin/elasticsearch-plugin install x-pack

基本全部新安装elk的朋友都遇到过相似问题，这里从网上搜索了资料，汇总的很是不错，这里记录下。原文来自http://www.dajiangtai.com/community/18136.do?origin=csdn-geek&dt=1214。特此说明。

10.启动 elasticsearch 如出现异常 can not run elasticsearch as root

解决方法：建立ES 帐户，修改文件夹文件所属用户组

11.启动异常：ERROR: bootstrap checks failed
system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

问题缘由：由于Centos6不支持SecComp，而ES5.2.1默认bootstrap.system_call_filter为true进行检测，因此致使检测失败，失败后直接致使ES不能启动。详见：https://github.com/elastic/elasticsearch/issues/22899

解决方法：在elasticsearch.yml中配置bootstrap.system_call_filter为false，注意要在Memory下面:
bootstrap.memory_lock: false
bootstrap.system_call_filter: false

12.启动后，若是只有本地能够访问，尝试修改配置文件 elasticsearch.yml
中network.host(注意配置文件格式不是以 # 开头的要空一格，：后要空一格)
为 network.host: 0.0.0.0
默认端口是 9200
注意：关闭防火墙或者开放9200端口

13.ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
max number of threads [1024] for user [lishang] likely too low, increase to at least [2048]

解决方法：切换到root用户，编辑limits.conf 添加相似以下内容

vi /etc/security/limits.conf
添加以下内容:
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096

14.max number of threads [1024] for user [lish] likely too low, increase to at least [2048]

解决：切换到root用户，进入limits.d目录下修改配置文件。
vi /etc/security/limits.d/90-nproc.conf
修改以下内容：
* soft nproc 1024
#修改成
* soft nproc 2048

15.max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]
解决：切换到root用户修改配置sysctl.conf
vi /etc/sysctl.conf
添加下面配置：
vm.max_map_count=655360
并执行命令：
sysctl -p
而后，从新启动elasticsearch，便可启动成功

16.安装npm报错了

npm ERR! Error: CERT_UNTRUSTED

SSH 使用错误,其实咱们关掉HTTPS就行了

npm config set strict-ssl fals

或者

npm config set registry=”http://registry.npmjs.org/”

我用第一种方法就行了,第二个方法我还没试

npm http 304 https://registry.npmjs.org/core-util-is/1.0.2

18:

> phantomjs-prebuilt@2.1.16 install /data/package/elasticsearch-head/node_modules/grunt-contrib-jasmine/node_modules/grunt-lib-phantomjs/node_modules/phantomjs-prebuilt

> node install.js

npm http 304 https://registry.npmjs.org/core-util-is/1.0.2

> phantomjs-prebuilt@2.1.16 install /data/package/elasticsearch-head/node_modules/grunt-contrib-jasmine/node_modules/grunt-lib-phantomjs/node_modules/phantomjs-prebuilt

> node install.js

/data/package/elasticsearch-head/node_modules/grunt-contrib-jasmine/node_modules/grunt-lib-phantomjs/node_modules/phantomjs-prebuilt/node_modules/request/node_modules/hawk/node_modules/boom/lib/index.js:5

const Hoek = require('hoek');

^^^^^

SyntaxError: Use of const in strict mode.

at Module._compile (module.js:439:25)

at Object.Module._extensions..js (module.js:474:10)

at Module.load (module.js:356:32)

at Function.Module._load (module.js:312:12)

at Module.require (module.js:364:17)

at require (module.js:380:17)

at Object.<anonymous> (/data/package/elasticsearch-head/node_modules/grunt-contrib-jasmine/node_modules/grunt-lib-phantomjs/node_modules/phantomjs-prebuilt/node_modules/request/node_modules/hawk/lib/index.js:5:33)

at Module._compile (module.js:456:26)

at Object.Module._extensions..js (module.js:474:10)

at Module.load (module.js:356:32)

npm ERR! weird error 8

npm ERR! not ok code 0

SyntaxError: Use of const in strict mode.

在网上找了一篇帖子,试了一下,能够了

1) Clear NPM's cache:

sudo npm cache clean -f

2) Install a little helper called 'n'

sudo npm install -g n

3) Install latest stable NodeJS version

sudo n stable

Update nodejs instructions taken from, SyntaxError: Use of const in strict mode

我虚拟机重启了,npm start就运行不起来了,一些常见的办法都启动不了

Logstash报错

查看下报错日志找到了下面这条

Cannot create pipeline {:reason=>"Expected one of #, input, filter, output at line 1, column 1 (byte 1) after "

这样是你的conf配置有问题,好好地检查一下,个人问题是IP配置错了