日志可视化之ELKF--logstash

上篇介绍了经过filebeat将日志直接写入到elasticsearch中去。但大多时候咱们须要对日志的不一样字段拆分后写入elasticsearch中，方便查询和统计，这就须要用到logstash了。

安装logstash（centos）

这里介绍经过yum方式安装：

1. 执行

   sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

2. 在/etc/yum.repos.d/目录下添加elastic.repo文件，内容以下：

   [logstash-7.x]
   name=Elastic repository for 7.x packages
   baseurl=https://artifacts.elastic.co/packages/7.x/yum
   gpgcheck=1
   gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
   enabled=1
   autorefresh=1
   type=rpm-md

3. 执行

   sudo yum install logstash

配置logstash

1. 在/etc/logstash/conf.d下建立规则文件xxx.conf,推荐一个grok规则验证site
2. 在xxx.conf写入：

   input { 
   	beats { 
   		port => 5044
   	}
   }
   
   filter { 
   	grok { 
           match => {  "message" => "%{TIMESTAMP_ISO8601:logtime} \[%{NOTSPACE:threadname}\] %{LOGLEVEL:loglevel} %{DATA:javamethod} - %{JAVALOGMESSAGE:logcontent}" }
       }
       date { 
       	match => ["logtime", "ISO8601"]
       	target => "@timestamp"
       }
   	mutate { 
   		add_field => {  "[@metadata][target_index]" => "fb-%{filetype}-%{+YYYY.MM.dd}" }
   		remove_field => ["logtime", "message", "tags"]
   	}
   }
   
   output { 
   	elasticsearch { 
   		hosts => ["127.0.0.1:9200"]
   		index => "%{[@metadata][target_index]}"
   		user => "elastic"
   		password => "search"
   	}
   }

grok对message的解析对应的日志格式以下：

%d [%t] %-5level %logger{ 36}.%M\(%file:%line\) - %msg%n

1. 重启logstash服务

   service logstash restart

=>logstash doc<=