springboot+redis+ELK（ElasticSearch+Logstash+kibana）

上一篇已经搭建好elk的环境

如今只要搭建一个elk+redis的环境

使用redis是是防止大批量日志的时候logstash没法及时处理

安装redis

    wget http://download.redis.io/releases/redis-3.2.8.tar.gz

解压

    tar -zxvf redis-3.2.8.tar.gz

移动

    mv redis-3.2.8.tar.gz /usr/local

修改redis配置文件

    vim /usr/local/redis-3.2.8.tar.gz/redis.conf

protected-mode：修改成no,不然redis只有本机才能访问

执行make编译Redis：
make MALLOC=libc
注意：make命令执行完成编译后，会在src目录下生成6个可执行文件，分别是redis-server、redis-cli、redis-benchmark、redis-check-aof、redis-check-rdb、redis-sentinel。

启动redis

    ./src/redis-server redis.conf

搭建redis和logstash
（1）编辑logstash的logstash.conf文件，输入端修改成redis

input {
        redis {
                data_type => "list"
                type => "redis-input"
                key => "logstash:redis"
                host => "192.168.42.128"
                port => 6379
                threads => 5
                codec => "json"
        }
}
output {
        elasticsearch {
                hosts => "192.168.212.37:9201"
                index => "logstash-test"
        }
        stdout {
                codec => rubydebug {}
        }
}

host改为本身的redis服务器地址便可。

从新启动logstash

    ./bin/logstash  -f /usr/local/logstash-5.6.3/etc/logstash.conf

下面新建一个springboot demo测试下

pom.xml    

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
   <groupId>com.cwbase</groupId>
   <artifactId>logback-redis-appender</artifactId>
   <version>1.1.5</version>
</dependency>

新建logback.xml

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
        <Target>System.out</Target>
        <encoder>
            <pattern>[%d{HH:mm:ss}][%t][%p][%c]-%m%n</pattern>
        </encoder>
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>info</level>
        </filter>
    </appender>
    <appender name="LOGSTASH" class="com.cwbase.logback.RedisAppender">
        <source>logstashdemo</source>
        <type>dev</type>
        <host>192.168.42.128</host>
        <key>logstash:redis</key>
        <tags>dev</tags>
        <mdc>true</mdc>
        <location>true</location>
        <callerStackIndex>0</callerStackIndex>
    </appender>

    <root level="info">
        <appender-ref ref="Console"/>
        <appender-ref ref="LOGSTASH"/>
    </root>
</configuration>

DemoApplication:

@SpringBootApplication
@RestController
public class DemoApplication {
    private static final Logger logger = Logger.getLogger(DemoApplication.class);

    @GetMapping("/")
    public String getTestString() {
        logger.info("哈哈哈哈哈哈哈哈哈哈哈哈哈");
        logger.error("错错错错错错错错错错错错错错错错");
        return "this is my first log";
    }

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }
}

运行程序 localhost:8080

控制台

logstash控制台

浏览器打开kibana  http://192.168.42.128:5601

自行进行条件筛选和搜索