logstash安装以及简单读取文件功能

Logstash 是开源的服务器端数据处理管道，能够同时从多个来源采集数据，转换数据，然后将数据发送到您最喜欢的“存储库”中。

上传logstash并解压缩
tar -zxvf logstash-7.3.1.tar.gz

编辑配置文件
vi test.conf

input {
  #标准输入
  stdin {
    #通用选项，用于输入数据的编解码器
    codec => plain{ charset => "UTF-8" }
  }
  #从文件读取数据
  file {
    type => "json"
    path => "/data/logstash/logdata/test.log"
    #选择logstash开始读取文件的位置，begining或者end
    start_position => "beginning"
    codec => plain{ charset => "UTF-8" }
  }
}
filter{
  #解析json
  json {
        source => "message"
    }
  #将timestamp转为时间类型，并重新定义一个createdate字段来存储
  date {
    match => [ "timestamp", "UNIX_MS" ]
    target => "createdate"
  }
}
output {
  #将事件存储到es中
  elasticsearch { hosts => ["192.168.98.128:9200"] }
  #标准输出，将事件输出到屏幕上
  stdout { codec => rubydebug }
}

测试数据文件如下
vi test.log

{"name": "zhangsan", "age": 21, "addr": "北京", "country": "中国", "timestamp": "1532933162361", "salary": 10000}
{"name": "tom", "age":20,"addr":"纽约", "country": "美国", "timestamp": "1532933163361", "salary": 12000}
{"name": "wangwu", "age":19,"addr":"上海", "country": "中国", "timestamp": "1532933163361", "salary": 11000}
{"name": "dabai", "age":19,"addr":"上海", "country": "中国", "timestamp": "1532933162361", "salary": 7000}
{"name": "xiaoming", "age":19,"addr":"北京", "country": "中国", "timestamp": "1532933165361", "salary": 15000}
{"name": "xiaohong", "age": 21, "addr": "北京", "country": "中国", "timestamp": "1532933162361", "salary": 9000}
{"name": "jack", "age":20,"addr":"纽约", "country": "美国", "timestamp": "1532933162361", "salary": 14000}

测试配置文件是否正确
./bin/logstash -t -f config/test.conf

启动logstash 并读取log文件，存入es

./bin/logstash -f config/test.conf

控制台输入hello 世界回车，自动存入es

遇到的问题：

重启logstash时，要想重新读取日志文件。
需要手工删除logstash-7.3.1/data/plugins/inputs目录下的file目录