FLume安装、配置、使用
发布时间
阅读量：
阅读量
简介
Flume是一种分布式的、可靠的、可用的服务，用于有效地收集、聚合和移动大量日志数据。它具有基于流数据流的简单而灵活的体系结构。它具有健壮性和容错性，具有可调可靠性机制和许多故障转移和恢复机制。它使用一个简单的可扩展数据模型，支持在线分析应用程序
下载、安装
-
地址:(flume-ng-1.6.0-cdh5.7.0)
http://archive.cloudera.com/cdh5/cdh/5/flume-ng-1.6.0-cdh5.7.0.tar.gz -
解压:tar -zxvf flume-ng-1.6.0-cdh5.6.1.tar.gz
启动
- 配置conf文件
- 启动服务:
bin/flume-ng agent --conf /conf --conf-file /conf/exec-memory-logger.conf --name a1 -Dflume.root.logger=INFO,console
-
–conf配置文件存放路径
-
–conf-file 配置文件路径
-
–name agent名称
-
报错
me a1
19/01/29 11:15:59 ERROR node.Application: A fatal error occurred while running. Exception follows.
org.apache.commons.cli.ParseException: The specified configuration file does not exist: /conf/exec-memory-logger.conf
at org.apache.flume.node.Application.main(Application.java:319)- 原因: 配置文件要配绝对路径
- bin/flume-ng agent --conf /home/hadoop/app/flume/conf --conf-file /home/hadoop/app/flume/conf/exec-memory-logger.conf --name a1 -Dflume.root.logger=INFO,console
- 再次启动:
2019-01-29 13:34:32,389 (lifecycleSupervisor-1-4) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.register(MonitoredCounterGroup.java:120)] Monitored counter group for type: SOURCE, name: r1: Successfully registered new MBean.
2019-01-29 13:34:32,389 (lifecycleSupervisor-1-4) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:96)] Component type: SOURCE, name: r1 started
2019-01-29 13:34:36,390 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 61 61 2C 62 62 aa,bb }
2019-01-29 13:34:36,391 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F hello }
2019-01-29 13:34:36,393 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F hello }
2019-01-29 13:34:36,393 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F hello }
2019-01-29 13:34:36,394 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F hello }
2019-01-29 13:34:36,395 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F hello }
2019-01-29 13:34:36,396 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F 32 hello2 }
2019-01-29 13:34:36,397 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F 32 33 hello23 }
2019-01-29 13:34:45,815 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F 32 33 hello23 }
2019-01-29 13:34:49,819 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F 32 33 hello23 }
2019-01-29 13:34:49,820 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F 32 33 hello23 }
2019-01-29 13:34:49,820 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F 32 33 hello23 }
2019-01-29 13:34:49,821 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F 32 33 hello23 }
2019-01-29 13:34:53,825 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F 32 33 hello23 }
2019-01-29 13:34:53,826 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F 32 33 hello23 }
^C2019-01-2测试
官方文档:http://archive.cloudera.com/cdh5/cdh/5/flume-ng-1.6.0-cdh5.7.0/FlumeUserGuide.html
全部评论 (0)
还没有任何评论哟~